Podcasts about distributed systems

In this episode of Flying High with Flutter, we're joined by Dominik Tornow, principal engineer and author of Thinking in Distributed Systems. Dominik shares his journey into distributed systems, breaks down complex concepts like the CAP theorem, liveness vs. safety, and item potency, and offers practical tips for building reliable and scalable systems.On the show:

Eiso Kant, CTO of poolside AI, discusses the company's approach to building frontier AI foundation models, particularly focused on software development. Their unique strategy is reinforcement learning from code execution feedback which is an important axis for scaling AI capabilities beyond just increasing model size or data volume. Kant predicts human-level AI in knowledge work could be achieved within 18-36 months, outlining poolside's vision to dramatically increase software development productivity and accessibility. SPONSOR MESSAGES:***Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/***Eiso Kant:https://x.com/eisokanthttps://poolside.ai/TRANSCRIPT:https://www.dropbox.com/scl/fi/szepl6taqziyqie9wgmk9/poolside.pdf?rlkey=iqar7dcwshyrpeoz0xa76k422&dl=0TOC:1. Foundation Models and AI Strategy [00:00:00] 1.1 Foundation Models and Timeline Predictions for AI Development [00:02:55] 1.2 Poolside AI's Corporate History and Strategic Vision [00:06:48] 1.3 Foundation Models vs Enterprise Customization Trade-offs2. Reinforcement Learning and Model Economics [00:15:42] 2.1 Reinforcement Learning and Code Execution Feedback Approaches [00:22:06] 2.2 Model Economics and Experimental Optimization3. Enterprise AI Implementation [00:25:20] 3.1 Poolside's Enterprise Deployment Strategy and Infrastructure [00:26:00] 3.2 Enterprise-First Business Model and Market Focus [00:27:05] 3.3 Foundation Models and AGI Development Approach [00:29:24] 3.4 DeepSeek Case Study and Infrastructure Requirements4. LLM Architecture and Performance [00:30:15] 4.1 Distributed Training and Hardware Architecture Optimization [00:33:01] 4.2 Model Scaling Strategies and Chinchilla Optimality Trade-offs [00:36:04] 4.3 Emergent Reasoning and Model Architecture Comparisons [00:43:26] 4.4 Balancing Creativity and Determinism in AI Models [00:50:01] 4.5 AI-Assisted Software Development Evolution5. AI Systems Engineering and Scalability [00:58:31] 5.1 Enterprise AI Productivity and Implementation Challenges [00:58:40] 5.2 Low-Code Solutions and Enterprise Hiring Trends [01:01:25] 5.3 Distributed Systems and Engineering Complexity [01:01:50] 5.4 GenAI Architecture and Scalability Patterns [01:01:55] 5.5 Scaling Limitations and Architectural Patterns in AI Code Generation6. AI Safety and Future Capabilities [01:06:23] 6.1 Semantic Understanding and Language Model Reasoning Approaches [01:12:42] 6.2 Model Interpretability and Safety Considerations in AI Systems [01:16:27] 6.3 AI vs Human Capabilities in Software Development [01:33:45] 6.4 Enterprise Deployment and Security ArchitectureCORE REFS (see shownotes for URLs/more refs):[00:15:45] Research demonstrating how training on model-generated content leads to distribution collapse in AI models, Ilia Shumailov et al. (Key finding on synthetic data risk)[00:20:05] Foundational paper introducing Word2Vec for computing word vector representations, Tomas Mikolov et al. (Seminal NLP technique)[00:22:15] OpenAI O3 model's breakthrough performance on ARC Prize Challenge, OpenAI (Significant AI reasoning benchmark achievement)[00:22:40] Seminal paper proposing a formal definition of intelligence as skill-acquisition efficiency, François Chollet (Influential AI definition/philosophy)[00:30:30] Technical documentation of DeepSeek's V3 model architecture and capabilities, DeepSeek AI (Details on a major new model)[00:34:30] Foundational paper establishing optimal scaling laws for LLM training, Jordan Hoffmann et al. (Key paper on LLM scaling)[00:45:45] Seminal essay arguing that scaling computation consistently trumps human-engineered solutions in AI, Richard S. Sutton (Influential "Bitter Lesson" perspective)

(05:29) Brought to you by Swimm.io⁠⁠⁠⁠Start modernizing your mainframe faster with Swimm.Understand the what, why, and how of your mainframe code.Use AI to uncover critical code insights for seamless migration, refactoring, or system replacement.Tired of API dependencies slowing down your development and testing?Dive into my conversation with Tom Akehurst, creator of WireMock, and discover the art of using API mocking to build successful software in complex distributed environments.Key topics discussed:The origin story of WireMock, born from integration challenges at DisneyHow WireMock became a leading API mocking tool with millions of monthly downloadsInsights on building and maintaining successful open-source projectsThe key benefits of API mocking for developer productivity and experienceThe shift from the traditional testing pyramid to a “testing trophy” approachLeveraging API mocking for API-first design and rapid prototypingThe distinction between API mocking and contract testingThe future of API testing and development in the age of microservices and AIWhether you're a seasoned developer or just starting out your journey in API development, this episode provides valuable insights into the power of API mocking and the journey of building a successful open-source project.  Timestamps:(02:11) Career Turning Points(08:08) WireMock OSS Success Story(15:15) Welcoming & Aligning with Contributors(18:05) Benefits of WireMock & API Mocking Tools(19:59) API Mocking & Testing Pyramid(22:05) API Mocking vs Contract Testing(25:25) The Economics of API Mocking(27:27) API First Design(32:32) Impact to the Developer Experience & Productivity(35:32) Working More Effectively with Distributed Systems(38:15) API Virtualization/Simulation(41:13) AI Advancement in API Development(44:25) Building API for AI Agents(47:25) 3 Tech Lead Wisdom_____Tom Akehurst's BioTom Akehurst is the creator of WireMock, the open source API mocking tool, which he's now been working on for well over a decade. Lately he's also the CTO and co-founder of WireMock, Inc., where he's helping complex engineering organisations effectively adopt API simulation techniques in order to build better software faster.Tom has been developing software for over 20 years. He's built large-scale web systems for media, travel, hospitality, retail and government, applying lean, eXtreme Programming, Continuous Delivery and DevOps principles along the way.Follow Tom:LinkedIn – linkedin.com/in/tomakehurstEmail – tom@wiremock.orgWireMock – wiremock.org_____Our SponsorsEnjoy an exceptional developer experience with JetBrains. Whatever programming language and technology you use, JetBrains IDEs provide the tools you need to go beyond simple code editing and excel as a developer.Check out FREE coding software options and special offers on jetbrains.com/store/#discounts.Make it happen. With code.Manning Publications is a premier publisher of technical books on computer and software development topics for both experienced developers and new learners alike. Manning prides itself on being independently owned and operated, and for paving the way for innovative initiatives, such as early access book content and protection-free PDF formats that are now industry standard.Get a 40% discount for Tech Lead Journal listeners by using the code techlead24 for all products in all formats.Like this episode?Show notes & transcript:techleadjournal.dev/episodes/210.Follow @techleadjournal onLinkedIn,Twitter, andInstagram.Buy me acoffee or become apatron.

Lex interviews Sam Williams - founder of Arweave. This episode delves into the innovative aspects of Arweave, a protocol designed for permanent data storage and computation within the Web3 ecosystem. The discussion covers a range of topics, from the economic models underpinning Arweave to its potential applications in decentralized finance (DeFi) and beyond. Notable discussion points: The Founding of Arweave and its Mission – Sam Williams' interest in distributed computing and concerns about authoritarianism led him to create Arweave in 2017. Inspired by the Snowden leaks, he saw the need for a blockchain-based permanent storage solution to protect journalism, historical records, and digital assets from censorship. Decentralized vs. Distributed Storage – Williams explained how Arweave differs from alternatives like IPFS and Filecoin. Unlike traditional storage, which requires ongoing payments, Arweave uses a one-time payment model. This storage endowment leverages declining storage costs to ensure long-term data persistence without relying on centralized infrastructure. Arweave's Expansion into Decentralized Compute – Arweave has evolved beyond storage to develop decentralized computing through "Arweave IO." This enables parallelized smart contract execution, making it possible to run AI models, financial automation, and decentralized apps on-chain—aligning with Web3's shift toward autonomous, intelligent systems.MENTIONED IN THE CONVERSATION Topics: Arweave, permanent data storage, Web3, decentralized systems, distributed systems, blockchain, economic models, IPFS, Filecoin, decentralized computing, decentralized finance, compute ABOUT THE FINTECH BLUEPRINT 

The GeekNarrator memberships can be joined here: https://www.youtube.com/channel/UC_mGuY4g0mggeUGM6V1osdA/joinMembership will get you access to member only videos, exclusive notes and monthly 1:1 with me. Here you can see all the member only videos: https://www.youtube.com/playlist?list=UUMO_mGuY4g0mggeUGM6V1osdA------------------------------------------------------------------------------------------------------------------------------------------------------------------About this episode: ------------------------------------------------------------------------------------------------------------------------------------------------------------------In this conversation, Unmesh Joshi discusses the patterns of distributed systems. He emphasizes the importance of understanding the context in which patterns are applied, the need to read code to grasp their implementation, and the common pitfalls that developers face when applying patterns without a clear understanding of the underlying problems. Chapters00:00 Introduction to Distributed Systems and Patterns05:39 Understanding Patterns in Distributed Systems19:23 Bridging Theory and Practice in Distributed Systems28:56 The Role of Developers in Understanding Patterns31:58 Understanding Patterns in Software Development40:58 The Human Aspect of Software Design44:37 Iterative Development and Real-World Applications49:03 The Future of Patterns in Cloud-Native Systems55:07 Common Misunderstandings of Distributed PatternsInteresting quotes:"Patterns capture wisdom of generations.""Reading code is the best way to understand.""Patterns help you see beyond abstractions.""Understanding patterns helps bridge the gap.""Expert generalists can operate across verticals.""There are no simple systems in the cloud era.""Patterns can add complexity if misunderstood.""Patterns are always useful within a context.""Design and development are human activities.""The deconstruction of databases is happening.""Paxos is the most misunderstood pattern."Unmesh Joshi :https://in.linkedin.com/in/unmesh-joshi-9487635Catalog of Patterns: https://martinfowler.com/articles/patterns-of-distributed-systems/I hope you liked the episode, if you did please like, share and subscribe. ------------------------------------------------------------------------------------------------------------------------------------------------------------------Like building real stuff?------------------------------------------------------------------------------------------------------------------------------------------------------------------Try out CodeCrafters and build amazing real world systems like Redis, Kafka, Sqlite. Use the link below to signup and get 40% off on paid subscription.https://app.codecrafters.io/join?via=geeknarrator------------------------------------------------------------------------------------------------------------------------------------------------------------------Link to other playlists. LIKE, SHARE and SUBSCRIBE------------------------------------------------------------------------------------------------------------------------------------------------------------------If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet.Database internals series: https://youtu.be/yV_Zp0Mi3xsPopular playlists:Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_dModern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsNStay Curios! Keep Learning!#distributedsystems #patterns #softwarearchitecture #consensus #algorithms #coding #patterns #softwaredevelopment #ThoughtWorks #softwareengineering #cloud #computing #software

Asanka Abeysinghe, CTO at WSO2, joins host Giovanni Asproni to discuss cell-based architecture -- a style that's intended to combine application, deployment, and team architecture to help organizations respond quickly to changes in the business environment, customer requirements, or enterprise strategy. Cell-based architecture is aimed at creating scalable, modular, composable systems with effective governance mechanisms. The conversation starts by introducing the context and some vocabulary before exploring details about the main elements of the architecture and how they fit together. Finally, Asanka offers some advice on how to implement a cell-based architecture in practice. Brought to you by IEEE Computer Society and IEEE Software magazine. Related Episodes SE Radio 396: Barry O'Reilly on Antifragile Architecture SE Radio 331: Kevin Goldsmith on Architecture and Organizational Design SE Radio 263: Camille Fournier on Real-World Distributed Systems SE Radio 236: Rebecca Parsons on Evolutionary Architecture SE Radio 213: James Lewis on Microservices SE Radio 210: Stefan Tilkov on Architecture and Micro Services SE Radio 203: Leslie Lamport on Distributed Systems

CTO Series: Navigating Growth, A Playbook for Scaling Engineering Teams With Toni Ala-Piirto   In this BONUS episode, we dive into the journey of Toni Ala-Piirto, a seasoned software leader with 15 years of experience designing and implementing distributed systems. Toni opens up about pivotal lessons from his leadership career, balancing tech strategy with business goals, and the nuances of scaling engineering teams during rapid growth. Whether you're a CTO, a team lead, or a tech enthusiast, this conversation is packed with practical insights.   The Evolution of a Leader: A Journey, Not a Single Moment   “Leadership isn't built in a single defining moment but shaped by many experiences.”   Toni recounts a key challenge early in his career involving a major performance issue for a customer. This experience taught him the importance of viewing systems holistically rather than focusing solely on individual contributions. His “boring” leadership style—marked by forward-thinking and crisis prevention—emphasizes preparation and identifying risks without over-engineering solutions.   Aligning Tech and Business: The Power of Collaboration   “Technology vision and business strategy should speak the same language.”   Toni highlights the importance of close collaboration with product managers, sales, and finance to ensure tech strategy aligns with broader business goals. Regular cross-department discussions foster trust and ensure that the product roadmap is both innovative and achievable.   Key Practice: Build relationships with key stakeholders through daily touchpoints to create alignment.   The Roadmap to Success: Vision vs. Execution   “Short-term details drive long-term visions.”   Toni explains their approach to roadmapping, with detailed 6-month plans that address “how” to achieve goals and a broader vision for the longer term. This allows the team to stay agile while keeping future innovations in view.   Pro Tip: Avoid spending excessive time on estimations; use past experience to guide epic-level planning.   “The first six months are about execution—the rest is about imagining what's possible.”   Scaling Teams During Rapid Growth   “The true challenge of scaling is transferring knowledge while preserving team culture.”   Toni reflects on the growth journey from a small team to a larger organization. As the team grew, onboarding and knowledge transfer became crucial. His solution? Pair testing and collaborative learning to help developers understand the product deeply, not just the code.   Tactical Tips: Implement a “test buddy” system for collaborative testing and learning. Encourage developers to test the product to build domain knowledge and foster cross-functional understanding.   “Your people need to understand the product—not just the code—to scale effectively.”   Maintaining Culture Amid Growth   “Growth changes culture—how you hire and lead defines the next chapter.”   Toni shares how adding new team members can shift team dynamics. The key to sustaining a positive culture is hiring individuals who take ownership and serve as role models. Leaders should seek out those who aim to improve the team, not just perform their tasks.   “The best hires don't just do their job—they make the whole team better.”   Cross-Functional Insights and Learning the CTO Role   “CTOs operate at the intersection of tech and business—a shift from pure development.”   Toni admits that stepping into the CTO role required him to expand his understanding of business operations, strategic planning, and cross-functional collaboration. He emphasizes that this broadened perspective is essential for impactful decision-making.   “The biggest shift for me was seeing the business as a whole—not just the tech stack.”   Key Influences: The Five Dysfunctions of a Team   “Understanding team dynamics is as crucial as technical expertise.”   Toni cites Patrick Lencioni's The Five Dysfunctions of a Team as a pivotal read. The book shaped his approach to fostering accountability and ensuring team commitment. Toni underscores that accountability isn't about blame—it's about ownership and follow-through.   Scaling with a Talent Strategy in Mind   “Growth requires not just more people but the right investments.”   Toni discusses integrating talent strategy into roadmaps by aligning with business goals, including company size and revenue targets. Strategic hiring and investment in growth ensure that the team remains equipped to deliver on future plans.   About Toni Ala-Piirto   Toni Ala-Piirto is a seasoned software professional with 15 years of experience leading architecture and design for projects of all sizes. He excels in creating practical, fit-for-purpose distributed systems and is known for his hands-on approach and commitment to continuous improvement. Toni consistently delivers solutions that meet specific project needs while aligning with broader business objectives. You can link with Toni Ala-Piirto on LinkedIn.a

This interview was recorded for GOTO Unscripted.https://gotopia.techRead the full transcription of this interview hereSam Newman - Microservices Expert & Author of "Monolith to Microservices" & "Building Resilient Distributed Systems"Julian Wood - Serverless Developer Advocate at AWSRESOURCESSamhttps://twitter.com/samnewmanhttps://hachyderm.io/@samnewmanhttps://www.linkedin.com/in/samnewmanhttp://samnewman.iohttp://samnewman.io/bloghttps://github.com/snewmanJulianhttps://bsky.app/profile/julianwood.comhttps://twitter.com/julian_woodhttps://www.linkedin.com/in/julianrwoodDESCRIPTIONJulian Wood and Sam Newman delve into the complexities of software architecture, focusing on critical concepts such as microservices, asynchronous communication, and the importance of architectural decision-making.Sam emphasizes the need for clear definitions in computing, particularly around terms like asynchronicity, advocating for an understanding of communication styles—event-driven versus request-response—as pivotal for effective system design. They discuss the value of architectural decision records (ADRs) as tools for fostering collaboration and documenting trade-offs in decision-making.Additionally, the conversation highlights the evolving role of AI in software development, stressing that while AI can assist in understanding and maintaining existing codebases, the human element of critical thinking remains essential in navigating the complexities of modern software systems.RECOMMENDED BOOKSSam Newman • Building Resilient Distributed Systems • https://www.oreilly.com/library/view/building-resilient-distributed/9781098163532Sam Newman • Monolith to Microservices • https://amzn.to/2Nml96ESam Newman • Building Microservices • https://amzn.to/3dMPbOsRonnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• https://amzn.to/3c4HmmLMitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • https://amzn.to/3fVNAb0BlueskyTwitterInstagramLinkedInFacebookCHANNEL MEMBERSHIP BONUSJoin this channel to get early access to videos & other perks:https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/joinLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

Lukas Gentele, CEO of Loft Labs, joins host Robert Blumen for a discussion of kubernetes vclusters (virtual clusters). A vcluster is a kubernetes cluster that runs kubernetes application on a host kubernetes cluster. The conversation covers: vcluster basics; sharing models; what is owned by the vcluster and what is shared with the host; attached nodes versus shared nodes; the primary use case: multi-tenancy vcluster per tenant; alternatives - namespace per tenant, full cluster per tenant; trade-offs - isolation; less resource use; spin up time; scalability; how many clusters and how many vclusters should an org have? Deployment models for vclusters - helm chart with standard resources; vcluster operator; persistent storage models for vclusters; vcluster snapshotting, recovery, and migration. how many vclusters can run on a cluster? ingress, TLS and DNS. Brought to you by IEEE Computer Society and IEEE Software magazine.

For the Season 13 finale, Elixir Wizards Dan and Charles are joined by Spin42 Engineers Marc Lainez, Thibault Poncelet, and Loïc Vigneron to discuss their work retrofitting a 2007 VW Polo and creating an Open Vehicle Control System (OVCS). Using Elixir, Nerves, and Raspberry Pis, the team is reimagining vehicle technology to extend the lifespan of older cars and reduce waste—all while making the process approachable and open source. The Spin42 team shares the technical details behind OVCS and how they use Elixir and Nerves to interact with the CAN bus and build a Vehicle Management System (VMS) to coordinate various vehicle components. They dive into the challenges of reverse engineering CAN messages, designing a distributed architecture with Elixir processes, and ensuring safety with fail-safe modes and emergency shutoffs. Beyond the technical, the team discusses their motivation for the project—upgrading older vehicles with modern features to keep them on the road, building an open-source platform to share their findings with others, and above all-- to just have fun. They explore potential applications for OVCS in boats, construction equipment, and other vehicles, while reflecting on the hurdles of certifying the system for road use. If you've ever wondered how Elixir and Nerves can drive innovation beyond software, this episode is packed with insights into automotive computing, hardware development, and the collaborative potential of open-source projects. Topics Discussed in this Episode: Retrofitting a 2007 VW Polo with electric engines and modern tech Building an open-source Vehicle Control System (OVCS) using Elixir and Nerves Leveraging Elixir to interact with the CAN bus and parse proprietary messages Designing a Vehicle Management System (VMS) to coordinate vehicle components Developing custom hardware for CAN communication Creating a YAML-based DSL for CAN message and frame descriptions Building a distributed architecture using Elixir processes Ensuring safety with fail-safe modes and emergency shutoffs Using Flutter and Nerves to build a custom infotainment system Exploring autonomous driving features with a ROS2 bridge Developing remote control functionality with a Mavlink transmitter Testing OVCS features at scale with a Traxxas RC car (OVCS Mini) Challenges of certifying OVCS for road use and meeting regulatory requirements Encouraging community contributions to expand OVCS functionality Balancing open-source projects with contract work to sustain development The fun and fulfillment of experimenting with Elixir beyond traditional applications Links mentioned: https://www.spin42.com/ https://nerves-project.org/ Quadcopter https://github.com/Spin42/elicopter https://github.com/linux-can/can-utils https://docs.kernel.org/networking/can.html https://github.com/open-vehicle-control-system/cantastic https://github.com/commaai/opendbc https://en.wikipedia.org/wiki/CANbus#CANFD https://comma.ai/ https://en.wikipedia.org/wiki/CANFD https://webkit.org/wpe/ https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/SD/WindowingSystems/WestonWayland.html https://buildroot.org/ https://vuejs.org/ https://flutter.dev/ https://github.com/smartrent/elixirflutterembedder https://www.raspberrypi.com/products/raspberry-pi-5/ The Rabbit Pickup https://www.hemmings.com/stories/value-guide-1980-83-volkswagen-pickup https://www.expresslrs.org/software/mavlink https://industrial-training-master.readthedocs.io/en/melodic/source/session7/ROS1-ROS2-bridge.html https://github.com/ros2/rcl https://github.com/open-vehicle-control-system/traxxas Contact Marc, Thibault, and Loïc: info@spin42.com Special Guests: Loïc Vigneron, Marc Lainez, and Thibault Poncelet.

AJ (Alykhan Jetha), CEO and CTO of Marketcircle, joins the Elixir Wizards to share his experience building and evolving Daylite, their award-winning CRM and business productivity app for Apple users. He details his experiences as a self-taught programmer and how Marketcircle has navigated pivots, challenges, and opportunities since its founding in 1999. AJ explains why they migrated Daylite's backend to Elixir, focusing on their sync engine, which demands high concurrency and fault tolerance. He highlights how Elixir has improved performance, reduced cloud costs, and simplified development with its approachable syntax and productive workflows. The conversation also touches on the technical hurdles of deploying native apps for Apple devices and the potential for integrating new technologies like LiveView Native to streamline cross-platform development. For technical founders, AJ emphasizes the importance of leveraging your strengths (“superpowers”), staying deeply connected to the development process, and finding stability in tools like Elixir amidst a rapidly evolving tech ecosystem. He also shares Marketcircle's roadmap for migrating more customers to Elixir-powered systems and explores the potential for new features in their native apps. Tune in for insights on building resilient systems, navigating technical and business challenges, and how Elixir is shaping Marketcircle's future. Topics discussed in this episode: AJ's journey as a self-taught programmer and entrepreneur Marketcircle's evolution since 1999 and lessons from their pivots Daylite's growth as a flagship product for Apple users Migrating to Elixir for high concurrency and fault tolerance How Elixir improved performance and reduced cloud costs The simplicity of Elixir and its impact on developer onboarding Challenges in managing a growing microservices architecture Insights into deploying native apps for the Apple ecosystem Exploring LiveView Native for future cross-platform development Advice for technical founders: leveraging your superpowers Staying connected to development to maintain system understanding The role of Elixir in improving development efficiency and stability Planning gradual customer migrations to an Elixir-powered backend Potential new features for Daylite's native apps Benefits of collaboration with the Elixir community #ElixirMullet -- native app in the front, Elixir in the back Navigating a rapidly evolving tech ecosystem as a founder Leveraging Elixir to future-proof Marketcircle's systems Balancing technical and business priorities in a startup environment AJ's thoughts on the future of Elixir in powering business tools Links mentioned: https://www.marketcircle.com/ Daylite.app https://www.nextcomputers.org/ https://www.digitalocean.com/ Python Async https://docs.python.org/3/library/asyncio.html https://github.com/sinatra/sinatra https://github.com/dependabot https://kafka.apache.org/ https://www.djangoproject.com/ https://github.com/socketry/falcon https://github.com/puma/puma https://www.swift.org/blog/announcing-swift-6/ https://en.wikipedia.org/wiki/Async/await https://www.ffmpeg.org/ https://www.sqlite.org/ https://github.com/commanded/commanded https://pragprog.com/titles/khpes/real-world-event-sourcing/ https://en.wikipedia.org/wiki/ShipofTheseus https://reactnative.dev/ https://www.electronjs.org/ https://en.wikipedia.org/wiki/WebOS https://www.linkedin.com/in/alykhanjetha/ https://bsky.app/profile/ajetha.bsky.social Special Guest: Alykhan Jetha.

Jonas Hübotter, PhD student at ETH Zurich's Institute for Machine Learning, discusses his groundbreaking research on test-time computation and local learning. He demonstrates how smaller models can outperform larger ones by 30x through strategic test-time computation and introduces a novel paradigm combining inductive and transductive learning approaches. Using Bayesian linear regression as a surrogate model for uncertainty estimation, Jonas explains how models can efficiently adapt to specific tasks without massive pre-training. He draws an analogy to Google Earth's variable resolution system to illustrate dynamic resource allocation based on task complexity. The conversation explores the future of AI architecture, envisioning systems that continuously learn and adapt beyond current monolithic models. Jonas concludes by proposing hybrid deployment strategies combining local and cloud computation, suggesting a future where compute resources are allocated based on task complexity rather than fixed model size. This research represents a significant shift in machine learning, prioritizing intelligent resource allocation and adaptive learning over traditional scaling approaches. SPONSOR MESSAGES: CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ Transcription, references and show notes PDF download: https://www.dropbox.com/scl/fi/cxg80p388snwt6qbp4m52/JonasFinal.pdf?rlkey=glk9mhpzjvesanlc14rtpvk4r&st=6qwi8n3x&dl=0 Jonas Hübotter https://jonhue.github.io/ https://scholar.google.com/citations?user=pxi_RkwAAAAJ Transductive Active Learning: Theory and Applications (NeurIPS 2024) https://arxiv.org/pdf/2402.15898 EFFICIENTLY LEARNING AT TEST-TIME: ACTIVE FINE-TUNING OF LLMS (SIFT) https://arxiv.org/pdf/2410.08020 TOC: 1. Test-Time Computation Fundamentals [00:00:00] Intro [00:03:10] 1.1 Test-Time Computation and Model Performance Comparison [00:05:52] 1.2 Retrieval Augmentation and Machine Teaching Strategies [00:09:40] 1.3 In-Context Learning vs Fine-Tuning Trade-offs 2. System Architecture and Intelligence [00:15:58] 2.1 System Architecture and Intelligence Emergence [00:23:22] 2.2 Active Inference and Constrained Agency in AI [00:29:52] 2.3 Evolution of Local Learning Methods [00:32:05] 2.4 Vapnik's Contributions to Transductive Learning 3. Resource Optimization and Local Learning [00:34:35] 3.1 Computational Resource Allocation in ML Models [00:35:30] 3.2 Historical Context and Traditional ML Optimization [00:37:55] 3.3 Variable Resolution Processing and Active Inference in ML [00:43:01] 3.4 Local Learning and Base Model Capacity Trade-offs [00:48:04] 3.5 Active Learning vs Local Learning Approaches 4. Information Retrieval and Model Interpretability [00:51:08] 4.1 Information Retrieval and Nearest Neighbor Limitations [01:03:07] 4.2 Model Interpretability and Surrogate Models [01:15:03] 4.3 Bayesian Uncertainty Estimation and Surrogate Models 5. Distributed Systems and Deployment [01:23:56] 5.1 Memory Architecture and Controller Systems [01:28:14] 5.2 Evolution from Static to Distributed Learning Systems [01:38:03] 5.3 Transductive Learning and Model Specialization [01:41:58] 5.4 Hybrid Local-Cloud Deployment Strategies

To kick off Elixir Wizards Season 13, The Creator's Lab, we're joined by Zach Daniel, the creator of Igniter and the Ash framework. Zach joins hosts Owen Bickford and Charles Suggs to discuss the mechanics and aspirations of his latest brainchild, Igniter—a code generation and project patching framework designed to revolutionize the Elixir development experience. Igniter isn't just about generating code; it's about generating smarter code. By leveraging tools like Sourcerer and Rewrite, Igniter allows developers to modify source code and batch updates by directly interacting with Elixir's AST instead of regex patching. This approach streamlines new project setup and package installations and enhances overall workflow. They also discuss the strategic implications of Igniter for the broader Elixir community. Zach hopes Igniter will foster a more interconnected and efficient ecosystem that attracts new developers to Elixir and caters to the evolving needs of seasoned Elixir engineers. Topics discussed in this episode: Advanced package installation and code generation improve the developer experience Scripting and staging techniques streamline project updates Innovative methods for smoother installation processes in Elixir packages High-level tools apply direct patches to source code Progressive feature additions simplify the mix phx.new experience Chaining installers and composing tasks for more efficient project setup Continuous improvement in developer experiences to boost Elixir adoption Encourage listeners to collaborate by sharing code generation patterns Introduction of a new mix task aimed at removing the "unless" keyword in preparation for Elixir 1.18 You can learn more in the upcoming book "Building Web Applications with Ash Framework" by Zach and Rebecca Links mentioned: https://smartlogic.io/ https://alembic.com.au/blog/igniter-rethinking-code-generation-with-project-patching https://hexdocs.pm/igniter/readme.html https://github.com/ash-project/igniter https://www.zachdaniel.dev/p/serialization-is-the-secret https://www.zachdaniel.dev/p/welcome-to-my-substack https://ash-hq.org/ https://hexdocs.pm/sourceror/readme.html https://smartlogic.io/podcast/elixir-wizards/s10-e09-hugo-lucas-future-of-elixir-community/ https://github.com/hrzndhrn/rewrite https://github.com/zachdaniel https://github.com/liveshowy/webauthn_components https://hexdocs.pm/elixir/Regex.html https://github.com/msaraiva/vscode-surface https://github.com/swoosh/swoosh https://github.com/erlef/oidcc https://alembic.com.au/ https://www.zachdaniel.dev/ Special Guest: Zach Daniel.

Sriram Panyam, CTO at DagKnows, discusses SaaS Control Planes with SE Radio host Brijesh Ammanath. The discussion starts off with the basics, examining what control planes are and why they're important. Sriram then discusses reasons for building a control plane and the challenges in designing one. They explore design and architectural considerations when building a SaaS control plane, as well as the key differences between a control plane and a data plane. This episode is sponsored by QA Wolf.

In this episode of the Crazy Wisdom podcast, Stewart Alsop speaks with Anand Dwivedi, a Senior Data Scientist at ICE, returning for his second appearance. The conversation covers a range of topics including the evolution of machine learning models, the integration of AI into operating systems, and how innovations like Neuralink may reshape our understanding of human-machine interaction. Anand also touches on the role of cultural feedback in shaping human learning, the implications of distributed systems in cybersecurity, and his current project—training a language model on the teachings of his spiritual guru. For more information, listeners can connect with Anand on LinkedIn.Check out this GPT we trained on the conversation!Timestamps00:00 Introduction and Guest Welcome00:25 Exploring GPT-4 and Machine Learning Innovations03:34 Apple's Integration of AI and Privacy Concerns06:07 Digital Footprints and the Evolution of Memory09:42 Neuralink and the Future of Human Augmentation14:20 Cybersecurity and Financial Crimes in the Digital Age20:53 The Role of LLMs and Human Feedback in AI Training29:50 Freezing Upper Layers and Formative Feedback30:32 Neuroplasticity in Sports and Growth32:00 Challenges of Learning New Skills as Adults32:44 Cultural Immersion and Cooking School34:21 Exploring Genetic Engineering and Neuroplasticity38:53 Neuralink and the Future of AI39:58 Physical vs. Digital World41:20 Existential Threats and Climate Risk45:15 Attention Mechanisms in LLMs48:22 Optimizing Positive Social Impact54:54 Training LLMs on Spiritual LecturesKey InsightsEvolution of Machine Learning Models: Anand Dwivedi highlights the advancement in machine learning, especially with GPT-4's ability to process multimodal inputs like text, images, and voice simultaneously. This contrasts with earlier models that handled each modality separately, signifying a shift towards more holistic AI systems that mirror human sensory processing.AI Integration in Operating Systems: The conversation delves into how AI, like Apple Intelligence, is being integrated directly into operating systems, enabling more intuitive interactions such as device management and on-device tasks. This advancement brings AI closer to daily use, ensuring privacy by processing data locally rather than relying on cloud-based systems.Neuralink and Transhumanism: Anand and Stewart discuss Neuralink's potential to bridge the gap between human and artificial intelligence. Neuralink's brain-computer interface could allow humans to enhance cognitive abilities and better compete in a future dominated by intelligent machines, raising questions about the ethics and risks of such direct brain-AI integration.Cultural Feedback and Learning: Anand emphasizes the role of cultural feedback in shaping human learning, likening it to how AI models are fine-tuned through feedback loops. He explains that different cultural environments provide varied feedback to individuals, influencing the way they process and adapt to information throughout their lives.Cybersecurity and Distributed Systems: The discussion highlights the dual-edged nature of distributed systems in cybersecurity. While these systems offer increased freedom and decentralization, they can also serve as breeding grounds for financial crimes and other malicious activities, pointing to the need for balanced approaches to internet freedom and security.Generative Biology and AI: A key insight from the episode is the potential of AI models, like those used for language processing, to revolutionize fields such as biology and chemistry. Anand mentions the idea of generative biology, where AI could eventually design new proteins or chemical compounds, leading to breakthroughs in drug discovery and personalized medicine.Positive Social Impact Through Technology: Anand introduces a thought-provoking idea about using AI and data analytics for social good. He suggests that technology can help bridge disparities in education and resources globally, with models being designed to measure and optimize for positive social impacts, rather than just profits or efficiency.

This is a recap of the top 10 posts on Hacker News on September 2nd, 2024.This podcast was generated by wondercraft.ai(00:39): The Art of FinishingOriginal post: https://news.ycombinator.com/item?id=41428705&utm_source=wondercraft_ai(01:44): Sleep on it: How the brain processes many experiences, even when 'offline'Original post: https://news.ycombinator.com/item?id=41425563&utm_source=wondercraft_ai(02:52): Greppability is an underrated code metricOriginal post: https://news.ycombinator.com/item?id=41430772&utm_source=wondercraft_ai(04:12): Telephone Line Rural Outside PlantOriginal post: https://news.ycombinator.com/item?id=41423303&utm_source=wondercraft_ai(05:22): Notes on Distributed Systems for Young BloodsOriginal post: https://news.ycombinator.com/item?id=41427185&utm_source=wondercraft_ai(06:37): Upgrading a Toshiba NAS HDD Firmware on LinuxOriginal post: https://news.ycombinator.com/item?id=41423577&utm_source=wondercraft_ai(07:49): DOjS – A DOS JavaScript Canvas with SoundOriginal post: https://news.ycombinator.com/item?id=41425259&utm_source=wondercraft_ai(08:58): Owners of 1-Time Passcode Theft Service Plead GuiltyOriginal post: https://news.ycombinator.com/item?id=41426982&utm_source=wondercraft_ai(10:16): They don't make 'em like that any more: the 3.5mm headphone jack socketOriginal post: https://news.ycombinator.com/item?id=41425383&utm_source=wondercraft_ai(11:41): Microsoft's 'Recall' feature can't be uninstalled after allOriginal post: https://news.ycombinator.com/item?id=41430757&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

In this episode we talk with Fabrizio Montesi, a Full Professor at the University of South Denmark. He is one of the creators of the Jolie Programming Language, President of the Microservices Community and Author of the book 'Introduction to Choreographies'. In today's episode we talk about the formal side of Distributed Sytems, session types, the calculi that model distributed systems, their type systems, their Curry-Howard correspondences, and all the main ideas around these concepts. Links Fabrizio's Website Fabrizio's Linkedin Fabrizio's X / Twitter Fabrizio's Mastodon Fabrizio's Youtube Jolie's Website

In this episode of The Geek Narrator podcast, hosted by Kaivalya Apte, we welcome a special guest, Kishore Gopalakrishna from StarTree, co-author of Apache Pinot and other notable projects. Kishore shares his extensive experience in building real-time analytics and streaming systems, including Apache Pino, Espresso, Apache Helix, and Third Eye. The episode delves into the motivations and challenges behind creating these systems, the innovations they brought to distributed systems, and the impact of community on open-source projects. Kishore also discusses the evolution of testing methodologies, cost optimizations in transactional and analytical systems, and key considerations for companies evaluating real-time analytics solutions. Don't miss this in-depth conversation packed with valuable insights for both seasoned developers and tech enthusiasts! Chapters: 00:00 Introduction 03:13 Building Distributed Systems at LinkedIn 08:57 Testing and Challenges in Distributed Systems 30:50 Advantages of Columnar Storage 33:04 The Importance of Upserts 34:24 Building a Strong Open Source Community 41:10 Challenges and Lessons in System Design 51:35 Real-Time Analytics: Do You Need It? StarTree: https://startree.ai/ Apache Pinot: https://pinot.apache.org/ If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #distributedsystems #kafka #s3 #streaming #realtimeanalytics #database #pinot #startree

In this episode of PodRocket, Joel Hooks, creator of egghead.io, talks about the power of durable, event-driven workflows, the practicalities and benefits of serverless as a billing model, the intricacies distributed systems, and more. Links https://joelhooks.com https://x.com/jhooks https://www.linkedin.com/in/joelhooks https://egghead.io https://www.coursebuilder.dev/tips/using-inngest-to-add-email-automation-feature-to-pro-next-js-adt43 We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: Joel Hooks.

A veteran of early Twitter's fail whale wars, Dmitriy joins the show to chat about the time when 70% of the Hadoop cluster got accidentally deleted, the financial reality of writing a book, and how to navigate acquisitions. Segments: (00:00:00) The Infamous Hadoop Outage (00:02:36) War Stories from Twitter's Early Days (00:04:47) The Fail Whale Era (00:06:48) The Hadoop Cluster Shutdown (00:12:20) “First Restore the Service Then Fix the Problem. Not the Other Way Around.” (00:14:10) War Rooms and Organic Decision-Making (00:16:16) The Importance of Communication in Incident Management (00:19:07) That Time When the Data Center Caught Fire (00:21:45) The "Best Email Ever" at Twitter (00:25:34) The Importance of Failing (00:27:17) Distributed Systems and Error Handling (00:29:49) The Missing README (00:33:13) Agile and Scrum (00:38:44) The Financial Reality of Writing a Book (00:43:23) Collaborative Writing Is Like Open-Source Coding (00:44:41) Finding a Publisher and the Role of Editors (00:50:33) Defining the Tone and Voice of the Book (00:54:23) Acquisitions from an Engineer's Perspective (00:56:00) Integrating Acquired Teams (01:02:47) Technical Due Diligence (01:04:31) The Reality of System Implementation (01:06:11) Integration Challenges and Gotchas Show Notes: - Dmitriy Ryaboy on Twitter: https://x.com/squarecog - The Missing README: https://www.amazon.com/Missing-README-Guide-Software-Engineer/dp/1718501838 - Chris Riccomini on how to write a technical book: https://cnr.sh/essays/how-to-write-a-technical-book Stay in touch: - Make Ronak's day by signing up for our newsletter to get our favorites parts of the convo straight to your inbox every week :D https://softwaremisadventures.com/ Music: Vlad Gluschenko — Forest License: Creative Commons Attribution 3.0 Unported: https://creativecommons.org/licenses/by/3.0/deed.en

#SecurityConfidential #DarkRhiinoSecurity Jake has been an innovator in the cloud-native ecosystem for over 15 years. After engineering roles at Amazon and Google, Jake founded Quay, the first private Docker registry, which was acquired by CoreOS. Jake then became an engineering leader at CoreOS, which was acquired by Red Hat (and then IBM). He is now the co-founder and CEO of AuthZed, the company commercializing SpiceDB, the industry-leading cloud-native permissions database. 00:00 Intro 00:58 Our guest 02:15 The Entrepreneur chip on your shoulder 06:58 The fear of failure 09:46 How do you pay salaries on open source when you use it daily 12:40 The basics of a Distributed Architecture 20:00 Distributed Databases 26:43 What if the platform isn't distributed? 31:38 AuthZed 43:21 What will AI do in your world? 47:01 News from Jake ---------------------------------------------------------------------- Kiteworks enables organizations to effectively manage risk in every send, share, receive, and save of sensitive content. To this end, they created a platform that delivers content governance, compliance, and protection to customers. The platform unifies, tracks, controls, and secures sensitive content moving within, into, and out of their organization, significantly improving risk management while ensuring regulatory compliance on all sensitive content communications. To learn more about Kiteworks, visit https://www.kiteworks.com/ ---------------------------------------------------------------------- To learn more about Jake visit https://kitcaster.com/jake-moshenko/ To learn more about Dark Rhiino Security visit https://www.darkrhiinosecurity.com ---------------------------------------------------------------------- SOCIAL MEDIA: Stay connected with us on our social media pages where we'll give you snippets, alerts for new podcasts, and even behind the scenes of our studio! Instagram: @securityconfidential and @Darkrhiinosecurity Facebook: @Dark-Rhiino-Security-Inc Twitter: @darkrhiinosec LinkedIn: @dark-rhiino-security Youtube: @DarkRhiinoSecurity ​ ----------------------------------------------------------------------

In this episode of ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠BragTalks⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠, host Heather VanCura interviews Patrick Chanezon about participating in standards and open source projects &/or communities. Patrick shares his experiences and the impact this participation in the projects and communities has had on his career. Listen to hear about how he approached getting involved and even some of the mistakes made along the way. Season 7 is about sharing the experiences of technical professionals and building on the interviews from the recently published book '⁠⁠Developer Career Masterplan'⁠⁠. This episode is a story that links to Chapters 11 and 14 of the book..hope you enjoy our new look and Season 7 of BragTalks! Biography: Patrick Chanezon manages the Cloud Developer Advocacy team in Developer Relations at Microsoft, helping developers achieve more with AI on Microsoft Cloud. Previously, at Docker Inc., he helped to build Docker, the world's leading software container platform, for developers and sysadmins. He helped establish open source and standards organizations such as Open Container Initiative, Cloud Native Computing Foundation or Green Software Foundation. Software developer and storyteller, he spent 8 years building platforms at Netscape & Sun, then 19 years evangelizing platforms at Google, VMware & Microsoft. His main professional interest is in building and kickstarting the network effect for these wondrous two-sided markets called Platforms. He has worked on platforms for AI, Cloud, Distributed Systems, Web, Social, Commerce, Ads, and Portals.

In this episode, Brian LeRoux, co-founder of Begin.com, discusses the evolution and rise of serverless full stack development. Brian shares insights on the history and future of JavaScript, the benefits of serverless architecture, and how front-end developers can leverage these technologies to build scalable and maintainable applications. Links https://brian.io https://webdev.rip https://github.com/brianleroux https://www.npmjs.com/~brianleroux https://twitter.com/brianleroux https://indieweb.social/@brianleroux https://www.linkedin.com/in/brianleroux https://begin.com https://arc.codes https://enhance.dev We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: Brian LeRoux.

In this episode of The GeekNarrator podcast, host Kaivalya Apte dives into the complexities of testing distributed systems with Will Wilson from Antithesis. If you're grappling with the challenges of testing databases, micro-services, and distributed systems, this episode is a must-watch. Will Wilson demystifies the concept of deterministic simulation testing, shares insights about its advantages over conventional testing methods, and explains how Antithesis helps developers ensure software reliability. Learn about the various strategies and techniques used to identify and resolve bugs, and explore how deterministic simulation can transform your software testing approach. Perfect for developers, engineers, and tech enthusiasts who are keen on improving their testing methodologies for complex systems. Chapters: 00:00 Introduction 03:04 Limitations of Conventional Testing Methods 04:09 Understanding Deterministic Simulation Testing 08:07 Implementing Deterministic Simulation Testing 14:30 Real-World Example: Chat Application 19:56 Antithesis Hypervisor and Determinism 27:06 Defining Properties and Assertions 38:34 Optimizing Snapshot Efficiency 40:44 Understanding Isolation in CI/CD Pipelines 43:39 Strategies for Effective Bug Detection 47:59 Exploring Program State Trees 51:17 Heuristics and Fuzzing Techniques 01:01:56 Mocking Third-Party APIs 01:05:54 Handling Long-Running Tests 01:09:06 Classifying and Prioritizing Bugs 01:15:35 Future Plans and Closing Remarks References: Hypervisor: https://antithesis.com/blog/deterministic_hypervisor/ AFL : https://github.com/google/AFL Antithesis website: https://antithesis.com/ Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #distributedsystems #databases #microservices #antithesis #fuzzer #testing

Summary In this week's episode, Anna (https://x.com/AnnaRRose) and Guille (https://x.com/GuilleAngeris) chat with Ying Tong Lai (https://x.com/therealyingtong) from Geometry Research (https://geometry.dev/) and Bryan Gillespie (https://x.com/bryan_gillespie) from Inversed Tech (https://inversed.tech/) about their latest research and works to date. They dive into the pair's recent work ‘SoK: Programmable Privacy in Distributed Systems (https://eprint.iacr.org/2024/982)', exploring the classifications and frameworks being introduced. Here's some additional links for this episode: SoK: Programmable Privacy in Distributed Systems by Benarroch, Gillespie, Lai and Miller (https://eprint.iacr.org/2024/982) Private Programmability in Zcash - Research Results and Community Discussion (https://forum.zcashcommunity.com/t/48016) Zcash Halo2 GitHub (https://github.com/zcash/halo2) Zk0x02 - An intro to Zcash and zkSNARKs - Ariel Gabizon (Zcash) (https://www.youtube.com/watch?v=Kx4cIkCY2EA) Moving SNARKs from the generic to algebraic group model by Ariel Gabizon (https://medium.com/@arielgabizon/moving-snarks-from-the-generic-to-algebraic-group-model-56549d60b90d) Explaining SNARKs Part I: Homomorphic Hidings by Ariel Gabizon (https://electriccoin.co/blog/snark-explain/) Differential Privacy in Constant Function Market Makers by Chitra, Angeris and Evans (https://fc22.ifca.ai/preproceedings/30.pdf) A Note on Privacy in Constant Function Market Makers by Angeris, Evans and Chitra (https://angeris.github.io/papers/cfmm-privacy.pdf) On Privacy Notions in Anonymous Communication by Kuhn, Beck, Schiffner, Jorswieck, and Strufe (https://arxiv.org/pdf/1812.05638) ZK Hack Montreal has been announced for Aug 9 - 11! Apply to join the hackathon here (https://zk-hack-montreal.devfolio.co/). Episode Sponsors Aleo (http://aleo.org/) is a new Layer-1 blockchain that achieves the programmability of Ethereum, the privacy of Zcash, and the scalability of a rollup. As Aleo is gearing up for their mainnet launch in Q1, this is an invitation to be part of a transformational ZK journey. Dive deeper and discover more about Aleo at http://aleo.org/ (http://aleo.org/). If you like what we do: * Find all our links here! @ZeroKnowledge | Linktree (https://linktr.ee/zeroknowledge) * Subscribe to our podcast newsletter (https://zeroknowledge.substack.com) * Follow us on Twitter @zeroknowledgefm (https://twitter.com/zeroknowledgefm) * Join us on Telegram (https://zeroknowledge.fm/telegram) * Catch us on YouTube (www.youtube.com/channel/UCYWsYz5cKw4wZ9Mpe4kuM_g)

Well-known for his insightful and meticulous write-ups on testing distributed systems, Kyle (aka Aphyr) joins the show to chat about the origins of Jepsen, how he built a business around testing distributed systems, his writing process, favorite databases, and more.   Segments: (00:03:29) From Physics to Software Engineering (00:07:47) The origins of Jepsen (00:09:41) Turning Jepsen into a full-time venture (00:13:14) Jepsen's testing philosophy (00:16:30) The consulting journey (00:19:16) Structuring a consultancy (00:22:32) Setting boundaries (00:24:32) Pricing misadventures (00:29:17) Pros and cons of being an independent consultant (00:32:08) Managing your time when working for yourself (00:38:23) Best part of the job (00:41:13) Early writing influences (00:45:25) LLMs and AI-generated content (00:48:17) “The period where you can trust what you read is actually very recent” (00:51:33) How to become a better writer (00:54:25) Developing a formal understanding of distributed systems (00:59:30) Common faults in distributed systems (01:01:17) The complexity of testing distributed systems (01:07:32) Communicating criticism effectively (01:10:26) Advice for distributed systems engineers (01:13:46) “Anybody trying to sell you a distributed lock is selling you sawdust and lies” (01:16:31) Failure mode documentation (01:18:52) The pitfalls of containerization (01:20:17) Lightning round - favorite databases   Show Notes: “Anybody who is trying to sell you a distributed lock is trying to sell you sawdust and Lies”: https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html Kyle's excellent write-ups on testing distributed systems: https://jepsen.io/analyses Kyle's blog: https://aphyr.com/posts Training courses that Kyle runs: https://jepsen.io/services/training   Stay in touch:

Welcome back to another episode of Building Better Developers, where we dive into essential aspects of software development. Today, we're delving into "Managing Hardware Resource Issues in Software Development: Best Practices and Real-World Examples." In this episode, we'll explore the often-overlooked yet critical area of hardware resource challenges and their impact on software performance. From memory and storage constraints to processing limitations, we'll navigate through common issues developers face and provide insights into effective debugging strategies and optimization techniques. Let's embark on this journey to understand and address hardware resource issues in software development. Listen to Host Rob and Michael discuss how they Manage Hardware Resource Issues: The Intersection of Hardware Resource Issues and Software Development Debugging isn't just about finding and fixing code errors; it's also about understanding hardware limitations. As developers, we might face memory, processing, and storage issues, particularly when dealing with large-scale applications, big data, or heavy user loads. Memory and Storage Issues: One of the biggest hurdles is managing memory and storage efficiently. For instance, when physical memory is exceeded, systems start caching to disk, significantly slowing down processing. This is especially true with older, disk-driven storage, where processing speeds can drop from microseconds to several seconds. Modern SSDs mitigate this to some extent, but challenges remain. Real-world Example: Imagine an application running smoothly until it hits a memory limit. Suddenly, it starts swapping data to disk, slowing down operations. Users, noticing the delay, might re-trigger processes, compounding the issue. A simple solution like adding more memory might seem tempting, but a more robust fix involves optimizing the software to handle resources better. Debugging Beyond Code: Addressing Hardware Resource Issues Deeper issues might be at play when software behaves unpredictably despite no code changes. These situations necessitate detailed logging and monitoring of the operating system and platform to identify the root cause. Example of Debugging Approach: A developer might initially write code that handles small datasets well but faces performance issues with larger datasets. Switching from processing entire files in memory to processing them line-by-line can help, but a complete paradigm shift is sometimes necessary. Using a relational database for complex operations instead of in-memory processing can drastically improve performance. Distributed Systems and Microservices: Managing Hardware Resource Issues Microservices and APIs are prevalent in today's distributed computing landscape. These systems can mask underlying problems as they scale up. Healthcare Application Case Study: An application was crashing due to a combination of insufficient processing power, memory, and disk space. The initial fix involved moving processes to different servers, but persistent issues led to a comprehensive hardware upgrade and better load management strategies. Tips for Better Resource Management to Avoid Hardware Resource Issues Understand Resource Utilization: Developers must understand how their code impacts system resources. Tools like profilers can help monitor CPU and memory usage, identifying leaks and bottlenecks. Optimize Data Storage and Retrieval: It is crucial to use databases and in-memory data stores. Techniques like indexing and temporary tables can drastically reduce processing times. Avoid Overloading Systems: Ensure that applications don't unnecessarily burden the system. Log files, for example, should be managed to prevent them from consuming excessive disk space. Prepare for Scalability: Design applications to scale efficiently, balancing the load across distributed systems and ensuring that containers or servers are adequately powered for their tasks. Best Practices for Handling Hardware Resource Issues in Software Development Local vs. Remote Resources: Store critical files locally to minimize dependencies on external resources. This ensures that applications remain operational even if external services fail. Turn Off Debugging in Production: Debugging tools consume resources and can slow down applications. Ensure they are disabled in production environments to optimize performance. Continuous Monitoring: Regularly monitor system performance and resource usage to address potential issues before they escalate preemptively. In conclusion, effective management of hardware resource issues is paramount in ensuring smooth software performance. By implementing best practices and learning from real-world examples, developers can optimize their code and mitigate potential bottlenecks. Remember to monitor system performance regularly, balance resource utilization, and prioritize scalability. With a proactive approach to debugging and resource management, you can build resilient software that performs optimally even under heavy loads. Stay Connected: Join the Developreneur Community We invite you to join our community and share your coding journey with us. Whether you're a seasoned developer or just starting, there's always room to learn and grow together. Contact us at info@develpreneur.com with your questions, feedback, or suggestions for future episodes. Together, let's continue exploring the exciting world of software development. Additional Resources Troubleshooting Tips for Common EC2 Problems Testing Challenges – Teach More Than Quality Software Architecture Patterns and Anti-Patterns Overview Behind the Scenes Podcast Video

Infrastructure engineer and Kubernetes ingress-Nginx maintainer James Strong joins host Robert Blumen to discuss the Kubernetes networking layer. The discussion draws on content from Strong's book on the topic and covers a lot of ground, including: the Kubernetes network's use of different IP ranges than the host network; overlay network with its own IP ranges compared to using expanded portions of the host network ranges; adding routes with kernel extension points; programming kernel extension points with IP tables compared to eBPF; how routes are updated as the host network gains or loses nodes, the use of the Linux network namespace to isolate each pod; routing between pods on the same host; routing between pods across the host network; the container-network interface (CNI); the CNI ecosystem; differences between CNIs; choosing a CNI when running on a public cloud service; the Kubernetes service abstraction with a cluster-wide IP address; monitoring and telemetry of the Kubernetes network; and troubleshooting the Kubernetes network. Brought to you by IEEE Software magazine and IEEE Computer Society.

From writing the first lines of Kafka over a Christmas break as a LinkedIn engineer to running a public company as the CEO of Confluent, Jay joins the show to chat about how he and his co-founders convinced investors to take a chance on their vision, what many engineers get wrong about communication, and why engineers can make great CEOs - even when coding is not in the job description. And much more. Segments: (00:01:16) The Shaved Head Bet (00:04:07) Fundraising (00:12:16) The Role of Technical Background in VCs (00:15:48) The power of believing in the possibility of important changes (00:18:29) The Journey to starting Confluent (00:27:11) Kafka's Controversial Beginnings (00:34:30) Effective Communication in Engineering (00:44:20) The Early Days of Kafka (00:48:31) The Power of Storytelling (00:57:19) Early days of Confluent (01:03:06) Do Engineers Make Good CEOs? (01:07:59) A Typical Day in the Life of a CEO (01:12:24) The Evolution of Data Streaming Show Notes: - “The log” blog post that solidified Jay and his co-founders' conviction to found Confluent: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying - Jay on twitter: https://x.com/jaykreps Stay in touch:

Avalonia XPF This episode of The Modern .NET Show is supported, in part, by Avalonia XPF, a binary-compatible cross-platform fork of WPF, enables WPF apps to run on new platforms with minimal effort and maximum compatibility. NService Bus This episode of The Modern .NET Show is supported, in part, by NServiceBus, the ultimate tool to build robust and reliable systems that can handle failures gracefully, maintain high availability, and scale to meet growing demand. Make sure you click the link in the show notes to learn more about NServiceBus. Show Notes When you talk to the Temporal guys and look at the way that they've done some of their work is they have a... they have stuff that can run for years. So for instance, they'll kick off a workflow for one of their customers, kicks off a workflow when the customer's created, and that workflow is like managed by Temporal for as long as that customer is a customer. So it could be, you know, I don't know, occasionally sending out an email to, you know, "happy birthday" or something, or sending them promotions or whatever. So they see, they see workflows as like lifetime things — John Kattenhorn Welcome to The Modern .NET Show! Formerly known as The .NET Core Podcast, we are the go-to podcast for all .NET developers worldwide and I am your host Jamie "GaProgMan" Taylor. In this episode, John Kattenhorn joined us to talk about Temporal.io and their platform for building durable workflows which can operate for years at a time. John in the CEO of Applicita and wanted to talk about the ways that developers can build applications and workflows which can live in the cloud for a very long time. And the great thing about Temporal is it manages all of those resources for you. So if you imagined me and you trying to do that, we'd end up standing up, I don't know, a running service or something that was constantly polling the data, looking for eligible customers or something. You'd be burning some resources looking at that stuff, and that's not how they do that. So if you've got like a million customers, the Temporal system dehydrates everything that isn't relevant and only hydrates the workflows that have an action to perform — John Kattenhorn So let's sit back, open up a terminal, type in dotnet new podcast and we'll dive into the core of Modern .NET. Supporting the Show If you find this episode useful in any way, please consider supporting the show by either leaving a review (check our review page for ways to do that), sharing the episode with a friend or colleague, buying the host a coffee, or considering becoming a Patron of the show. Full Show Notes The full show notes, including links to some of the things we discussed and a full transcription of this episode, can be found at: https://dotnetcore.show/season-6/temporal-orchestrating-success-in-distributed-systems-with-security-and-simplicity-with-john-kattenhorn-with-john-kattenhorn/ Useful Links temporal.io Polly Durable Tasks from Microsoft Azure Event Hubs [Azure] Service Bus Cadence Godot Hangfire Saga pattern System.Text.Json Namespace GitHub samples eShop Temporal's YouTube channel Temporal's Slack John Kattenhorn on X Supporting the show: Leave a rating or review Buy the show a coffee Become a patron Getting in touch: via the contact page joining the Discord Music created by Mono Memory Music, licensed to RJJ Software for use in The Modern .NET Show Remember to rate and review the show on Apple Podcasts, Podchaser, or wherever you find your podcasts, this will help the show's audience grow. Or you can just share the show with a friend. And don't forget to reach out via our Contact page. We're very interested in your opinion of the show, so please get in touch. You can support the show by making a monthly donation on the show's Patreon page at: https://www.patreon.com/TheDotNetCorePodcast.

This interview was recorded for the GOTO Book Club.http://gotopia.tech/bookclubRead the full transcription of the interview hereUnmesh Joshi - Principal Consultant at Thoughtworks & Author of "Patterns of Distributed Systems"James Lewis - Principal Consultant & Technical Director at ThoughtworksRESOURCESUnmeshhttps://twitter.com/unmeshjoshihttps://www.linkedin.com/in/unmesh-joshi-9487635https://www.thoughtworks.com/profiles/u/unmesh-joshiJameshttps://twitter.com/boicyhttps://linkedin.com/in/james-lewis-microserviceshttps://github.com/boicyhttps://www.bovon.orghttps://www.thoughtworks.com/profiles/j/james-lewisDESCRIPTIONA Patterns Approach to Designing Distributed Systems and Solving Common Implementation ProblemsMore and more enterprises today are dependent on cloud services from providers like AWS, Microsoft Azure, and GCP. They also use products, such as Kafka and Kubernetes, or databases, such as YugabyteDB, Cassandra, MongoDB, and Neo4j, that are distributed by nature. Because these distributed systems are inherently stateful systems, enterprise architects and developers need to be prepared for all the things that can and will go wrong when data is stored on multiple servers--from process crashes to network delays and unsynchronized clocks."Patterns of Distributed Systems" describes a set of patterns that have been observed in mainstream open-source distributed systems. Studying the common problems and the solutions that are embodied by the patterns in this guide will give you a better understanding of how these systems work, as well as a solid foundation in distributed system design principles.* Book description: © O'ReillyRECOMMENDED BOOKSUnmesh Joshi • Patterns of Distributed SystemsDarnell, Harrison & Seldess • CockroachDB: The Definitive GuideGuy Harrison • Next Generation DatabasesBurns, Beda & Hightower • Kubernetes: Up & RunningJez Humble & Dave Farley • Continuous DeliveryTwitterInstagramLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

In this episode, James Quick and Amy Dutton chat with James Cowling, co-founder of Convex, about designing infrastructure for product engineers. James explains the innovative features of Convex, including its JavaScript-based queries and real-time data subscriptions, and compares it to Firebase. They also discuss the challenges of edge computing, the importance of user state, and the role of AI in modern development.Show Notes[00:00:00] - Introduction to the Episode[00:01:00] - James Cowling's Background and Convex OverviewConvex[00:01:52] - Deep Dive into Convex[00:05:29] - User State and Application Development[00:07:05] - Challenges of Edge Computing[00:09:53] - Automatic Caching and Real-Time Updates[00:13:22] - AI and Backend Integration[00:17:01] - Leveraging AI in Applications[00:21:11] - Convex's Infrastructure and Technology[00:25:28] - Comparisons with Other Platforms[00:30:03] - Server Rendering and Data Storage[00:33:19] - Physical Challenges in Data Centers[00:37:04] - Cost Efficiency and Cloud Platforms[00:40:56] - Final Thoughts on Infrastructure[00:43:30] - Picks and Plugs Introduction[00:44:11] - James Cowling's Picks and Plugs[00:45:41] - Amy Dutton's Picks and Plugs[00:48:28] - James Quick's Picks and Plugs

Kubernetes and back - Why I don't run distributed systems, NetApp's strategic contributions to FreeBSD: a deep dive into upstreaming efforts, Make your own E-Mail server - Part 2 - Adding Webmail and More with Nextcloud, Poudriere on Apple Silicon, One less Un*xy option for 32-bit PowerPC, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Kubernetes and back - Why I don't run distributed systems (https://www.davd.io/posts/2024-03-20-kubernetes-and-back-why-i-dont-run-distributed-systems/) NetApp's strategic contributions to FreeBSD: a deep dive into upstreaming efforts (https://freebsdfoundation.org/blog/netapps-strategic-contributions-to-freebsd-a-deep-dive-into-upstreaming-efforts/) News Roundup Make your own E-Mail server - Part 2 - Adding Webmail and More with Nextcloud (https://it-notes.dragas.net/2024/03/21/make-your-own-email-server-freebsd-adding-nextcloud-part2/) Poudriere on Apple Silicon (https://oliver-epper.de/posts/poudriere-on-m1-mac/) One less Un*xy option for 32-bit PowerPC (http://tenfourfox.blogspot.com/2024/02/one-less-unxy-option-for-32-bit-powerpc.html) Beastie Bits Powering up the future: the new FreeBSD cluster in Chicago (https://freebsdfoundation.org/blog/powering-up-the-future-the-new-freebsd-cluster-in-chicago/) Dragonflybsd 6.5 Snapshot Release on Acer Nitro AN515-51/58-XXX Series Laptops (https://github.com/catfacedck/Dragonflybsd-Acer-Nitro-Laptops-AN515-5158-XXX) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

In Elixir Wizards Office Hours Episode 2, "Discovery Discoveries," SmartLogic's Project Manager Alicia Brindisi and VP of Delivery Bri LaVorgna join Elixir Wizards Sundi Myint and Owen Bickford on an exploratory journey through the discovery phase of the software development lifecycle. This episode highlights how collaboration and communication transform the client-project team dynamic into a customized expedition. The goal of discovery is to reveal clear business goals, understand the end user, pinpoint key project objectives, and meticulously document the path forward in a Product Requirements Document (PRD). The discussion emphasizes the importance of fostering transparency, trust, and open communication. Through a mutual exchange of ideas, we are able to create the most tailored, efficient solutions that meet the client's current goals and their vision for the future. Key topics discussed in this episode: Mastering the art of tailored, collaborative discovery Navigating business landscapes and user experiences with empathy Sculpting project objectives and architectural blueprints Continuously capturing discoveries and refining documentation Striking the perfect balance between flexibility and structured processes Steering clear of scope creep while managing expectations Tapping into collective wisdom for ongoing discovery Building and sustaining a foundation of trust and transparency Links mentioned in this episode: https://smartlogic.io/ Follow SmartLogic on social media: https://twitter.com/smartlogic Contact Bri: bri@smartlogic.io What is a PRD? https://en.wikipedia.org/wiki/Productrequirementsdocument Special Guests: Alicia Brindisi and Bri LaVorgna.

In this video, I talk to Stephan Ewen from Restate, who is popularly known from the world of Apache Flink. We have talked about the problems in the world of Distributed systems and the complex solutions developers have to deal with. This complexity makes the architecture so complex that it eventually creates reliability, Observability and delivery velocity problems. Restate aims to solve it by making resilience and durability for your services, functions and RPC a lot simpler. Chapters: 00:00 Introduction 00:45 Introducing Restate: A Solution for Distributed System Challenges 01:22 Deep Dive into Restate with Stefan: From Apache Flink to Building Resilient Systems 06:04 The Complexities of Distributed Systems and How Restate Addresses Them 15:49 The Vision of Restate: Simplifying Developer Experience in Distributed Systems 24:42 Integrating Restate into Your Architecture: A User's Perspective 33:16 Exploring Restate: The Durable Service Mesh 33:32 The Power of Restate in Handling Transactions 34:26 Restate's Role in Service Communication and Durability 35:40 Deep Dive into Restate's Mechanisms and Benefits 38:04 Practical Example: Email Pipeline with Restate 39:40 Understanding Restate's Log and Event Handling 58:43 Restate's Unique Features and Programming Model 01:04:22 Final Thoughts on Restate's Impact and Deployment Restate: https://restate.dev/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #distributedsystems #faulttolerance #reliability #resilience

Paul Frambot is the Co-Founder and CEO of Morpho Labs, a research and development company responsible for building and growing the Morpho protocol.  Paul co-founded Morpho Labs whilst studying towards his, now-completed, Master's in Parallel & Distributed Systems from the Institut Polytechnique de Paris in 2021. During his studies, Paul raised $18M from top investors, including Andreessen Horowitz (a16z) and Variant for Morpho, which has since grown into a multi-billion-dollar lending protocol. The latest version, Morpho Blue, is a completely independent but remarkably simple protocol that serves as a secure, efficient, and flexible base layer for users and applications.In this conversation, we discuss:- Liquid staking and yield farming- Next evolution of DeFi Spending- Moving from App to Infra = the future of DeFi- DeFi 2.0 will define crypto's coming bull market- Gauntlet leaving Aave to join Morpho- 2024 will see a shift from monolithic to layered protocols- DAOs are not suited for managing the risk of protocols- Morpho Blue- Lowering the barrier to entry- Raising $18M while still in schoolMorpho LabsWebsite: morpho.orgX: @MorphoLabsDiscord: discord.morpho.orgPaul FrambotX: @PaulFrambotLinkedIn: Paul Frambot   ---------------------------------------------------------------------------------  This episode is brought to you by PrimeXBT.  PrimeXBT offers a robust trading system for both beginners and professional traders that demand highly reliable market data and performance. Traders of all experience levels can easily design and customize layouts and widgets to best fit their trading style. PrimeXBT is always offering innovative products and professional trading conditions to all customers.  PrimeXBT is running an exclusive promotion for listeners of the podcast. After making your first deposit, 50% of that first deposit will be credited to your account as a bonus that can be used as additional collateral to open positions.  Code: CRYPTONEWS50  This promotion is available for a month after activation. Click the link below:  PrimeXBT x CRYPTONEWS50 

Distributed systems are ubiquitous yet complex. They can be particularly demanding for software developers and architects tasked with dealing with the sometimes unpredictable nature of the interactions between their various parts. That's why Thoughtworker Unmesh Joshi wrote Patterns of Distributed Systems. Published at the end of 2023, the book explores a number of patterns that characterize distributed systems, and uses them to not only help readers better understand how such systems work but also to solve problems and challenges that often arise. On this episode of the Technology Podcast, Unmesh joins hosts Scott Shaw and Rebecca Parsons to talk about his book, explaining where the idea came from, how he put it together and why it's important to get beneath neat abstractions to really get to grips with the inner workings of distributed systems. Learn more about Patterns of Distributed Systems: https://www.pearson.com/subject-catalog/p/patterns-of-distributed-systems/P200000011305/9780138221980

Ivo de Carvalho Peixinho, Head Cybercrime Intelligence Unit, INTERPOL, has a BS degree in Computer Science at Universidade Federal da Bahia, with two post-graduations, one in Distributed Systems and another on Mechatronics. He is also a BS7799 certified auditor. Ivo has more than 10 years of experience on network security, and worked the last two years on security research and incident handling. Prior to Interpol, he works as a Forensics Expert at the Brazilian Federal Police Department.In this interview, Ivo shared insights in 6 topics:1. Information sharing challenges: Addressing conflicts between private sectors and law enforcement priorities, particularly in cases like ransomware, where the need to restore operations clashes with preserving evidence.2. Data Processing Regulations: Exploring Interpol's regulations for data processing and exchange, considering cultural barriers, language differences, and data sovereignty concerns when sharing information among different countries.3. Project Gateway Initiative: Understanding the process and significance of Project Gateway, a collaboration framework between Interpol and private entities, including the steps for private organizations to join this initiative.4. AI's Impact on Cybersecurity: Recognizing AI as a productivity enhancer, both for defenders and attackers, and the importance of balancing technological advancements with legal frameworks in the evolving cybercrime landscape.5. Training and Collaboration: Emphasizing the need for regular training sessions and exercises to foster a common understanding and language among global law enforcement agencies, crucial for effective collaboration during global cybercrime operations.6. Skill Set for Investigators: Discussing the essential skills for law enforcement investigators, including self-driven motivation, the importance of work-life balance, and the role of teamwork, personal interests, and joy in maintaining stamina and perseverance in the field of cybersecurity.Recorded 7th December, ISC2 Secure Asia Pacific 2023, 10.30am.#interpol #mysecuritytv #cybersecurity

William Morgan, founder of the Linkerd service mesh and CEO of Bouyant, joins SE Radio's Robert Blumen for a discussion of sidecars, service mesh, and a forthcoming enhancement to kubernetes to support sidecars natively. The conversation explores the origin of sidecars, sidecars and service mesh, and migrating service mesh to kubernetes. They take a deep dive into some aspects of running service mesh on kubernetes, the difficulties in running a sidecar container in a pod, and Kubernetes Enhancement Proposal (KEP) 753, which is intended to provide better native support for sidecar containers. William also gives some thoughts on the continuing relevance of service mesh.

Dean Tribble is CEO of Agoric OpCo, the software development company supporting Agoric, the JavaScript smart contract platform, and Proof of Stake (PoS) blockchain. Dean drives the vision to build a safe and composable JavaScript smart contract platform to onboard the next 14M+ developers into Web3.An OG Cypherpunk and pioneer in distributed systems and smart contract technology, Dean, co-designed the negotiation process and contract for the first smart contracting system, AMiX. As a Principal Architect at Microsoft, he co-designed the Midori distributed object-capability operating system. As founder and CTO for Agorics (acquired by Microsoft), he architected the brokerage information system for Schwab's active traders and the WebMart electronic contract system for SunLabs. He was CTO of VerifyValid and VP of Deluxe, where his echeck systems have carried billions of dollars.In this conversation, we discuss:- Hardened JavaScript- MetaMask Snaps- Building the first smart contract in 1989- History of distributed systems- Getting acquired by Microsoft- Babylon integration for bitcoin time stamping- Agoric partnership with BitGo- $BLD and $IST- Institutional adoption of DeFiAgoricWebsite: agoric.comX: @agoricTelegram: t.me/agoricsystemsDean TribbleX: @DeanTribbleLinkedIn: E. Dean Tribble   ---------------------------------------------------------------------------------  This episode is brought to you by PrimeXBT.  PrimeXBT offers a robust trading system for both beginners and professional traders that demand highly reliable market data and performance. Traders of all experience levels can easily design and customize layouts and widgets to best fit their trading style. PrimeXBT is always offering innovative products and professional trading conditions to all customers.  PrimeXBT is running an exclusive promotion for listeners of the podcast. After making your first deposit, 50% of that first deposit will be credited to your account as a bonus that can be used as additional collateral to open positions.  Code: CRYPTONEWS50  This promotion is available for a month after activation. Click the link below:  PrimeXBT x CRYPTONEWS50 

In this episode of Elixir Wizards, Xiang Ji and Nathan Hessler join hosts Sundi Myint and Owen Bickford to compare actor model implementation in Elixir, Ruby, and Clojure. In Elixir, the actor model is core to how the BEAM VM works, with lightweight processes communicating asynchronously via message passing. GenServers provide a common abstraction for building actors, handling messages, and maintaining internal state. In Ruby, the actor model is represented through Ractors, which currently map to OS threads. They discuss what we can learn by comparing models, understanding tradeoffs between VMs, languages, and concurrency primitives, and how this knowledge can help us choose the best tools for a project. Topics discussed in this episode: Difference between actor model and shared memory concurrency Isolation of actor state and communication via message passing BEAM VM design for high concurrency via lightweight processes GenServers as common abstraction for building stateful actors GenServer callbacks for message handling and state updates Agents as similar process abstraction to GenServers Shared state utilities like ETS for inter-process communication Global Interpreter Lock in older Ruby VMs Ractors as initial actor implementation in Ruby mapping to threads Planned improvements to Ruby concurrency in 3.3 Akka implementation of actor model on JVM using thread scheduling Limitations of shared memory concurrency on JVM Project Loom bringing lightweight processes to JVM Building GenServer behavior in Ruby using metaprogramming CSP model of communication using channels in Clojure Differences between BEAM scheduler and thread-based VMs Comparing Elixir to academic languages like Haskell Remote and theScore are hiring! Links mentioned in this episode: theScore is hiring! https://www.thescore.com/ Remote is also hiring! https://remote.com/ Comparing the Actor Model and CSP with Elixir and Clojure (https://xiangji.me/2023/12/18/comparing-the-actor-model-and-csp-with-elixir-and-clojure/) Blog Post by Xiang Ji Comparing the Actor model & CSP concurrency with Elixir & Clojure (https://www.youtube.com/watch?v=lIQCQKPRNCI) Xiang Ji at ElixirConf EU 2022 Clojure Programming Language https://clojure.org/ Akka https://akka.io/ Go Programming Language https://github.com/golang/go Proto Actor for Golang https://proto.actor/ RabbitMQ Open-Source Message Broker Software  https://github.com/rabbitmq JVM Project Loom https://github.com/openjdk/loom Ractor for Ruby  https://docs.ruby-lang.org/en/master/ractor_md.html Seven Concurrency Models in Seven Weeks: When Threads Unravel (https://pragprog.com/titles/pb7con/seven-concurrency-models-in-seven-weeks/)by Paul Butcher Seven Languages in Seven Weeks (https://pragprog.com/titles/btlang/seven-languages-in-seven-weeks/) by Bruce A. Tate GenServer https://hexdocs.pm/elixir/1.12/GenServer.html ets https://www.erlang.org/doc/man/ets.html Elixir in Action (https://pragprog.com/titles/btlang/seven-languages-in-seven-weeks/) by Saša Jurić Redis https://github.com/redis/redis Designing for Scalability with Erlang/OTP (https://www.oreilly.com/library/view/designing-for-scalability/9781449361556/) by Francesco Cesarini & Steve Vinoski Discord Blog: Using Rust to Scale Elixir for 11 Million Concurrent Users (https://discord.com/blog/using-rust-to-scale-elixir-for-11-million-concurrent-users) Xiang's website https://xiangji.me/ Feeling Good: The New Mood Therapy (https://www.thriftbooks.com/w/feeling-good-the-new-mood-therapy-by-david-d-burns/250046/?resultid=7691fb71-d8f9-4435-a7a3-db3441d2272b#edition=2377541&idiq=3913925) by David D. Burns Special Guests: Nathan Hessler and Xiang Ji.

On today's episode, Elixir Wizards Owen Bickford and Dan Ivovich compare notes on building web applications with Elixir and the Phoenix Framework versus Ruby on Rails. They discuss the history of both frameworks, key differences in architecture and approach, and deciding which programming language to use when starting a project. Both Phoenix and Rails are robust frameworks that enable developers to build high-quality web apps—Phoenix leverages functional programming in Elixir and Erlang's networking for real-time communication. Rails follows object-oriented principles and has a vast ecosystem of plug-ins. For data-heavy CRUD apps, Phoenix's immutable data pipelines provide some advantages. Developers can build great web apps with either Phoenix or Rails. Phoenix may have a slight edge for new projects based on its functional approach, built-in real-time features like LiveView, and ability to scale efficiently. But, choosing the right tech stack depends heavily on the app's specific requirements and the team's existing skills. Topics discussed in this episode: History and evolution of Phoenix Framework and Ruby on Rails Default project structure and code organization preferences in each framework Comparing object-oriented vs functional programming paradigms CRUD app development and interaction with databases Live reloading capabilities in Phoenix LiveView vs Rails Turbolinks Leveraging WebSockets for real-time UI updates Testing frameworks like RSpec, Cucumber, Wallaby, and Capybara Dependency management and size of standard libraries Scalability and distribution across nodes Readability and approachability of object-oriented code Immutability and data pipelines in functional programming Types, specs, and static analysis with Dialyzer Monkey patching in Ruby vs extensible core language in Elixir Factors to consider when choosing between frameworks Experience training new developers on Phoenix and Rails Community influences on coding styles Real-world project examples and refactoring approaches Deployment and dev ops differences Popularity and adoption curves of both frameworks Ongoing research into improving Phoenix and Rails Links Mentioned in this Episode: SmartLogic.io (https://smartlogic.io/) Dan's LinkedIn (https://www.linkedin.com/in/divovich/) Owen's LinkedIn (https://www.linkedin.com/in/owen-bickford-8b6b1523a/) Ruby https://www.ruby-lang.org/en/ Rails https://rubyonrails.org/ Sams Teach Yourself Ruby in 21 Days (https://www.overdrive.com/media/56304/sams-teach-yourself-ruby-in-21-days) Learn Ruby in 7 Days (https://www.thriftbooks.com/w/learn-ruby-in-7-days---color-print---ruby-tutorial-for-guaranteed-quick-learning-ruby-guide-with-many-practical-examples-this-ruby-programming-book--to-build-real-life-software-projects/18539364/#edition=19727339&idiq=25678249) Build Your Own Ruby on Rails Web Applications (https://www.thriftbooks.com/w/build-your-own-ruby-on-rails-web-applications_patrick-lenz/725256/item/2315989/?utm_source=google&utm_medium=cpc&utm_campaign=low_vol_backlist_standard_shopping_customer_acquisition&utm_adgroup=&utm_term=&utm_content=593118743925&gad_source=1&gclid=CjwKCAiA1MCrBhAoEiwAC2d64aQyFawuU3znN0VFgGyjR0I-0vrXlseIvht0QPOqx4DjKjdpgjCMZhoC6PcQAvD_BwE#idiq=2315989&edition=3380836) Django https://github.com/django Sidekiq https://github.com/sidekiq Kafka https://kafka.apache.org/ Phoenix Framework https://www.phoenixframework.org/ Phoenix LiveView https://hexdocs.pm/phoenixliveview/Phoenix.LiveView.html#content Flask https://flask.palletsprojects.com/en/3.0.x/ WebSockets API https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API WebSocket connection for Phoenix https://github.com/phoenixframework/websock Morph Dom https://github.com/patrick-steele-idem/morphdom Turbolinks https://github.com/turbolinks Ecto https://github.com/elixir-ecto Capybara Testing Framework https://teamcapybara.github.io/capybara/ Wallaby Testing Framework https://wallabyjs.com/ Cucumber Testing Framework https://cucumber.io/ RSpec https://rspec.info/

Today on Elixir Wizards, Manuel Rubio, author of Erlang/OTP: A Concurrent World and Dan Plyukhin, creator of the UIGC Actor Garbage Collector for Akka, join host Dan Ivovich to compare notes on garbage collection in actor models. The discussion digs into the similarities and differences of actor-based garbage collection in Erlang and Akka and introduces Dan's research on how to perform garbage collection in a distributed actor system. Topics discussed: Akka is akin to Erlang actors for the JVM using Scala, with similar principles like supervision trees, messages, and clustering Erlang uses generational garbage collection and periodically copies live data to the old heap for long-lived elements Actor GC aims to determine when an actor's memory can be reclaimed automatically rather than manually killing actors Distributed actor GC is more challenging than object GC due to the distributed nature and relationships between actors across nodes Challenges include reasoning about failures like dropped messages and crashed nodes GC balance requires optimization of resource release and CPU load management Immutability helps Erlang GC, but copying data for messages impacts performance Research into distributed actor GC is still ongoing, with opportunities for improvement Fault tolerance in Erlang relies on user implementation rather than low-level guarantees Asynchronous messages in Erlang/Elixir mean references may become invalid which is similar to the distributed GC approaches in Dan's research Idempotent messaging is recommended to handle possible duplicates from failures Help your local researcher! Researchers encourage communication from practitioners on challenges and use cases Links mentioned: Erlang/OTP Volume 1: A Concurrent World by Manuel Rubio https://altenwald.com/en/book/en-erlang-i  Scala https://www.scala-lang.org/  Akka Framework https://github.com/akka  JVM (Java Virtual Machine) https://www.java.com/en/download/  The BEAM VM https://www.erlang.org/blog/a-brief-beam-primer/ Hadoop Framework https://hadoop.apache.org/   Pony Programming Language https://www.ponylang.io/  SLSA Programming Language https://wcl.cs.rpi.edu/salsa/#:~:text=SALSA%20 Paxos Algorithm https://en.wikipedia.org/wiki/Paxos(computerscience)  Raft library for maintaining a replicated state machine https://github.com/etcd-io/raft  Dan's Website https://dplyukhin.github.io/  Dan Plyukhin on Twitter: https://twitter.com/dplyukhin  Dan Plyukhin's YouTube channel: https://m.youtube.com/@dplyukhin UIGC on GitHub https://github.com/dplyukhin/UIGC  Manuel's Website https://altenwald.com/  Manuel Rubio on Twitter: https://twitter.com/MRonErlang Special Guests: Dan Plyukhin and Manuel Rubio.

Lane chats with his distributed systems professor from when he was a computer science undergraduate, Dr. Russ Ross. They talk about the state of the hiring market in 2023, LLVM, and of course, distributed systems!Learn back-end development - https://boot.devListen on your favorite podcast player: https://www.backendbanter.comRuss Ross's Twitter: https://twitter.com/_russross?lang=enLike & subscribe for the algo if you enjoyed the video!

In today's episode, Sundi and Owen are joined by Yordis Prieto and Stephen Chudleigh to compare notes on HTTP requests in Elixir vs. Ruby, JavaScript, Go, and Rust. They cover common pain points when working with APIs, best practices, and lessons that can be learned from other programming languages. Yordis maintains Elixir's popular Tesla HTTP client library and shares insights from building APIs and maintaining open-source projects. Stephen has experience with Rails and JavaScript, and now works primarily in Elixir. They offer perspectives on testing HTTP requests and working with different libraries. While Elixir has matured, there is room for improvement - especially around richer struct parsing from HTTP responses. The discussion highlights ongoing efforts to improve the developer experience for HTTP clients in Elixir and other ecosystems. Topics Discussed in this Episode HTTP is a protocol - but each language has different implementation methods Tesla represents requests as middleware that can be modified before sending Testing HTTP requests can be a challenge due to dependence on outside systems GraphQL, OpenAPI, and JSON API provide clear request/response formats Elixir could improve richer parsing from HTTP into structs Focus on contribution ergonomics lowers barriers for new participants Maintainers emphasize making contributions easy via templates and clear documentation APIs drive adoption of standards for client/server contracts They discuss GraphQL, JSON API, OpenAPI schemas, and other standards that provide clear request/response formats TypeScript brings types to APIs and helps to validate responses Yordis notes that Go and Rust make requests simple via tags for mapping JSON to structs Language collaboration shares strengths from different ecosystems and inspires new libraries and tools for improving the programming experience Links Mentioned Elixir-Tesla Library: https://github.com/elixir-tesla/tesla Yordis on Github: https://github.com/yordis Yordis on Twitter: https://twitter.com/alchemist_ubi Yordis on LinkedIn: https://www.linkedin.com/in/yordisprieto/ Yordis on YouTube: https://www.youtube.com/@alchemistubi Stephen on Twitter: https://twitter.com/stepchud Stephen's projects on consciousness: https://harmonicdevelopment.us Owen suggests: Http.cat HTTParty: https://github.com/jnunemaker/httparty Guardian Library: https://github.com/ueberauth/guardian Axios: https://axios-http.com/ Straw Hat Fetcher: https://github.com/straw-hat-team/nodejs-monorepo/tree/master/packages/%40straw-hat/fetcher Elixir Tesla Wiki: https://github.com/elixir-tesla/tesla/wiki HTTPoison: https://github.com/edgurgel/httpoison Tesla Testing: https://hexdocs.pm/tesla/readme.html#testing Tesla Mock: https://hexdocs.pm/tesla/Tesla.Mock.html Finch: https://hex.pm/packages/finch Mojito: https://github.com/appcues/mojito Erlang Libraries and Frameworks Working Group: https://github.com/erlef/libs-and-frameworks/ and https://erlef.org/wg/libs-and-frameworks Special Guests: Stephen Chudleigh and Yordis Prieto.

This episode of the #citizenweb3 podcast features Zenobia Godschalk, SVP of Communications at Hedera Hashgraph, an open source, public network governed by leading organizations around the world The Hedera Governing Council claims to be a fully decentralized and transparent governing body of independent, global organizations consisting of enterprises, web3 projects, and prestigious universities. We spoke to Zenobia (https://twitter.com/zenobiazag?lang=en) about Hedera (https://hedera.com/) and: - Centralized governance - Understanding a public ledger - What aspects of a project or product excites consumers - Catalysts for mass adoption - Tokenization of real estate - Block chain solving real world problems - Why choose web3 - Cybersecurity as an arms race - The Trust layer of the internet - Self sovereign identity in Web3 - Cybersecurity - The secret on how to succeed while wearing many hats - Living in an amazing time of discovery - Social responsibility - Empathy and giving back as a source of motivation - Importance of education - Data control - Balance of ethics in Data and adverting - Consumers taking back the power - Zoroastrianism - Adoption If you like what we do at Citizen Cosmos: - Stake with Citizen Cosmos validator (https://www.citizencosmos.space/staking) - Help support the project via Gitcoin Grants (https://gitcoin.co/grants/1113/citizen-cosmos-podcast) - Listen to the YouTube version () - Read our blog (https://citizen-cosmos.github.io/manuscripts/) - Check out our GitHub (https://github.com/citizenweb3) - Join our Telegram (https://t.me/citizen_web_3) - Follow us on Twitter (https://twitter.com/cosmos_voice) - Sign up to the RSS feed (https://www.citizencosmos.space/rss) Special Guest: Zenobia Godschalk.

Kevin and I Have a Talk about micro distributed systems Single condensing units hooked up to a glycol loop.

This episode of the #citizencosmos podcast features Edward FitzGerald Chief Technology Officer at Fetch.ai An “open” collective learning system that enables privacy-preserving collaborative machine learning and inferences on the Fetch.ai network Fetch.ai is a blockchain platform that uses artificial intelligence to help people automate everyday tasks such as booking a parking space or a flight We spoke to Edward (https://twitter.com/EFG_AI) about Fetch AI (fetch.ai) and: - Journey to the top of corporate - AI - Distributed systems - Full tolerance shenanigans - Being a cog in a larger machine - The importance of rapid decision making in business - The cost of making the wrong decisions - The beast that is decentralization - Mistakes will happen, but facilitates adaptation. - Proof of work as the most elegant solution - Actively seeking hard problems - Healthy Paranoia - Open source vs Closed source - The lifecycle of corporations - The power of the web3 structure - Critical safety systems - The difference between and dec ai - The advantage of tangible technology - Barriers of entry - Connectivity of autonomous services - Large Language models and new efficiencies - Architecture of the Agents - The birth of SkyNet - Affects of AI on the wider society - Hierarchy of abstraction - One big model to rule them all If you like what we do at Citizen Cosmos: - Stake with Citizen Cosmos validator (https://www.citizencosmos.space/staking) - Help support the project via Gitcoin Grants (https://gitcoin.co/grants/1113/citizen-cosmos-podcast) - Listen to the YouTube version (https://youtu.be/vj4dipYHIDY) - Read our blog (https://citizen-cosmos.github.io/manuscripts/) - Check out our GitHub (https://github.com/citizen-cosmos/Citizen-Cosmos) - Join our Telegram (https://t.me/citizen_cosmos) - Follow us on Twitter (https://twitter.com/cosmos_voice) - Sign up to the RSS feed (https://www.citizencosmos.space/rss) Special Guest: Edward FitzGerald.

Matteo Collina, Co-Founder and CTO of Platformatic, Node.js TSC member, and Lead Maintainer of Fastify, joins us again to talk about modular monoliths, his new backend development platform Platformatic, and many of his other projects including Pino. Links https://nodeland.dev https://platformatic.dev https://getpino.io https://www.linkedin.com/in/matteocollina https://twitter.com/matteocollina Tell us what you think of PodRocket We want to hear from you! We want to know what you love and hate about the podcast. What do you want to hear more about? Who do you want to see on the show? Our producers want to know, and if you talk with us, we'll send you a $25 gift card! If you're interested, schedule a call with us (https://podrocket.logrocket.com/contact-us) or you can email producer Kate Trahan at kate@logrocket.com (mailto:kate@logrocket.com) Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket combines frontend monitoring, product analytics, and session replay to help software teams deliver the ideal product experience. Try LogRocket for free today. (https://logrocket.com/signup/?pdr) Special Guest: Matteo Collina.