Podcasts about Internals

Nesse episódio trouxemos as notícias e novidades do mundo da programação que nos chamaram atenção dos dias 17/08 a 23/08.

Nesse episódio trouxemos as notícias e novidades do mundo da programação que nos chamaram atenção dos dias 17/08 a 23/08.

Speaker: Pastor Armstrong Passage: Matthew 7:1-5

CIO Scott Glasser and Head of Strategy Jeff Schulze see the economy slowing but avoiding recession in the second half of 2024 into 2025. Meanwhile, the stock market remains healthy yet bears watching due to deterioration beyond the mega caps. They also explain how these conditions create compelling opportunities for active managers.

In this interview with Rosa Gutierrez of 37signals, Rosa elaborates on her work, particularly her contributions to Solid Queue and its implementation alongside Active Job. She describes the motivations behind developing Solid Queue, addressing performance issues with existing systems like Resque, and transitioning some workloads to database-backed queues. Rosa also explains the technical details of Solid Queue's architecture, including supervisor and dispatcher roles, how it handles job prioritization, concurrency, and the challenges of ensuring performance in high-load environments. Essential advice is provided on when to use or avoid Solid Queue, depending on specific needs and existing setups.Solid Queue37signals

Trading full-time since 1999. He's been a principal and head trader for RBC capital and Translucent Capital Management. Now he's a retail trader trading weekly options on a strict technical basis. Founder of Shadowtrader.net. Confessions of a Market Maker presents Peter Reznicek.------------------------------------------------------------------------------------------------------------------https://www.shadowtrader.net/Peter on Twitter: https://x.com/PeterReznicekJJ on Twitter: https://x.com/vwaptrader1Ray on Twitter: https://x.com/AllxDayxRayx------------------------------------------------------------------------------------------------------------------Microefutures.com trading room is now offering a 7 days free trial with no obligations. Offering not only ES where JJ teaches and trades, but other futures instruments, stocks, crypto, DFS and sports betting. A 7 day free trial with no obligation or commitment. Come join us at mircoefutures.com

 In today's episode, we dive into the market's cautious stance ahead of the latest look at inflation with tomorrow's CPI data. Tyler highlights the positive internal market metrics juxtaposed against mixed external signs and notes the standout performance of small caps and tech sectors. 

https://backend.win https://databases.win I'm a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows.  Everything goes to pages of fixed size and those pages are flushed to disk.  Each database define page size differently based on their workload, for example MongoDB default page size is 32KB, MySQL InnoDB is 16KB and PostgreSQL is 8KB. The trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API.   In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills. Let us get started.

I. The Signs of a False Teacher They have a DESIRE w/out discipleship - v. 7a They have a CALLING w/out Character & Content - v. 7b They have the PRIDE of Position and Prominence They PRIORITIZE the Externals bc they lack the Internals - They want to BE SERVED rather than Be Servants They see God's Sheep as RESOURCES for riches - They love money and use people They LACK a Love for God's Law - 1:5-6-11 They LACK a Christ-Centered Testimony of Humility - 12-15 They LACK a High View of God - 16-17 They MUST BE silenced & removed The Purpose of the Law & The Parallel of Paul's Testimony! (The Law does not save us - cannot save; The Law can and does sanctify us - Obedience is what makes our saving faith grow - Matthew 7:24-27; James 1:22) II. The Law is: Notice the adoration of the Law - the action of the Law - the attitude we should have of the Law! good - 1:8 not for the just - 1:9a for the ungodly- 1:9b bad news part of the gospel - 1:11 hushes the self-righteous - Romans 3:19-20 holy - Romans 7:12 helpful - Galatians 3:24-25 delightful (full of) - Psalm 1:2 perfect - Psalm 19:7a restorative - Psalm 19:7b wonderful (full of) - Ps. 119:18 truthful (full of) - Ps. 119:29, 42 obeyed - Psalm 119:34, 44 derided - Psalm 119:51, 61 bothersome - Psalm 119:54 valuable - Psalm 119:72 merciful (full of) - Ps. 119:77 lovable - Psalm 119:97a contemplative - Psalm 119:97b straightforward - Ps. 119:113 emotional - Psalm 119:136 anchoring - Psalm 119:153 peaceful (full of) - Ps. 119:165

(2/8/24) The S&P set another, new all-time high on Wednesday, closing just short of the psychologically-important 5,000-mark (see article link below). Market internals, however, are showing a very different view than headline numbers would infer. The number of stocks above their 50-week moving average is getting lower; the number of stocks above their 200-DMA is turning lower, showing a deterioration in participation within the market even though the index is rising. Thank you, top-ten stocks dragging about 35% of passive inflows along for the ride. The problem is, the rest of the market is selling off, and that's a deviation worth paying attention to. Hosted by RIA Chief Investment Strategist, Lance Roberts, CIO  Produced by Brent Clanton, Executive Producer ------- Articles mentioned in this report: "S&P Index Set To Hit 5000 As Bull Run Continues" https://realinvestmentadvice.com/sp-index-set-to-hit-5000-as-bull-run-continues/ ------- Register for our next Candid Coffee: "Five Money Habits of Unhappy Couples," Saturday, February 24, 2024: https://streamyard.com/watch/TJz2bu2Xn9Hu --------  Get more info & commentary:  https://realinvestmentadvice.com/insights/real-investment-daily/ ------- Watch the video version of this report by subscribing to our new "Before the Bell" YouTube channel:  https://www.youtube.com/watch?v=GbDq654nLvI&list=PLwNgo56zE4RAbkqxgdj-8GOvjZTp9_Zlz&index=1 ------- Visit our Site: https://www.realinvestmentadvice.com Contact Us: 1-855-RIA-PLAN -------- Subscribe to SimpleVisor: https://www.simplevisor.com/register-new -------- Connect with us on social: https://twitter.com/RealInvAdvice https://twitter.com/LanceRoberts https://www.facebook.com/RealInvestmentAdvice/ https://www.linkedin.com/in/realinvestmentadvice/ #InvestingAdvice #SP5000 #NASDAQ #Russell2000 #SmallCap #MidCap #MarketRally #MarketDivergence #Markets #Money #Investing 

Jesús Espino is a Senior Staff Engineer at Mattermost specializing in Full Stack Development on the API side of the platform. Mattermost is an open source platform for secure collaboration across the entire software development lifecycle. Mattermost's platform powers over 800,000 workspaces worldwide with the support of over 4,000 contributors from across the developer community.  In this episode, Jesús takes us on a journey through his time in the tech industry while sharing valuable insight and entertaining stories along the way.00:00 Introduction05:29 What is Jesús Doing Today? 10:50 First Memory of a Computer17:32 Thoughts on University25:21 Joining the Workforce33:03 First Job after University47:08 Moving Companies51:30 Transitioning to Go58:40 Deciding to Focus on Go1:03:40 Solving Problems with Internals of Go1:11:50 Thoughts on Mattermost1:13:16 Contact InfoConnect with Jesús: Twitter: https://twitter.com/LinkedIn: https://www.linkedin.com/in/jesus-espinoMentioned in today's episode:Mattermost: https://mattermost.com/Gophercon EU: https://gophercon.eu/Want more from Ardan Labs? You can learn Go, Kubernetes, Docker & more through our video training, live events, or through our blog!Online Courses : https://ardanlabs.com/education/ Live Events : https://www.ardanlabs.com/live-training-events/ Blog : https://www.ardanlabs.com/blog Github : https://github.com/ardanlabs

Deep Dive into Victoria Metrics with Alex and Roman Join the insightful discussion with Vitoriametrics creators, Alex and Roman, in the Geekneritor podcast hosted by Kaivalya Apte. This episode explores the internals of Victoria Metrics - a highly scalable monitoring solution and time series database. Discover the origins of Victoria Metrics, understand how it evolved, and learn about its unique architecture and functionality. From the concept of time series, the usage of consistent hashing in data distribution to real-world applications, it's all packed into this engaging conversation. 00:00 Introduction 01:52 The Genesis of VictoriaMetrics 02:18 The Journey from Postgres to Clickhouse 03:19 The Transition from Prometheus to Victoria Metrics 05:08 The Birth and Evolution of Victoria Metrics 13:01 The Architecture of Victoria Metrics 20:10 Data Ingestion and Integration in Victoria Metrics 29:15 Understanding the Vector Metric Architecture 30:30 Comparing Shared Storage and Object Store 31:00 Designing the VictoriaMetrics Architecture 32:01 The Role of Object Storage 36:15 The Importance of Indexing 43:19 Understanding the Ingestion Process 45:46 Exploring the Select Process 55:55 Future Plans for Victoria Metrics Important Links: 1. Architecture Overview: https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#architecture-overview 2. How ClickHouse Inspired Us to Build a High Performance Time Series Database https://altinity.com/wp-content/uploads/2021/11/How-ClickHouse-Inspired-Us-to-Build-a-High-Performance-Time-Series-Database.pdf 3. Frequently asked questions. https://docs.victoriametrics.com/FAQ.html =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

Join us on a deep dive into the intricacies of TiDB with Li Shen from PingCap. In this episode, Li Shen provides a comprehensive exploration of TiDB, its unique features, and how it tackles scalability and reliability issues commonly associated with MySQL. If you're dealing with struggles in your MySQL cluster and seeking a more dependable and scalable system, TiDB might be the solution for you. This conversation touches on various aspects of this cutting-edge database, its operational mechanism, use case scenarios, and how it's optimized for different workloads. Key topics include: the architecture of TiDB, the journey of data from API to storage node, embracing analytical use cases, the importance of database reliability, and the process of migrating to TiDB. Dive in now! 00:00 Introduction and Welcome 02:47 Defining TIDB: A Disputed SQL Database 04:55 The Role of MySQL Compatibility in TIDB 05:54 Primary Use Cases for TIDB 09:38 Understanding the Data Ingestion Process in TIDB 16:52 Understanding Indexing in TIDB 23:01 Pushing Down Table Scans and Partial Aggregation 24:39 Introduction to Columnary Extension: Flash 24:54 Understanding Data Replication and Learner Nodes 26:23 Ensuring Strong Consistency in Data 27:12 Balancing Transactional and Analytical Use Cases 27:57 Understanding Data Replication and Consistency Model 28:42 Exploring Ty Flash Storage Layer 28:54 Understanding High Concurrency Insert and Update 32:09 Exploring the Read Path and Caching Mechanism 37:50 Understanding the Importance of High Reliability 43:01 Exploring Migration from Other Databases 48:01 Comparing TiDB with Other Distributed SQL Databases 52:21 Identifying Use Cases Where TiDB Might Not Be the Best Choice Stay Curios! Keep Learning!

Deep Dive into DuckDB with CTO Mark Raasveldt Decode the insights of databases with Geek Narrator podcast. In this episode, host Kaivalya Apte converses with Mark Raasveldt, the CTO of DuckDB labs, discussing his journey from being a database enthusiast to creating DuckDB. They delve into how DuckDB, an analytical database, differs from other databases, the design decisions, its internal mechanisms, and much more. The episode also highlights the advantages of DuckDB in analytics, the motivation behind its ACID compliance, and how DuckDB handles ingestion, transaction isolation, mutations, and queries. Join in to learn how your data workloads can benefit from DuckDB. 00:00 Introduction and Guest Introduction 00:44 Guest's Journey into Databases 03:40 The Birth of DuckDB 04:30 Challenges with Existing Databases 05:15 Technical Difficulties 05:16 Why Existing Databases Fall Short for Data Scientists 09:16 The Role of SQLite and Its Limitations 13:59 Defining DuckDB 16:48 Comparing DuckDB with Other Analytical Databases 19:50 Deployment Models for DuckDB 22:47 Data ingestion into DuckDB 22:51 Data Ingestion in DuckDB 30:24 How DuckDB Handles Updates and Mutations 35:35 Understanding Column Granularity and Rewrites 35:58 Implications of Compression on Data Updates 36:38 Trade-offs in Row Group Size 37:32 Benefits of Column Storage Model 38:15 Row Groups and Parallelism 39:02 Choosing Row Group Size: An Experimental Approach 40:00 Handling Data Type Changes in Columns 41:00 Internal Data Structures in DuckDB 42:21 Reading Data: Point Lookups, Aggregations, and Joins 47:22 Optimization for Full Table Scans 53:49 Understanding ACID Compliance in DuckDB 55:49 Multi-Version Concurrency Control (MVCC) in DuckDB 59:50 Use Cases and Applications of DuckDB 01:01:42 The Story Behind DuckDB's Name 01:02:34 Future Vision for DuckDB References: DuckDB: https://duckdb.org/ Mark's blog: https://mytherin.github.io/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

In this episode we talk about ScyllaDB internals with Felipe Mendes. Chapters: 0:00 ScyllaDB internals with Felipe Mendes 07:51 Write Path - API to Storage 11:40 What makes it faster than Cassandra? 13:39 Optimisations: Sea Star, shard per core architecture 15:49 Optimisations: No Garbage collection and Custom Cache Implementation 18:15 Optimisations: Scheduling groups and IO priority classes 20:07 Optimisations: IO scheduler 22:55 Benefits of shard per core architecture 30:16 Write path - Hows is a coordinator chosen? 38:20 Read path 39:27 Read path optimisations - Index Caching 41:48 Shard vs Partition 43:10 Shard per core architecture tradeoff 44:03 Observability of Database References: ScyllaDB architecture: https://opensource.docs.scylladb.com/stable/architecture/ Sea star: https://seastar.io/ ScyllaDB Caching: https://www.scylladb.com/2018/07/26/how-scylla-data-cache-works/ Shard per core architecture: https://www.scylladb.com/product/technology/shard-per-core-architecture/ Database performance at Scale: https://www.scylladb.com/2023/10/02/introducing-database-performance-at-scale-a-free-open-source-book/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

In this episode I talk to Michael Hunger from Neo4j about Graph Database Internals (Neo4J) Chapters: 0:00 Introduction and historical context 20:51 Data Modelling 25:16 Problem with SQL for Graph Model 26:21 Cypher - Query Language 28:23 Write Path 31:36 Neo4J Storage Layer 33:51 Graph API on top of Relational Model vs Native Graph Databases 37:05 Create Node Relationships 40:42 What makes Graph Database's performance better? 46:00 Partitioning Strategy 53:20 Read path 59:27 Schema Migration 01:04:41 Graph database use cases =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

Hey Everyone, In this video I talk to Franck Pachot about internals of YugabyteDB. Franck has joined the show previously to talk about general database internals and its again a pleasure to host him and talk about DistributedSQL, YugabyteDB, ACID properties, PostgreSQL compatibility etc. Chapters: 00:00 Introduction 01:26 What does Cloud Native means? 02:57 What is Distributed SQL? 03:47 Is DistributedSQL also based on Sharding? 05:44 What problem does DistributedSQL solves? 07:32 Writes - Behind the scenes. 10:59 Reads: Behind the scenes. 17:01 BTrees vs LSM: How is the data written do disc? 25:02 Why RocksDB? 29:52 How is data stored? Key Value? 33:56 Transactions: Complexity, SQL vs NoSQL 42:51 MVCC in YugabyteDB: How does it work? 45:08 Default Transaction Isolation level in YugabyteDB 51:57 Fault Tolerance & High Availability in Yugabyte 56:48 Thoughts on Postgres Compatibility and Future of Distributed SQL 01:03:53 Usecases not suitable for YugabyteDB Previous videos: Database Internals: Part1: https://youtu.be/DiLA0Ri6RfY?si=ToGv9NwjdyDE4LHO Part2: https://youtu.be/IW4cpnpVg7E?si=ep2Yb-j_eaWxvRwc Geo Distributed Applications: https://youtu.be/JQfnMp0OeTA?si=Rf2Y36-gnpQl18yj Postgres Compatibility: https://youtu.be/2dtu_Ki9TQY?si=rcUk4tiBmlsFPYzY I hope you liked this episode, please hit the like button and subscribe to the channel for more. Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Franck's Twitter and Linkedin: https://twitter.com/FranckPachot and https://www.linkedin.com/in/franckpachot/ Connect and follow here: https://twitter.com/thegeeknarrator and https://www.linkedin.com/in/kaivalyaapte/ Keep learning and growing. Cheers, The GeekNarrator

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Impact stories for model internals: an exercise for interpretability researchers, published by Jenny Nitishinskaya on September 25, 2023 on The AI Alignment Forum. Inspired by Neel's longlist; thanks to @Nicholas Goldowsky-Dill and @Sam Marks for feedback and discussion, and thanks to AWAIR attendees for participating in the associated activity. As part of the Alignment Workshop for AI Researchers in July/August '23, I ran a session on theories of impact for model internals. Many of the attendees were excited about this area of work, and we wanted an exercise to help them think through what exactly they were aiming for and why. This write-up came out of planning for the session, though I didn't use all this content verbatim. My main goal was to find concrete starting points for discussion, which have the right shape to be a theory of impact are divided up in a way that feels natural cover the diverse reasons why people may be excited about model internals work (according to me). This isn't an endorsement of any of these, or of model internals research in general. The ideas on this list are due to many people, and I cite things sporadically when I think it adds useful context: feel free to suggest additional citations if you think it would help clarify what I'm referring to. Summary of the activity During the session, participants identified which impact stories seemed most exciting to them. We discussed why they felt excited, what success might look like concretely, how it might fail, what other ideas are related, etc. for a couple of those items. I think categorizing existing work based on its theory of impact could also be a good exercise in the future. I personally found the discussion useful for helping me understand what motivated some of the researchers I talked to. I was surprised by the diversity. Key stats of an impact story Applications of model internals vary a lot along multiple axes: Level of human understanding needed for the application If a lot of human understanding is needed, does that update you on the difficulty of executing in this direction? If understanding is not needed, does that open up possibilities for non-understanding-based methods you hadn't considered? For example, determining whether the model does planning would probably require understanding. On the other hand, finding adversarial examples or eliciting latent knowledge might not involve any. Level of rigor or completeness (in terms of % model explained) needed for the application If a high level of rigor or completeness is needed, does that update you on the difficulty of executing in this direction? What does the path to high rigor/completeness look like? Can you think of modifications to the impact story that might make partial progress be more useful? For example, we get value out of finding adversarial examples or dangerous capabilities, even if the way we find them is somewhat hacky. Meanwhile, if we don't find them, we'd need to be extremely thorough to be sure they don't exist, or sufficiently rigorous to get a useful bound on how likely the model is to be dangerous. Is using model internals essential for the application, or are there many possible approaches to the application, only some of which make use of model internals? Steering model behaviors can be done via model editing, or by prompting or finetuning; but, there are reasons (mentioned below) why editing could be a better approach. Many impact stories (at least as I've categorized them) have variants that live at multiple points on these spectra. When thinking about one, you should think about where it lands, and what variants you can think of that might be e.g. easier but still useful. The list Some are more fleshed out than others; some of the rest could be fleshed out with a bit more effort, while others are more ...

In this video I talk to David Jacot who works as a Staff Software Engineer at  @Confluent  and has been a long time Kafka user, committer and PMC member. We covered how Kafka works internally in great depth. We use Kafka for various use cases and it works great, but going one level below the abstraction and truly understanding the protocols, techniques and algorithms used is a fun ride. Chapters: 00:00 Kafka Internals with David Jacot 03:33 Defining Kafka 05:16 Kafka Architecture(s) 11:39 Write Path - Producer sending data 18:35 How does replication work? 25:47 How do we track replication progress? 30:42 Failure Modes: Leader fails 38:18 Consumers: Push vs Pull 40:54 Consumers: How does fetch works? 49:03 Consuming number of bytes vs records 50:50 Optimising consumption 01:00:21 Offset management and choosing partitions 01:09:10 Ending notes I hope you like this episode and more importantly you learnt some amazing techniques Kafka uses to ensure durability, low latency, simplicity and scalability in its architecture. Do give this episode a like and share it with your network. Also please subscribe to the channel for content like this. Other playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Other episodes: KsqlDB: https://youtu.be/2yE86P6uD_0 Exactly once semantics: https://youtu.be/twgbAL_EaQw David's Linkedin: https://www.linkedin.com/in/davidjacot/ our website: www.geeknarrator.com Cheers, The GeekNarrator

This is a recap of the top 10 posts on Hacker News on August 12th, 2023.Wondercraft HN Launch: https://news.ycombinator.com/item?id=37088087(00:42): The Carrot ProblemOriginal post: https://news.ycombinator.com/item?id=37100226&utm_source=wondercraft_ai(02:27): Exploring the Internals of Linux v0.01Original post: https://news.ycombinator.com/item?id=37101588&utm_source=wondercraft_ai(04:15): A conversation with a newspaper owner raided by copsOriginal post: https://news.ycombinator.com/item?id=37102271&utm_source=wondercraft_ai(06:03): tRPC – Build and consume typesafe APIs without schemas or code generationOriginal post: https://news.ycombinator.com/item?id=37098875&utm_source=wondercraft_ai(07:54): Deep Learning SystemsOriginal post: https://news.ycombinator.com/item?id=37101515&utm_source=wondercraft_ai(09:31): Auditory illusions with examples from Daft PunkOriginal post: https://news.ycombinator.com/item?id=37103802&utm_source=wondercraft_ai(10:00): My $0->$100M->$0 in 5 years storyOriginal post: https://news.ycombinator.com/item?id=37102442&utm_source=wondercraft_ai(11:45): 20 Years of VimOriginal post: https://news.ycombinator.com/item?id=37099750&utm_source=wondercraft_ai(13:51): I feel like I made a mistake investing professionally into FlutterOriginal post: https://news.ycombinator.com/item?id=37098591&utm_source=wondercraft_ai(15:21): Illegal raids contribute to death of newspaper co-ownerOriginal post: https://news.ycombinator.com/item?id=37105764&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

In this episode, I had the pleasure of speaking with Oren Eini about RavenDB and diving deep into the world of database internals. We explore the intricacies of how databases work under the hood and discuss the challenges - especially around performance.Oren Eini, also known as Ayende Rahien, is the Founder and CEO of RavenDB, a leading NoSQL document database management platform, empowering both startups and Fortune 500 companies. With over 20 years of software development experience and business leadership, Oren is an internationally acclaimed presenter, a thought leader and an author, committed to continue and lead innovation of quality software and best practices. RavenDB delivers fully transactional (ACID), smart & agile, cutting-edge NoSQL end-to-end solutions, empowering businesses in efficient data management.For a full list of show notes, or to add comments - please see the website here

Our fifth season dives into large language models (LLMs), from understanding the internals to the risks of using them and everything in between. While we're at it, we'll be enjoying our morning brew.In this session, we interviewed Chengyin Eng (Senior Data Scientist, Databricks), Sam Raymond (Senior Data Scientist, Databricks), and Joseph Bradley (Lead Production Specialist - ML, Databricks) on the best practices around LLM use cases, prompt engineering, and how to adapt MLOps for LLMs (i.e., LLMOps).

This Week's episode is about Windows Internals in depth, we've talked about things from an offensive and defensive perspective. Things like Hooks, Kernel callbacks, how security companies are using them and how Red Teamers are leveraging them as well. We've talked about many other concepts such as user space mode and kernel space mode, Patch Guard and many others. This episode is part of The Cyber Riddler podcast, Check out the other episodes on any of your favorite podcast apps. Just search the name "The Cyber Riddler". Not all of the episodes are in YouTube. links belowFull Interview below:https://www.youtube.com/watch?v=7pQpc1g7focPavel's YouTube Channelhttps://www.youtube.com/@zodiacon https://thecyberriddler.com/blog/windows-internalsTwitter: @almorabeaTwitter: @CyberRiddlerWebsite: https://thecyberriddler.com

Nikolay and Michael celebrate the podcast's 1 year anniversary by going through several questions and suggestions received over the year. Here are the questions and some links to things we mentioned: Question 1: Effect of wal_log_hints=on after bulk Deletes: Why next select runs slow and generated tons of WAL? https://twitter.com/dmx551/status/1598253188926570496 wal_log_hints https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-WAL-LOG-HINTS Exploring how SELECT Queries can produce disk writes https://blog.okmeter.io/postgresql-exploring-how-select-queries-can-produce-disk-writes-f36c8bee6b6fQuestion 2: How to get started reading PostgreSQL source code. Maybe a PostgreSQL style C reference guide to consult with for non C programmers https://twitter.com/andatki/status/1578088843940593678 So, you want to be a developer? https://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer%3FGitHub search https://github.com/search?q=repo%3Apostgres%2Fpostgres+wal_log_hints&type=code The Internals of PostgreSQL (by Hironobu SUZUKI) https://www.interdb.jp/pg/ PostgreSQL 14 Internals (by Egor Rogov) https://postgrespro.com/community/books/internals Question 3: Isolation Levels Primer/Strategies — their uses in different scenarios, battle tested strategies and insights, performance tradeoffs, edge cases to consider at scale (with replication and sharding, etc.) I remember reading some interesting stuff on the jepsen analysis https://jepsen.io/analyses/postgresql-12.3 about Postgres's Serializable Isolation Level behaving more like Snapshot Isolation. Has this type of behavior or another one similar to this affected you or your clients in any significant way?Transaction Isolation https://www.postgresql.org/docs/current/transaction-iso.htmlWhat developers find surprising about Postgres transactions https://blog.lawrencejones.dev/isolation-levels/  Question 4: Data encryption in PostgresCybertec Transparent Data Encryption https://www.cybertec-postgresql.com/en/products/postgresql-transparent-data-encryption/ EDB Transparent Data Encryption https://www.enterprisedb.com/docs/tde/latest/Question 5: Migration from other DBMSsPostgreSQL transition/migration guide https://github.com/postgresql-transition-guide/guide  Question 6: Latest failover best practicesPatroni https://github.com/zalando/patroni~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork 

Noah Weiss is the Chief Product Officer of Slack, overseeing the product team's strategy and development. Over his seven years at Slack, Noah has led various parts of the product organization, including the self-service SMB business and product-led growth; the Virtual HQ team that launched huddles and clips; and the search and machine learning teams. Prior to Slack, Noah served as SVP of Product and Analytics at Foursquare. He started his career at Google leading the structured data search team and working on display ads. In Today's Episode with Noah Weiss We Discuss: 1.) Entry into Product and Road to Slack CPO: How did Noah make his first foray into the world of product with Google? What are 1-2 of his single biggest takeaways from his time with Google and Foursquare? What model did Noah learn at Google that he applies to product today? 2.) Product 101: The Foundations: Is product more art or science? If Noah were to put a number on it what would it be? What are product principles? What makes good vs bad product principles? What are the biggest mistakes that founders make when instilling product principles? Does Noah believe with Gustav Soderstrom, "talk is cheap and so we should do more of it"? 3.) How to Master Product-Led-Growth: What are some of Noah's biggest lessons on how to master PLG? What are the biggest mistakes Noah sees early stage founders make today when going for the PLG approach? How does he advise them? When is the right time to move into enterprise? What needs to change? How do you change who you build product for? The buyer or the user? Why does Noah believe product speed will always be the most important thing in product? 4.) The Internals of Slack: How does Slack do post-mortems today? Who comes? Who sets the agenda? How has this changed in a world of remote? What does it take to do them well? How do Slack do product testing pre-launch of new products? Do they know when something is going to be a hit? What did they think would be a massive hit that turned into a flop? What does Noah believe is the biggest near death product experience for Slack? What happened? How did they get through it? Why do Slack buy other companies? How do they think through the decision of buy vs build? When do acquisitions work? When do they not work?

Link to slide deck: https://bit.ly/3ILNoZJ   - Today, we note the extreme divergence in equity market themes over the last week that is a hidden development underneath the strong performance of the major US indices on Friday. We look at the moderate impact thus far of the debt ceiling agreement-in-principle reached at the weekend, discuss the strong earnings season and few interesting names set to report this week. Energy and the European natural gas market are in focus in commodities while we also try to gauge where we are in terms of timing an eventual recession with US yield-curve and consumer confidence survey indicators. This and more in today's, which features Peter Garnry on equities, Ole Hansen on commodities, with John J. Hardy hosting and on FX. Read daily in-depth market updates from the Saxo Market Call and SaxoStrats Market Strategy Team here. Click here to open an account with Saxo   - Intro and outro music by AShamaluevMusic

What's with the tension? Aren't we trying to accomplish the same thing?   There's nothing more counterproductive in all of hiring than the adversarial relationship that (some) internal and external recruiters have. Internals begrudgingly use externals on hard searches when they don't really want to pay a fee. Externals angle around internals when they don't feel they're on a level playing ground.   It's silly. But what causes it? And why is it so unique to the talent acquisition space compared to other areas of professional services?   Jeff Smith and James Hornick will dig into the insanity and provide a more productive path forward in The 10 Minute Talent Rant, Episode 65 "Can internal and external recruiters be best friends?"  

Follow: https://stree.ai/podcast | Sub: https://stree.ai/sub | New episodes every Monday! Who remembers taxis? They were these yellow cars that appeared at random with seemingly inexplicable service charges. You may be more familiar with taxi 2.0: Uber - a platform fully powered by real-time analytics. In this episode, Tim sits down with Rong Rong (Software Engineer, StarTree) to talk about how Uber made use of Apache Pinot, when he came over to StarTree, as well as a deep dive into some Apache Pinot internals.

2023 - 03 - 25 - Men's Weekend - Lonny Jones - Internals Vs Externals by Buford Church of Christ

...in which I update you on life right now HINT IT'S A SEASON OF DEATH and on what grief looks like, and why literally ANY day of being 27 years old was harder for me than the day I put my mom in the ground. Upbeat episode? Nope. As real as it gets from beginning to end? Absolutely. Hosted on Acast. See acast.com/privacy for more information.

TCW Podcast Episode 179 - Space Invaders and Nishikado   We look at the life and career of Tomohiro Nishikado, and the life influences that led to his creation of one of the most influential games ever made. Space Invaders! Starting life at the tail end of World War II, Nishikado always had a rebellious streak wanting to get into electronics then follow his mother into education. After some difficulty with school he eventually graduated from Denki University with an electronics degree, during this time, this time he fell in love with coin-op arcade games. Eventually through the help of a friend he landed a job at Taito's subsidiary, Pacific Amusements. Despite not liking his placement at the company, Nishikado continued to work there out of loyalty for his friend. He eventually worked on Sky Fighter, Basketball, and Super Road 7.  Eventually as the industry shifted to microprocessors, and TTL  hardware he eventually made Space Invaders!   TCW 077 - Invading Taito Part 1: http://podcast.theycreateworlds.com/e/invading-taito-part-1/ TCW 078 - Invading Taito Part 2:  http://podcast.theycreateworlds.com/e/invading-taito-part-2 Sky Fighter II: https://www.youtube.com/watch?v=RxpCrYBWdtk Taito Basketball: https://www.youtube.com/watch?v=AiTrWumX00s Super Road 7: https://www.youtube.com/watch?v=vO6BZIEZ8xw Super Road 7 Internals: https://www.youtube.com/watch?v=ja8x_ZCklg0 Speed Race US and Japan Ver: https://www.youtube.com/watch?v=5jc9_xrD2_M Western Gun: https://www.youtube.com/watch?v=XbnYBB42J6c Interceptor: https://www.youtube.com/watch?v=50zEbS2cgEc Midway Sea Wolf: https://www.youtube.com/watch?v=ipssJjkN2bY Space Invaders: https://www.youtube.com/watch?v=DJMr36hLY38 Space Invaders Cocktail: https://www.youtube.com/watch?v=V3hyLWCuMew   New episodes are on the 1st and 15th of every month!   TCW Email: feedback@theycreateworlds.com  Twitter: @tcwpodcast Patreon: https://www.patreon.com/theycreateworlds Alex's Video Game History Blog: http://videogamehistorian.wordpress.com Alex's book, published Dec 2019, is available at CRC Press and at major on-line retailers: http://bit.ly/TCWBOOK1     Intro Music: Josh Woodward - Airplane Mode -  Music - "Airplane Mode" by Josh Woodward. Free download: http://joshwoodward.com/song/AirplaneMode  Outro Music: RolemMusic - Bacterial Love: http://freemusicarchive.org/music/Rolemusic/Pop_Singles_Compilation_2014/01_rolemusic_-_bacterial_love    Copyright: Attribution: http://creativecommons.org/licenses/by/4.0/

Here are links to a few things we mentioned: The Internals of PostgreSQL chapter 5 (book by Hironobu SUZUKI) PostgreSQL 14 internals chapter 7 (book by Egor Rogov)  Transaction ID Wraparound (blog post from Sentry) What We Learned from the Recent Mandrill Outage (blog post from Mailchimp) How to simulate transaction ID wraparound (blog post by Prashant Dixit)Add 64-bit XIDs into PostgreSQL 16 (commitfest entry)  Partitioning (docs)  Consider using VACUUM's INDEX_CLEANUP option (tweet by Peter Geoghan) Add wraparound failsafe to VACUUM (commit) Do you vacuum everyday? (talk by Hannu Krosing) Multixacts wraparound monitoring (mailing list thread) Subtransactions Considered Harmful (blog post by Nikolay) Buffer management in PostgreSQL (talk by Alexander Korotkov) OrioleDB pageinspectpg_hexedit  pg_visibility  Visibility Map (docs)------------------------What did you like or not like? What should we discuss next time? Let us know by tweeting us on @samokhvalov / @michristofides / @PostgresFM, or by commenting on our Google doc.If you would like to share this episode, here's a good link (and thank you!)Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork 

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Basic Facts about Language Model Internals, published by Beren Millidge on January 4, 2023 on The AI Alignment Forum. This post was written as part of the work done at Conjecture. As mentioned in our retrospective, while also producing long and deep pieces of research, we are also experimenting with a high iteration frequency. This is an example of this strand of our work. The goal here is to highlight interesting and unexplained language model facts. This is the first in a series of posts which will be exploring the basic ‘facts on the ground' of large language models at increasing levels of complexity. Understanding the internals of large-scale deep learning models, and especially large language models (LLMs) is a daunting task which has been relatively understudied. Gaining such an understanding of how large models work internally could also be very important for alignment. If we can understand how the representations of these networks form and what they look like, we could potentially track goal misgeneralization, as well as detect mesaoptimizers or deceptive behaviour during training and, if our tools are good enough, edit or remove such malicious behaviour during training or at runtime. When faced with a large problem of unknown difficulty, it is often good to first look at lots of relevant data, to survey the landscape, and build up a general map of the terrain before diving into some specific niche. The goal of this series of works is to do precisely this – to gather and catalogue the large number of easily accessible bits of information we can get about the behaviour and internals of large models, without commiting to a deep dive into any specific phenomenon. While lots of work in interpretability has focused on interpreting specific circuits, or understanding relatively small pieces of neural networks, there has been relatively little work in extensively cataloging the basic phenomenological states and distributions comprising language models at an intermediate level of analysis. This is despite the fact that, as experimenters with the models literally sitting in our hard-drives, we have easy and often trivial access to these facts. Examples include distributional properties of activations, gradients, and weights. While such basic statistics cannot be meaningful ‘explanations' for network behaviour in and of themselves, they are often highly useful for constraining one's world model of what can be going on in the network. They provide potentially interesting jumping off points for deeper exploratory work, especially if the facts are highly surprising, or else are useful datapoints for theoretical studies to explain why the network must have some such distributional property. In this post, we present a systematic view of basic distributional facts about large language models of the GPT2 family, as well as a number of surprising and unexplained findings. At Conjecture, we are undertaking follow-up studies on some of the effects discussed here. Activations Are Nearly Gaussian With Outliers If you just take the histogram of activity values in the residual stream across a sequence at a specific block (here after the first attention block), they appear nearly Gaussianly distributed. The first plot shows the histogram of the activities of the residual stream after the attention block in block 0 of GPT2-medium. This second plot shows the histogram of activities in the residual stream after the attention block of layer 10 of GPT2-medium, showing that the general Gaussian structure of the activations is preserved even deep inside the network. This is expected to some extent due to the central limit theorem (CLT), which enforces a high degree of Gaussianity on the distribution of neuron firing rates. This CLT mixing effect might be expected to destroy information in t...

Hey Everyone, This is the part2 of our episode with Franck Pachot. In this episode I talk to Franck Pachot from YugabyteDB who has vast experience in the world of Databases. We have talked about how modern databases work internally. BTrees, LSM trees and various design choices. Cheers, The GeekNarrator

Hey Everyone, This is the part1 of our episode with Franck Pachot. In this episode I talk to Franck Pachot from YugabyteDB who has vast experience in the world of Databases. We have talked about how modern databases work internally. BTrees, LSM trees and various design choices. Cheers, The GeekNarrator

YCharts recently published a fantastic report that details the advisor-wholesaler relationship. This report is applicable for Heads of Distribution, National Sales Managers, Divisional Managers, Externals and Internals alike.The host highlights a few key survey results, and offers an opinion on how these data points might impact distribution professionals. Disclaimer: the opinions expressed by the host are purely his own and are not affiliated with YCharts or any other entity. Click here to download the full report Start Your FREE 7-Day YCharts Trial 

What happens when you need to store more than a few petabytes of data? Rittika Adhikari (Software Engineer, Confluent) discusses how her team implemented tiered storage, a method for improving the scalability and elasticity of data storage in Apache Kafka®. She also explores the motivating factors for building it in the first place: cost, performance, and manageability. Before Tiered Storage, there was no real way to retain Kafka data indefinitely. Because of the tight coupling between compute and storage, users were forced to use different tools to access cold and hot data. Additionally, the cost of re-replication was prohibitive because Kafka had to process large amounts of data rather than small hot sets.As a member of the Kafka Storage Foundations team, Rittika explains to Kris Jenkins how her team initially considered a Kafka data lake but settled on a more cost-effective method – tiered storage. With tiered storage, one tier handles elasticity and throughput for long-term storage, while the other tier is dedicated to high-cost, low-latency, short-term storage. Before, re-replication impacted all brokers, slowing down performance because it required more replication cycles. By decoupling compute and storage, they now only replicate the hot set rather than weeks of data. Ultimately, this tiered storage method broke down the barrier between compute and storage by separating data into multiple tiers across the cloud. This allowed for better scalability and elasticity that reduced operational toil. In preparation for a broader rollout to customers who heavily rely on compacted topics, Rittika's team will be implementing tier compaction to support tiering of compacted topics. The goal is to have the partition leader perform compaction. This will substantially reduce compaction costs (CPU/disk) because the number of replicas compacting is significantly smaller. It also protects the broker resource consumption through a new compaction algorithm and throttling. EPISODE LINKSJun Rao explains: What is Tiered Storage?Enabling Tiered StorageInfinite Storage in Confluent PlatformKafka Storage and Processing FundamentalsKIP-405: Kafka Tiered StorageOptimizing Apache Kafka's Internals with Its Co-Creator Jun RaoWatch the video version of this podcastKris Jenkins' TwitterStreaming Audio Playlist Join the Confluent CommunityLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperLive demo: Intro to Event-Driven Microservices with ConfluentUse PODCAST100 to get an additional $100 of free Confluent Cloud usage (details)  

Here are links to a few things we mentioned: Heap-only tuple updates (docs)README file (source code)Heap Only Tuple chapter (Internals of PostgreSQL by Hironobu SUZUKI) How partial, covering, and multicolumn indexes may slow down UPDATEs (blog post by Nikolay) Why Uber switched from Postgres to MySQL (blog post)  Fighting write amplification with HOT updates (Adyen blog post) HOT Updates vs Bottom-Up Index Deletion (video by Lukas Fittl from pganalyze) Reduced Bloat with Bottom-Up Deletion (blog post by Hamid Akthar from Percona) What is fillfactor and how does it affect performance? (blog post by Kaarel Moppel from Cybertec)------------------------What did you like or not like? What should we discuss next time? Let us know by tweeting us on @samokhvalov / @michristofides / @PostgresFM, or by commenting on our Google doc.If you would like to share this episode, here's a good link (and thank you!)Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork 

Episode 91: “Our Internals > Those Externals” With the US midterm elections looming and fear and angst on the increase, JDK asks us all to step within and connect with our own feelings and life compass first. Why? And how can that possibly help us? Listen to find out, then pay close attention to how you feel after what he says, and you'll experience what he means.

Подкаст RadioDotNet выпуск №60 от 31 октября 2022 года Традиционный выпуск про конференцию DotNext — самую лучшую .NET конференцию в мире. У микрофонов Антон Оникийчук, Игорь Лабутин и Анатолий Кулаков. Члены программного комитета обсуждают спикеров и помогают определиться с докладами.Приглашаем всех на конференцию DotNext 2022 Moscow: Сайт DotNext.ruСкидка на Personal билет: DotNetRu2022JRGpc Сайт подкаста: radio.dotnet.ru Темы: [00:03:50] — Trends [00:17:21] — Best Practices [00:30:21] — Internals and Performance [00:39:27] — Architecture [00:46:15] — Soft skills Фоновая музыка: Максим Аршинов «Pensive yeti.0.1»

Fr. Frank Pavone's Homily for October 11, 2022: Both Internals And Externals Matter by Priests for Life

And few things we mentioned: Topic request on Reddit — thanks HerbyHoover!Haki Benita's blog PostgreSQL documentation (table of contents) Planet PostgreSQL (blog aggregator) MVCC Unmasked (by Bruce Momjian) The Internals of PostgreSQL (by Hironobu SUZUKI)PostgreSQL 14 Internals — parts I and II (by Egor Rogov)Cybertec blogmodern-sql.com (by Markus Winand)use-the-index-luke.com (by Markus Winand)The Art of PostgreSQL (by Dimitri Fontaine) explain.depesz.comexplain.dalibo.com ------------------------What did you like or not like? What should we discuss next time? Let us know by tweeting us on @samokhvalov and @michristofidesIf you would like to share this episode, here's a good link (and thank you!)Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork  

In this episode of Scaling Postgres, we discuss a new Postgres playground, a book about PG14 internals, how to corrupt your database and using anti-joins. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.crunchydata.com/blog/learn-postgres-at-the-playground https://www.crunchydata.com/developers/tutorials https://www.youtube.com/watch?v=Ryj5c8zLF50 https://postgrespro.com/blog/pgsql/5969682 https://www.cybertec-postgresql.com/en/how-to-corrupt-your-postgresql-database/ https://www.crunchydata.com/blog/rise-of-the-anti-join https://www.enterprisedb.com/blog/aws-rds-postgresql-deployment-pgadmin-4 https://www.enterprisedb.com/blog/leverage-new-way-import-existing-postgres-database-kubernetes https://postgrespro.com/blog/pgsql/5969676 https://www.cybertec-postgresql.com/en/aliases-for-sub-selects-in-from-clause/ https://postgres.fm/episodes/monitoring-checklist https://pganalyze.com/blog/5mins-postgres-security-patch-releases-pgspot-pghostile https://postgresql.life/post/hou_zhijie/ https://www.rubberduckdevshow.com/episodes/55-code-quality-with-ernesto-tagwerker/  

PHP Internals News: Episode 103: Disjunctive Normal Form (DNF) Types London, UK Friday, June 24th 2022, 09:07 BST In this episode of "PHP Internals News" I talk with George Peter Banyard (Website, Twitter, GitHub, GitLab) about the "Disjunctive Normal Form Types" RFC that he has proposed with Larry Garfield. The RSS feed for this podcast is https://derickrethans.nl/feed-phpinternalsnews.xml, you can download this episode's MP3 file, and it's available on Spotify and iTunes. There is a dedicated website: https://phpinternals.news Transcript Derick Rethans 0:15 Hi, I'm Derick. Welcome to PHP internals news, a podcast dedicated to explaining the latest developments in the PHP language. This is episode 103. Today I'm talking with George Peter Banyard again, this time about a disjunctive normal form types RFC, or DNF, for short, which he's proposing together with Larry Garfield. George Peter, would you please introduce yourself? George Peter Banyard 0:39 Hello, my name is George Peter Banyard, I work on PHP paid part time, by the PHP foundation. Derick Rethans 0:44 Just like last time, we are still got colleagues. George Peter Banyard 0:46 Yes, we are indeed still call it. Derick Rethans 0:48 What is this RFC about? What is it trying to solve? George Peter Banyard 0:52 The problems of this RFC is to be able to mix intersection and union types together. Last year, when intersection types were added to PHP, they were explicitly disallowed to be used with Union types. Because: a) mental framework, b) implementation complexity, because intersection types were already complicated on their own, to try to get them to work with Union types was kind of a big step. So it was done in chunks. And this is the second part of the chunk, being able to use it with Union types in a specific way. Derick Rethans 1:25 What is the specific way? George Peter Banyard 1:27 The specific way is where the disjoint normal form thing comes into play. So the joint normal form just means it's a normalized form of the type, where it's unions of intersections. The reason for that it helps the engine be able to like handle all of the various parts it needs to do, because at one point, it would need to normalize the type anyway. And we currently is just forced on to the developer because it makes the implementation easier. And probably also the source code, it's easier to read. Derick Rethans 1:54 When you say, forcing it up on a developer to check out you basically mean that PHP won't try to normalize any types, but instead throws a compilation error? George Peter Banyard 2:05 Exactly. It's, it's the job of the developer to do the normalization step. The normalization step is pretty easy, because I don't expect people to do too many stuff as intersection types. But as can always be done as a future scope of like adding a normalization step, then you get into the issues of like, maybe not having deterministic code, because normalization steps can take very, very long, and you can't necessarily prove that it will terminate, which is not a great situation to be in. Imagine just having PHP not running at all, because it's stuck in an infinite loop trying to normalize the format. It's just like, oh, I can't compile Derick Rethans 2:39 Would a potential type alias kind of syntax help with that? George Peter Banyard 2:44 Maybe, I'm not really sure. Actually reading like research about it from computer scientists, in functional programming languages, which is everything is compiled on my head. And they have the whole thing was like, well, they need to type type normalize, and especially with type aliases, they haven't really figured out a way yet. So I'm not sure how we are going to figure out a way if experts and PhD students and researchers haven't really figured out a way. Derick Rethans 3:08 And is the reason for that mostly, because PHP, resolves types while it is running code sometimes because it has to overload classes, and then it might find out it is an inherited class, for example? George Peter Banyard 3:19 Yes, I think it's like this weird thing where might maybe PHP has like kind of an advantage, because it doesn't need to, like resolve all of the types at once. And if you have a type alias, it's just oh, if it's used, and you just need to resolve it, and then try to figure it out. There's also the added complexity of like, variance checks, because most functional programming languages, they have variance to some degree, but they don't have the whole inheritance of like typical OOP languages have. It's kind of a very strange field, the fact that yeah, PHP is just like, well, we kind of do stuff at runtime, and you don't necessarily need everything. And it just works is like, well, we'll do. That's mainly the reason why the dev needs to do the normalization step, the form is done. It's also I think, the most easiest to understand, it's just like, Oh, you have this and this, or this group, or stuff, or this group of stuff, or this thing, simple type. The other form would be another normalized form would be conjunctive normal form, which is a list of ANDs of ORs to just have this thing, or X, like (A or B or C) and X and (Y or Z), which I think is harder to understand. Derick Rethans 4:26 What is the exact syntax then? George Peter Banyard 4:28 So the exact syntax is, if you want to have an intersection type was in a union type, you need to like bracket it by parentheses. And then you have like the normal pipe union operator and you can mix it with single types, you can mix it with true, you can mix it with false, which are literal types, which now exist, or just normal, bool types. Derick Rethans 4:48 The parenthesis is actually required. You don't rely on operator precedence to make things work? George Peter Banyard 4:53 Yes. Relying on operator precedence is terrible. Derick Rethans 4:57 Yep, I agree. George Peter Banyard 4:58 I'd say Oh, yeah, but I think I've heard this argument on the list like a couple of times, it's just, oh, yeah, but maths, like, has like, and as priority over like, or, I mean, I did three years of a maths degree and not gonna lie. Maths notation is terrible for most of us. People don't even agree on terminology. I'm just gonna say, let's, let's just do better. Derick Rethans 5:19 I agree. I mean, most coding standards for any sort of variable for like conditions, will already require parenthesis around multiple complex clauses anyway, right? I mean, it's a sensible thing to do, just for readability, in my opinion. So the RFC also talks about a few syntax that you aren't allowed to do, and that you have to normalize or deconstruct yourself, what kinds of things are these? George Peter Banyard 5:41 if you would want to have a type which has an intersection of a class A with at least one other class, so let's say X or Y, but you can always convert it into DNF form, how this type would be, it would be (A and X) or (A and Y). This seems to be the more unusual case, I would imagine. One of the motivating cases of DNF types is to do something like Array or (Traversable and Countable). I don't really see mixing and matching various different object interfaces in differencing, the most useful user land cases to be able to do Array or (Traversable and Countable) so that you can use just count or seeing something as an array, or you have like Traversable and Countable and ArrayAccess. And it's just like, Oh, here's an object, which kind of behaves like an array. Derick Rethans 6:32 I think there's currently another RFC just being proposed, that extends iterator_to_array to multiple types as well to accept more things. So that sort of fits into this category of things to do with iterables and traversals then I suppose. George Peter Banyard 6:49 yeah Derick Rethans 6:50 I'm hoping to talk to the author of that RFC as well. At the moment where two and a half weeks or so before a feature freeze, you now see a whole flurry of RFCs while it was a bit quiet in the last few months. So because you're adding to the type system, that's also usually has consequences for variance rules, or rather, how inheriting works with return types and argument types, as well as property types. What do DNF types mean for these variance checks? George Peter Banyard 7:19 The variance is checks, kind of follow the similar rules as before. So property types are easy. They are invariant, so you can't change them. You can reorder types, like was in your union if you want to. But that was already the case with Union types previously, because PHP will just check that, well, the types match. So contravariant, you can always restrict types, meaning you can either add intersections, or you can remove unions, broadly speaking. What you could do, for example, if you have like A or B or C, you could do A and X as a subtype, because you're restricting A to be of an extra, like an extra interface. Derick Rethans 8:06 So then you will have (A and X) or B or C. George Peter Banyard 8:10 Yes. So that's one restriction. You can add how many interfaces you want and do an intersection type, you can add them on every type you can. On the other side, you can just add like unions. So if for contravariance, or like an an argument type, it's like, well, I just want to return something new, well, then you can add unions, but you can't add an intersection to a type, you can only widen types of arguments. So if your type is A or B or C, you can't do A and B, and you can't do (A and X) or B or C, because you're restricting the type. If your type would be (A and X) or (B and Y) or (C and Z), then you could lift the restriction to A or B or (C and Z) because you loosening the requirements on on the type that you're accepting. Derick Rethans 8:55 To summarize this: argument types, you can always widen; return types you can only restrict, and, and property types you can't change at all. I specifically wanted to summarize that because I always find contravariance and covariance. These names confuse me. So that's why I prefer to talk about widening types and restricting types instead. Because there are so close together for me. We spoke a little bit about redundant types. What is this new functionality do if you specify redundant types? George Peter Banyard 9:30 Redundant types how they currently work in PHP are done at compile time. And they do exact class matches or constant class aliasing matches. Derick Rethans 9:41 That will need an explanation. George Peter Banyard 9:44 Class names and interface names in PHP are case insensitive. So you can write a lower-case a or upper-case A and it means the same class. If you provide let's say lower-case a or upper-case A, the engine realize this, this is the same class, so we'll serve it on the type error. So PHP has use statements, or use as. So these are compile time aliases. If you define a class A, and then you say use A as B. So B is a compile time alias of A. And then you do a type which has A or B, PHP already knows these things refer to the same class. So it will raise a compile time error. Derick Rethans 10:25 These use aliases are per file only, right? George Peter Banyard 10:28 Yes, that's usually to do with if you import traits or like a namespaces. And you get conflicting class names. That's how you handle it about. PHP has also this feature, which you can do this at runtime, using the function called class_alias. Now, obviously, compile time checks are done at compile time. So it doesn't know at runtime that you aliasing these classes or using this name as an alias. So then PHP won't complain. Derick Rethans 10:53 But will don't complain during runtime. George Peter Banyard 10:56 No. Derick Rethans 10:56 You really just wanted to shoot yourself in the foot, we'll let you do this. George Peter Banyard 11:00 Yet, during this at runtime, just as like a whole layer of time, because it's not it's not really useful. Basically, what it means that PHP won't guarantee you the type is minimal. I.e. you might have redundant types, but it will just try to tell you, it's like oh, the- these are exactly the same types. And I know these are the same types, you probably do get mistake. So if it can determine this at compile time, it will tell you. Derick Rethans 11:23 The variance is still checked when you're passing in things. George Peter Banyard 11:26 Yes, so variance is checked on inheritance. When the class is inherited and compiled, because it needs to load the parent class, it will then check that it's built properly, and otherwise it will raise an error, that's fine. But just checking that the types is minimal is not possible. A) because inheritance, you don't know how it works, because it will only do the checks on basically on the name of the strings, it will do like compare strings of class names. And if it doesn't know the class name, or if it or if it needs to do some inheritance, it just won't do an instance of check. They just ignore that. It's just like, well, maybe it is maybe it's not I don't know. And that's fine. Derick Rethans 12:08 Of course, if you pass in a wrong type at runtime, then it will still get rejected during runtime anyway. George Peter Banyard 12:14 Yes, that hasn't changed. Derick Rethans 12:16 The only thing that you might end up in a situation where you don't get warned during compile time whether type is redundant. George Peter Banyard 12:23 Yes. So that's the behaviour we currently are the behaviour is added. So, it will check that two intersection types within the union are identical using the same class stuff. So for example, if you have class A, and you say use a as B, and then you have a type which is (A and X) or (B and X), it will tell you: Okay, these classes are the same. The check it adds now also it will check that you don't have a more restrictive type with a wider type. So if your type is T or (T and X), because T is wider than T and X, it will error at compile time, it'll tell you well, T is less restrictive than T and X. So the T and X type is redundant. Derick Rethans 13:11 Okay, so nothing strange. Basically, what you expect to happen will happen. And PHP does its best telling you at compile time whether you've done something wrong or not. George Peter Banyard 13:22 Yes. Derick Rethans 13:24 I think we've spoken mostly about the functionality itself and types. I'm a little bit interested in whether you encountered some interesting things while implementing this feature. George Peter Banyard 13:33 This feature basically, was a bit in limbo for the implementation, because I was waiting on a change to make Iterable, a compile time alias of Array or Traversable, which shouldn't affect userland. Because previously, all of the checks needed to cater to if you get Iterable, then you need to check for the variance. Has it Array , has it a Traversable type, does this accept? Is it why the is it more restrictive, it's identical. It's just this weird edge case, which makes the variance code harder. Moving this to a compile time alias, where now it just uses the standard, a standard union type in some sense, just makes a lot of the variance checks already streamlined and simpler. And because this is simpler, in some sense, was DNF types. When you hit the intersection, you need to recurse one step to check the variance. This helps. This is also kind of why DNF types are enforced like as like the structure on the dev because otherwise, you could potentially get into the whole like, oh, infinite recursion if you do like very nested types, because it's just like, oh, you hit one nested type and so, oh okay, now I'm again in unnecessary time and then you recurse again and then you recurse again, and so that's all you get into the thing: Oh you need to normalize the type. The variance check is: Can you see if it's a union type is the first type a sub list So a list of intersection types, okay, is it balanced? And then just recall the same function in some sense, like, check the types for variance, is this correct? Okay, move to the next type back into the Union and everything. So the implementation is conceptually simple, because all of the implementation details already exist. And all the everything hard has already been done. Now, it's just like, in some sense, it was extracting it into its own function, and then like recurse into it, and not forget to update opcache properly. Derick Rethans 15:31 You mentioned that in order to make the DNF types work, you were waiting on this Array or Iterable or Traversable kind of type. Is this also type people can use it and userland? Or is it internal only? George Peter Banyard 15:44 It is the standard Iterable type that you can already use. So currently, PHP considered Iterable, a full type in some sense. And what the this implementation change basically makes it Iterable into ... compile time alias of Array or Traversable. Iterable exists since, PHP, 7.1, I think. Can still use it, reflection should still be fine if you use it as a single type. Derick Rethans 16:08 So to change there is more, instead of: if you encounter Iterable, we check for both Array and Traversable. Then, instead of making the check every time you look at Iterable is already part of the type system, so you don't have to make the check every time. George Peter Banyard 16:23 Exactly, you basically move when it's being transformed in some sense. Now it has some repercussion on other parts, which needed to be taken care of, which is probably why it was in limbo for 10 months. I had already done the implementation of DNF types, basically, working on my local copy of that branch. It's just like: Okay, this got merged, nice, I can now open the PR onto PHP SRC. So I didn't wait for it to land until start working on it. Derick Rethans 16:50 Things like that also often affect reflection, because you're adding more complex types to the type system. So what kind of changes does that make to PHP's reflection system? And does this end up breaking backwards compatibility? George Peter Banyard 17:04 So in theory, no, it doesn't. How the reflection API works around the type system is that most method calls will turn a reflection type interface, ReflectionNameType, ReflectionUnionType, and ReflectionIntersectionType, are all instances of a ReflectionType. And methods if you would call on the list. So on a union type, the type it would return if you get like getTypes is a ReflectionType. The type system and how the reflection idea was designed, there is no BC break. How the standard was working, it's like, Oh, if you had like a union type, or an intersection type, if you call the getList or getListOfTypes, or getTypes, I don't remember exactly what the method name is actually called, you will always get an array of reflection name types, because you can only have like one level of list in some sense. However, now, if your top type is a union type, then if you get getTypes, you might get an array of ReflectionNameTypes with ReflectionIntersectionTypes. So that's the case that you now need to cater to. So if you get another ReflectionIntersectionType in between. There, you could only have ReflectionNameTypes, there was no nesting, whereas now if you have a union type, one of the types that you get back from the getTypes method in the array will be a ReflectionIntersectionType. Technically, all of the types of the part of the reflection type, so it's an array of reflection types that you get. How it worked before is that you didn't need to care about this distinction between: Oh, it returns a ReflectionType and a ReflectionNameType because well, it only return a ReflectionNameType. But now this is not the case. So you now need to cater to that that oh, you might have nesting. Which kind of boils down to like if in the future, we decide to like have oh, you can nest union types in an intersection type, then the getTypes method might return a union type with other name types. Derick Rethans 19:03 You just need to make sure that you check for more than just one thing that it previously would have done. You can't assume not everything is a ReflectionType any more. It could also be ReflecionIntersectionType. George Peter Banyard 19:18 Yes, exactly. Derick Rethans 19:20 I think that sort of what's in the RFC, is there any future scope? George Peter Banyard 19:25 I mean, the future scope is type alias. As usual. Everything I feel when you talk about the type system, it's like type aliases. At one point when your types gets very complicated. It would be nice to just be able to refer this as a as a named type in some sense, instead of needing to retype every time the whole union slash intersection of it. Hopefully we can get this running for 8.3. We are starting to get kind of complicated types. It would be nice being able to have this feature. The other obvious future scope in some sense, who knows if it's actually desirable is to allow either having conjunctive normal form so you can have like a list of ANDs or ORs Derick Rethans 20:05 You call these conjunctive normal forms? George Peter Banyard 20:08 Yes. Or just a type, which is not normalized. Not sure if it's really desirable to have this feature, because then you get into the whole thing of, if PHP doesn't, either PHP doesn't know how to like normalize it, or it's not in the best form, and then you get into like, very long compilation units or just checking. It's like, okay, does it respect the type? Does it do all of the instance of checks? And I'm not sure if it's super desirable. Derick Rethans 20:38 So it could be considered future scope. But from what I gather from you, you don't actually know what it is actually a desirable thing to add to the language? George Peter Banyard 20:46 Yes. Derick Rethans 20:47 Okay, George, thank you for taking the time this morning to talk about this new DNF types RFC. George Peter Banyard 20:54 Thank you for having me. As always. Derick Rethans 20:59 Thank you for listening to this installment of PHP internals news, a podcast dedicated to demystifying the development of the PHP language. I maintain a Patreon account for supporters of this podcast as well as the Xdebug debugging tool. You can sign up for Patreon at https://drck.me/patreon. If you have comments or suggestions, feel free to email them to derick@phpinternals.news. Thank you for listening, and I'll see you next time. Show Notes RFC: Disjunctive Normal Form Types Credits Music: Chipper Doodle v2 — Kevin MacLeod (incompetech.com) — Creative Commons: By Attribution 3.0

PHP Internals News: Episode 102: Add True Type London, UK Thursday, June 2nd 2022, 09:06 BST In this episode of "PHP Internals News" I talk with George Peter Banyard (Website, Twitter, GitHub, GitLab) about the "Add True Type" RFC that he has proposed. The RSS feed for this podcast is https://derickrethans.nl/feed-phpinternalsnews.xml, you can download this episode's MP3 file, and it's available on Spotify and iTunes. There is a dedicated website: https://phpinternals.news Transcript Derick Rethans 0:00 Hi I'm Derick. Welcome to PHP internals news, the podcast dedicated to explaining the latest developments in the PHP language. This is episode 102. Today I'm talking with George Peter Banyard about the Add True Type RFC that he's proposing. Hello George Peter, would you please introduce yourself? George Peter Banyard 0:33 Hello, my name is George Peter Banyard, I work part time for the PHP Foundation. And I work on the documentation. Derick Rethans 0:40 Very well. We're co workers really aren't we? George Peter Banyard 0:43 Yes, indeed, we all co workers. Derick Rethans 0:45 Excellent. We spoke in the past about related RFCs. I remember, which one was that again? George Peter Banyard 0:51 Making null and false stand alone types Derick Rethans 0:53 That's the one I was thinking of him. But what is this RFC about? George Peter Banyard 0:56 So this RFC is about adding true as a single type. So we have false, which is one part of the Boolean type, but we don't have true. Now the reasons for that are a bit like historical in some sense, although it's only from PHP 8.0. So talking about something historical. When it's only a year ago, it's a bit weird. The main reason was that like PHP has many internal functions, which return false on failure. So that was a reason to include it in the Union types RFC, so that we could probably document these types because I know it would be like, string and Boolean when it could only return false and never true. So which is a bit pointless and misleading, so that was the point of adding false. And this statement didn't apply to true for the most part. With PHP 8, we did a lot of warning to value error promotions, or type error promotions, and a lot of cases where a lot of functions which used to return false, stopped returning false, and they would throw an exception instead. These functions now always return true, but we can't type them as true because we don't have it, and have so they are typed as bool, which is kind of also misleading in the same sense, with the union type is like, well, it only returns false. So no point using the boolean, but these functions always return true. But if you look at the type signature, you can see like, well, I need to cater to the case where the returns true and when returns false. Derick Rethans 2:19 Do they return true or throw an exception? George Peter Banyard 2:22 Yeah, so they either return true, or they either throw an exception. If you would design these functions from scratch, you would make them void, but legacy... and we did, I know it was like PHP 8.0, we did change a couple of functions from true to void. But then you get into these weird shenanigans where like, if you use the return value of the function in a in an if statement, null gets because in PHP, any function does return a value, even a void function, which returns null. Null gets coerced to false. So you now get like, basically a BC break, which you can't really? Yeah, we did a bit and then probably we sort of, it's probably a bad idea. That's also the point of like, making choices, things that are static analysers can be like, more informants being like, Okay, your if statement is kind of pointless here. Derick Rethans 3:06 Yeah, you don't want to end up breaking BC. Now, we already had false and bool, you're adding true to this. How does that work with Union types? Can you make a union type of true or false? George Peter Banyard 3:18 No. So there are two reasons mainly. A. true and false is the same as like boolean, which is like just use Boolean in this case. But you can say, well, it's more specific, so just allow it. So that's would be reasonable. But the problem is, false has different semantics than boolean. False does not coerce values. So it only accepts false as a literal value. Whereas boolean, if you're not in strict type, which is a lot of code, it will cause values like zero to false one, or any other integers to true. It will coerce every other integer to true, like the true type follows the behaviour of false of being a value type. So it only accepts true, you would get into this weird distinction of does true or false, mean exactly true or false? Or do you get the same behaviour as using the boolean type? Derick Rethans 4:07 So I would say that true or false would than be more restrictive than bool. George Peter Banyard 4:12 Exactly, which is a bit of a problem, because PHP internally has true and false and separate types, which also makes the implementation of this RFC extremely easy, because PHP already makes the distinction of them. But at the same time, the boolean type is just a union of the bitmask of true and false. You can't really distinguish between the types, true or false, or the boolean type within the type system. Currently just does it by checking if it only has one then it can do like two checks. Specifically, you would need to add like an extra flag. I mean, it's doable, but it's just like, Well, who knows which semantics we want? Therefore, just leave it for future discussion because I'm not very keen on it to be fair. Derick Rethans 4:55 True or false are really only useful for return values and not so much for arguments types, because if you have an argument that that always must be true, then it's kind of pointless to have of course. George Peter Banyard 5:05 Same as like it was with the null type RFC. Although there might be one case where PHP internal functions might change the value to true for an argument, I can maybe two types, would be like with the define function, this thing being like case insensitive or case sensitive, I don't remember what the parameter actually; could actually either be false or true, because at the moment, I think emits a notice, things do like the this thing is not supported, therefore the values what was ignored. But we could conceivably see that in PHP 9, we would actually implement this as a proper like: Okay, this only accepts true, yes, this argument is pointless, but it's in the middle of the function signature, so you can't really move it. The spl_register_overload function has like as its second argument, the throw on error or not, which since PHP 8 only accepts true, but it's in the middle of the function. The last argument is still very useful. It's prepend, instead of append the autoloader, I think, or might be the other way around, check the docs. Since PHP 8, this only accepts true. So if you pass in false, it will emit a notice and saying you'd like this argument has just been ignored. So whatever. But we can't really remove the argument. Because well, it's, if you use the third argument, as with positional arguments, then you would change like the signature and you would break it. Now, we don't have a way to enforce in PHP to use named arguments, because that would be a solution. It's just like, well, if you want to set this argument, you need to use named arguments, but we can't do that. Otherwise, then creating a new function, which has an alias, which is also kind of terrible. That would be one of the maybe only cases where you would actually get like true as a as an argument Derick Rethans 6:39 is that now currently bool? And there's a specific check for it? George Peter Banyard 6:42 It's currently bool, and if you pass in false enrolment, like a warning, or notice. Derick Rethans 6:47 How would inheritance work? As return types, you can always make them smaller, right? More restrictive. George Peter Banyard 6:53 Yes, that's also the thing. But that already exists in some sense a problem of. Like if you go from boolean to false, you're already restricting the type. And that problem existed, even before the restricting, well allowing false as a stand-alone type if you had like, as a union, because you could always say like, I don't know. That problem already existed with Union types. Because you could have something like overturn an array or bool and then you change it to either an array or false. And then if you try to return like zero, then you will get like a coercion problem. So the same problem applies with true, because it only affects return values. And like you control the code within a function compared to like how you pass it, that's less of an issue. It applies also, with argument types where you can go from true to like boolean, or true and like a union type. Derick Rethans 7:44 So there's nothing surprising here. I see that the RFC also talks a little bit about future scope. Can you tell a bit more about that? George Peter Banyard 7:53 True and false are part of what are called value types, they are a specific value within the type. One possible future scope would be to expand value types to all possible types. So that you could say, oh, this function returns one, two or three. Derick Rethans 8:08 Would you not rather use an enum for that? George Peter Banyard 8:09 Exactly. That's the point I was going to make is that enums serve this purpose, in my opinion. And as a type purist, ideally, I would have preferred that we didn't have to enforce because the code, it kind of goes against the grain in this sense. Derick Rethans 8:23 We've had it for 25 years, booleans. George Peter Banyard 8:26 Yes, right. But boolean is its own type, in some sense, which you could say is a special enum. Enums are types. But we have false, and not having true is just so weird to me. It's like, oh, you've got this thing, but you don't have this other thing. And there are loads of cases where functions return true, or due to legacy reasons and to preserve BC, and PHP 8 promoted a bunch of warnings to to error. So now you've got functions which used to return false, don't return false any more. And they only return true. Now, some of the famous examples are probably like array_sort of, like actually, the sorting array functions, return true for basically all of PHP 7. I think there was something changed in PHP 7, probably was the engine or something like that, that they stopped returning false, which is strange. And I've made the discovery somewhat recently, I'm like, this is so pointless, because you see loads of loads of code checking like that the return value of the sort function is correct. Derick Rethans 9:20 It's also that most of the sort functions actually sort by reference instead of returning the sorted array, which I can understand as a performance reason to do but... George Peter Banyard 9:29 it's not very functional. You modify stuff in place and like passing it around. And because yeah, I think the initial thing was that like, well do it would return a false or true because sometimes it could, the sort could fail. Derick Rethans 9:42 I don't understand how a sort could failure, but there we go. George Peter Banyard 9:46 I mean, I suppose if you have like incomparable values within the array like that somewhat logical, I suppose. Derick Rethans 9:53 Was there anything else in future scope? George Peter Banyard 9:56 One of the future scope, I feel was everything type related. It's like type aliases, because when you start making more complicated types, having a way to type alias, it is probably nice. Don't think we'll get this for PHP 8.2. I don't think we any of us had the time to work on it. Derick Rethans 10:11 Well, we only have a month left anyway. George Peter Banyard 10:13 Yeah. And I mean, I'll probably be back on here. I'm trying to get DNF types working, but... Derick Rethans 10:19 Can you explain that these are? George Peter Banyard 10:20 Disjoint normal form types? Derick Rethans 10:22 That did not help. George Peter Banyard 10:24 But it's the being able to combine union types with intersection types together, Derick Rethans 10:28 I can understand that doing that is kind of complicated. You also need to sort of come up with a with a language to define them almost right? I mean, you then get the argument, are you going to require a parenthesis around things? George Peter Banyard 10:38 I'm requiring parentheses. People have told me the argument of like: Yeah, but in maths like and takes priority, it's just like, have you seen mathematicians, mathematicians don't agree on notation, and it's terrible, or they call stuff and the different they call it something is like, oh, sometimes a ring is commutative, and sometimes it's not. Don't follow mathematicians, don't follow mathematician, Derick Rethans 10:57 Type aliases is something that would only apply to single files. See, that's what you're suggesting. And then there's exported type definitions, which I guess could be autoloaded at some point; would be nice to have, I guess. George Peter Banyard 11:09 I think that's the trouble just like defining the semantics. Type aliases within a file are nice, but they're not very useful. Most of the time, you would want to export the type. For example, if you say: Oh, I accept, I don't know, something which looks like an array, which is like an array and like Traversable, and ArrayAccess or something. I'm sure, it's nice to have it in your own file. But like, if you use it around a project, and you need to redefine the type, every single file kind of defeats the purpose. Derick Rethans 11:35 It's kind of tricky to do with type definitions, because you sort of need to make sure that there are available and maybe can be autoloaded, just like classes can be right. And that makes things tricky. Because having a type definition and just three lines in a file, is kind of annoying, but I guess that is sort of necessary to do the kind of thing in a PHP ish way. George Peter Banyard 11:55 Yeah, we talked about it with Ilija because he he was on about it. And I was like: Well, ideally, you would want the separate autoload of types. That's how I initially conceived it, it's like having a different autoloading for types. But then the problem is, is like if anytime you hit a class, like in an argument, if you autoload the type first, it will go through all of the type definitions. And if, okay, at the moment, that wouldn't be there wouldn't be much. But if you go into like importing 30 composer projects, or libraries, which are define their own types, it will go through all of those first, before going to the classes autoloaded, and trying to find it then, which is not ideal. Yeah, it's going to be a tricky problem. It's either you merge these symbols together. But then the class table is not always a class. And sometimes you can't do new type. Like I said, tricky problems. Derick Rethans 12:43 Yeah, that's a tricky problem, but an interesting one. George Peter Banyard 12:47 Yeah. Derick Rethans 12:47 So that's future scope then. George Peter Banyard 12:50 Exactly. That is future scope. Derick Rethans 12:52 Do you have anything else to add? George Peter Banyard 12:54 Um, no, not really. I think I've said all I have to say it's pretty straightforward. Should be uncontroversial, hopefully. Derick Rethans 13:02 It currently looks like it's 20 for, and zero again. So I guess it will pass. George Peter Banyard 13:07 Brilliant. Derick Rethans 13:08 Who said that, that if your RFC ends up passing unanimously, it is too boring? George Peter Banyard 13:13 Nikita. Derick Rethans 13:14 Which is not incorrect. George Peter Banyard 13:16 It is not incorrect. But I mean, at the beginning, because I was like: Well, this is pretty straightforward. So I wrote the RFC, it was tiny. And I put it on to the list and people was like: Yeah, but what's the motivation for? I understand for adding false, because they already exist. But what's the motivation for adding a new type, and I was like, I now need to go back to the drawing board and write more. To be fair, that was a smart, because I then discovered the whole issue about true and false. That false is just a value type and doesn't do coercions. And it's like, okay, how do you handle the semantics and everything? Derick Rethans 13:46 I'm glad to hear it. Then all I have to say thank you for taking the time today to talk about this new RFC. George Peter Banyard 13:53 Thank you for having me as usual. Derick Rethans 13:59 Thank you for listening to this installment of PHP internals news, a podcast dedicated to demystifying the development of the PHP language. I maintain a Patreon account for supporters of this podcast as well as the Xdebug debugging tool. You can sign up for Patreon at https://drck.me/patreon. If you have comments or suggestions, feel free to email them to derick@php internals.news. Thank you for listening, and I'll see you next time. Show Notes RFC: Add True Type RFC: Allow Null and False as Standalone Types Credits Music: Chipper Doodle v2 — Kevin MacLeod (incompetech.com) — Creative Commons: By Attribution 3.0

PHP Internals News: Episode 101: More Partially Supported Callable Deprecations London, UK Thursday, May 19th 2022, 09:05 BST In this episode of "PHP Internals News" I talk with Juliette Reinders Folmer (Website, Twitter, GitHub) about the "More Partially Supported Callable Deprecations" RFC that she has proposed. The RSS feed for this podcast is https://derickrethans.nl/feed-phpinternalsnews.xml, you can download this episode's MP3 file, and it's available on Spotify and iTunes. There is a dedicated website: https://phpinternals.news Transcript Derick Rethans 0:14 Hi, I'm Derick. Welcome to PHP internals news, the podcast dedicated to explaining the latest developments in the PHP language. This is episode 101. Today I'm talking with Juliette Reinders Folmer, about the Expand Deprecation Notice Scope for Partially supported Callables RFC that she's proposing. That's quite a mouthful. I think you should shorten the title. Juliette, would you please introduce yourself? Juliette Reinders Folmer 0:37 You're starting with the hardest questions, because introducing myself is something I never know how to do. So let's just say I'm a PHP developer and I work in open source, nearly all the time. Derick Rethans 0:50 Mostly related to WordPress as far as I understand? Juliette Reinders Folmer 0:52 Nope, mostly related to actually CLI tools. Things like PHP Unit polyfills. Things like PHP Code Sniffer, PHP parallel Lint. I spend the majority of my time on CLI tools, and only a small portion of my time consulting on the things for WordPress, like keeping that cross version compatible. Derick Rethans 1:12 All right, very well. I actually did not know that. So I learned something new already. Juliette Reinders Folmer 1:16 Yeah, but it's nice. You give me the chance now to correct that image. Because I notice a lot of people see me in within the PHP world as the voice of WordPress and vice versa, by the way in WordPress world to see me as far as PHP. And in reality, I do completely different things. There is a perception bias there somewhere and which has slipped in. Derick Rethans 1:38 It's good to clear that up then. Juliette Reinders Folmer 1:39 Yeah, thank you. Derick Rethans 1:40 Let's have a chat about the RFC itself then. What is the problem that is RFC is trying to solve? Juliette Reinders Folmer 1:46 There was an RFC or 8.2 which has already been approved in October, which deprecates partially supported callables. Now for those people listening who do not know enough about that RFC, partially supported callables are callables which you can call via a function like call_user_func that which you can't assign to variable and then call as a variable. Sometimes you can call them just by using the syntax which you used for defining the callable, so not as variable but as the actual literal. Derick Rethans 2:20 And as an example here, that is, for example, static colon colon function name, for example. Juliette Reinders Folmer 2:26 Absolutely, yeah. Derick Rethans 2:27 Which you can use with call_user_func by having two array elements. You can call it with literal syntax, but you can't assign it to a variable and then call it. Do I get that, right? Juliette Reinders Folmer 2:36 Absolutely. That's it. There's eight of those. And basically, the original RFC from Nikita proposed to deprecate support for them in 8.2, add deprecation notices and remove support for them altogether in PHP nine. And the original RFC explicitly excluded two particular things from those deprecation notices. That's the callable type and using the syntaxes in combination with the is_callable function, where you're checking if the syntax is callable. The argument used in the original RFC was to keep those side effect free. The problem with this is that with the callable type, this means you go from absolutely no notice or nothing, to a fatal error in PHP 9. Everything works, and you're not getting any notification. But in PHP 9, its fatal error at the moment that callable is being passed to a function. Derick Rethans 3:31 This is the callable type in function declarations. Juliette Reinders Folmer 3:33 Yeah, absolutely. And with is_callable, I discovered a pattern in my wanderings across the world where people use the syntax in is_callable, but then use it in a literal call. So not using call_user_func, not using a variable to call it, but it's callable static double colon method name, and then called static double colon method name as literal. And that pattern basically, for valid calls would mean that that function would no longer be called in PHP 9 without any notification whatsoever. Derick Rethans 4:13 So it's a silent change that you can't detect at all. Juliette Reinders Folmer 4:17 Yeah, which to me sounded dangerous. I started asking some questions about that. But six weeks ago, the conclusion was, well, maybe this should be changed. But as this was explicit in the original RFC, we can't just change it. We need to have a new RFC to basically amend the original RFC and remove the exception for these two situations and allow them to throw deprecation notices. Derick Rethans 4:44 What are you proposing to change with this RFC than? Juliette Reinders Folmer 4:47 What this RFC is proposing is simply to remove the exception that the callable type and is_callable are not throwing a deprecation notice. This RFC is proposing that they should throw a deprecation notice, so that more of these type situations can be discovered in time for PHP 9 to prevent users getting fatal errors. Derick Rethans 5:08 Now, of course, we have no idea when PHP nine is actually showing up, but I don't think it will be this year. Well, I know it won't be this year, and it certainly won't be be next year neither, I think. Juliette Reinders Folmer 5:17 That's all the same. I mean, it makes there'll be two, three years ahead, but it doesn't really make sense to have the main deprecation in 8.2 and then have the additional deprecation in 8.4 or something. Derick Rethans 5:29 Absolutely. Juliette Reinders Folmer 5:30 It's a lot more logical to have it all in in the same version. Because it's all related. It's basically the same thing without the exception for callable type. And is_callable. Derick Rethans 5:42 Although there is no current application, would this be able to be found if you had like a comprehensive test suite? Juliette Reinders Folmer 5:48 Yes and no. Yes, you can find this with a test suite. But one, you're presuming that there are tests. Two, that the tests covered the effected code with enough path coverage. Three, imagine a test you've written yourself at some point in the past where which affected callables, you might have, you know, a data provider where you say: Okay, valid callable function, which you've mocked or, you know, closure, which you've put in and second, this function does not exist. Okay, so now you're testing this function, which at some point in its logic has a callable, and expects that type to receive that type. But are you actually testing with the specific deprecated partially supported callables? Even if you have a test, and the test covers the affected code, if you do not test with one of these eight syntaxes, which has been deprecated, you still cannot detect it. And then, four, you still need to make sure that the tests are routinely run, and in open source, that's generally not a problem. Most open source projects, use GitHub actions by now to run the tests automatically on every pull request, etc. But, have the tests been turned on to actually run against PHP 8.2. Are the tests run against pull requests? I mean, there are still plenty of projects, which don't do that kind of thing. Yes, you can detect it with a good test suite. But there's a lot of caveats when you will not detect it. And more importantly, you will not be able to detect it until PHP 9. Derick Rethans 7:23 Yes, when your code and stops behaving as you were expecting it to be. Juliette Reinders Folmer 7:28 Yeah, because in 8.2, you're gonna get deprecation notices for everything else, but these two situations. But not in 8.2, not in 8.3, not in 8.4, and then whatever eights we're gonna get until nine, you will not be able to detect without deprecation notices, until PHP 9 actually removes support for these partials deprecated callables. Yes, but no. Derick Rethans 7:53 We already touched a little bit on how you found out for the need for this RFC or for changing behaviours. But as people have stated in the past, adding deprecation notices is a BC break. That's a subject that we will leave for some other time because I don't necessarily believe that. But would, the changes in your RFC not add more backwards compatibility issues? Juliette Reinders Folmer 8:14 The plain and simple the backward compatibility break is in the original RFC. That's where the deprecation is happening. This RFC just makes it clearer where the BC break is going to be in PHP 9. It's not PHP 8.2, which has a backward compatibility break. It's PHP 9 which will have to backward compatibility break. Yes, I've heard all those arguments, people saying deprecation notes are BC break, no they're not. But they are annoying. And Action List, to for everything that needs to be fixed before 9. Given big enough projects, you cannot say: Okay, I'm gonna do this at the last moment, just before 9 comes out. It literally means 10 months of the year I for one am working on getting rid of deprecation notices in project to prepare them all to be ready for PHP 9 when PHP 9 comes round. Derick Rethans 9:06 But it's still better to have them than to not,.and then you code starts breaking right? Because that is exactly why you're proposing this RFC as far as I understand. Juliette Reinders Folmer 9:16 Yes, absolutely. I mean, I'm always very grateful for deprecation notices, but it would be nice if we had fewer changes, which would cost them, for a year or two, so I can actually catch my breath again. Derick Rethans 9:28 I think PHP 8.2 will have fewer of these changes in there. There will still be some of course. Juliette Reinders Folmer 9:35 Well, I mean, this one is one deprecation. And then we have the deprecated Dynamic Properties and that one is already giving me headaches before I can actually start changing it in a lot of projects. I'm not joking, that one really is going to cause a shitload of problems. Derick Rethans 9:51 It's definitely for products have been going on for so long, where dynamic properties are used all over the place. And I see that in my own code as well. I just noticed this morning does actually breaks Xdebug. Juliette Reinders Folmer 10:03 I know it's currently breaking mockery, we're gonna have to have a discussion how to fix that or whether or not to fix it. If Mockery is broken, that means all your tests are broken. So the test tooling needs to be fixed first. Derick Rethans 10:18 That's always the case, if you work with CLI tools that make people run code on newer PHP versions, that's always a group of tools that needs to be upgraded first, which is your sniffers, your static analysis, your debugger still will always need to go first. Juliette Reinders Folmer 10:27 Which is why I look at things a lot earlier, probably then the majority of people. I mean, I see him huge difference between the open source and closed source community. For open source, I started looking at it well, I've been looking at 8.2 since the beginning. And I started running tests for all the CLI tools. As soon as 8.1 comes out, 8.2 gets added to the matrix for running in continuous integration. And then for applications, it gets added like in you know, once alpha 1-2-3 has come out. For the extensions, it gets added in September once the first RFC gets added. And all of them are trying to get ready before the release of 8.1 or 8.2 in this case, because you do not know as an open source maintainer, what version people are going to run your code on. And you can say IP, you can manage that via Composer, no you can't. Sorry, you can only do that if your users are actually installing via Composer. If your users are downloading a zip file, and uploading it to a web host via FTP, there's literally no way you can control whether they're running on 8.0, or 8.1, except for maybe during check: You cannot run on 8.1 yet. Derick Rethans 11:52 Upgrading software with version support is an issue that's been going on for 40 years and will go on for at least another 40 more. This is not a problem that we can solve easily. Juliette Reinders Folmer 12:03 But what I see there is like the closed source community is like, oh, yeah, but you know, by the time I want to upgrade my server to 8.1, or 8.2, I just run Rector and all will be fine. And I'm like, yeah, sorry, that does not work for open source. We need cross version compatible with multiple versions. And I try to keep that range of version small for the project, I initiate, I don't always have control over it. If for instance, one of the projects I maintain is Requests. And that's a project which does HTTP requests. It's used by WordPress, it cannot be let go of the minimum of 5, PHP 5.6, until WordPress, lets go of that. Derick Rethans 12:48 Well, the alternative is that WordPress uses an older version until it can let go of it. Juliette Reinders Folmer 12:54 Yeah, the only problem then is that we don't want to maintain multiple stable branches. For security fixes. Derick Rethans 13:03 For Xdebug, what I do is I support what the PHP project support when a PHP release comes out, which is a bit longer than PHP itself usually, but not by much more than a year or two. Juliette Reinders Folmer 13:15 I understand that. And I mean, I applaud Sebastian for at some point, having the guts to say to the community, I'm limiting the amount of versions I'm supporting. And I'm sticking to the officially supported PHP versions. That does not mean that that didn't give a large part of community which does need to support a wider range of PHP versions a problem. I fully support that people limit the amount of fish and stay support and like Sebastian, who I know got half the community up in arms against him when he said, I'm not going to support older PHP versions any more. It did create a problem and but the problem which I've tried to solve for instance with the PHP unit polyfills, which now is solvable by using the PHP Unit polyfills in quite a transparent way, which is helpful for everyone. It takes the complainers of Sebastian's back, and at the same time, it allows them to run the test. Derick Rethans 14:10 I think another good thing that Sebastian recently has done is make sure that deprecation notices are no longer failing your tests. Juliette Reinders Folmer 14:17 I don't agree. The thing is, I do understand him making that change. But changing that default from not showing those deprecation notices or not not allowing deprecation notes to fail the test, or not in a patched version, I don't think was the right thing to do. That should have been in a minor, let alone or maybe even in a major not in a 9.5.18 patch version. Also with the whole idea, I mean, again, this is very much an open source versus closed source discussion for closed source I completely understands that people say I don't want to know until I actually am ready to upgrade to that version. Derick Rethans 14:56 I understood it's more of a difference not necessarily between open and closed source, but rather between library maintainers and application maintainers. And the applications can then also be closed source. Juliette Reinders Folmer 15:06 The open source work I work in, I mean, I do want to see them. And the problem with the deprecation notices anyhow, and I've seen various experiments via Twitter fly past for the past year. Say you build something on top of something else, you want to see the deprecation notices and the errors which apply to your code. We don't want to see the ones which come from the framework on which you build on top. The silencing deprecation notices or not, allow tests to error out on deprecation and just not solve that problem. Derick Rethans 15:39 The only thing it does is make things a little bit less noisy so that fewer people complain to library authors isn't it? That's pretty much what it does. Juliette Reinders Folmer 15:48 The thing would I see what it has done is that people think the tests are passing. Derick Rethans 15:54 Well they are passing, but... Juliette Reinders Folmer 15:56 Yeah, but most people don't read change logs of PHP unit, especially as releases don't get actually have to change log included. When PHP Unit releases its actual release, it doesn't actually post a release on GitHub. So people who watch the PHP unit repo for releasing doesn't, don't get notifications, let alone a changelog. So they actually have to go into the repo to find out what has changed. Most people don't do that. They just get you know depend-a-bot update, which won't say much, because again, it doesn't have release information. Derick Rethans 16:28 It'd be nice, maybe if Composer ,when you upgrade packages, that it can show like the high level changes when you do an upgrade. The Debian project does that if you upgrade packages that have like either critical or behavioural changes, you actually get a log when you run the update. Juliette Reinders Folmer 16:43 And then the change should have been in major or minor, because in a patch release, you don't expect it kind of changes. I also know the struggle there. They've been going through to four PHP units and which is similar to what I'm struggling with with the amount of changes from PHP 8.0 and 8.1 which has to be deal dealt with. Projects are being delayed, we're having trouble keeping up as an open source community, we still need to look after our own mental health as well. Derick Rethans 17:10 What has the feedback been to far on the RFC or non? Juliette Reinders Folmer 17:13 The feedback on this particular RFC has been next to nothing. And that's not surprising. I mean, basically, the discussion has happened before. And I started the discussion six weeks ago, eight weeks ago, which led to this RFC. So far the responses, which I have seen, either on Twitter or in private or in our people will read through the RFC. They're like, yeah, it makes sense. Derick Rethans 17:37 I think this is quite a nicer way of getting RFCs done, you discuss them first. And if there's then found a need actually spend a time on writing an RFC. In other cases, the other way around happens, right? People write a long, complicated RFC, and then complain that nobody wants to talk about it. Juliette Reinders Folmer 17:53 When I started the previous discussion, it was I see this, I noticed this, was this discussed? And then I got back: yeah, nobody actually discussed the previous RFC and I'm like: Okay, so what's this whole point about under discussion if nobody's discussing? Derick Rethans 18:10 Well, you can't force people to talk, of course. Juliette Reinders Folmer 18:14 It does make me wonder, again, what we were talking about before, people who work in managed environments versus people who will have to support multiple PHP says, I sometimes wonder how many people who actually have voting rights work in those closed environments, and think, you know, upgrading is something you do with Rector. Now I have a feeling that often open source gets a little forgotten. Derick Rethans 18:38 Yeah, that's perhaps true. Thank you for taking the time this morning to talk about this RFC then. Juliette Reinders Folmer 18:44 Thank you Derick for having me. It was a pleasure to do you like always. Derick Rethans 18:49 Thanks. Derick Rethans 18:54 Thank you for listening to this installment of PHP internals news, a podcast dedicated to demystifying the development of the PHP language. I maintain a Patreon account for supporters of this podcast as well as the Xdebug debugging tool. You can sign up for Patreon at https://drck.me/patreon. If you have comments or suggestions, feel free to email them to derick@phpinternals.news. Thank you for listening, and I'll see you next time. Show Notes RFC: Expand deprecation notice scope for partially supported callables Credits Music: Chipper Doodle v2 — Kevin MacLeod (incompetech.com) — Creative Commons: By Attribution 3.0

You already know Apache Kafka® is a distributed event streaming system for setting your data in motion, but how does its internal architecture work? No one can explain Kafka's internal architecture better than Jun Rao, one of its original creators and Co-Founder of Confluent. Jun has an in-depth understanding of Kafka that few others can claim—and he shares that with us in this episode, and in his new Kafka Internals course on Confluent Developer. One of Jun's goals in publishing the Kafka Internals course was to cover the evolution of Kafka since its initial launch. In line with that goal, he discusses the history of Kafka development, including the original thinking behind some of its design decisions, as well as how its features have been improved to better meet its key goals of durability, scalability, and real-time data. With respect to its initial design, Jun relates how Kafka was conceived from the ground up as a distributed system, with compute and storage always maintained as separate entities, so that they could scale independently. Additionally, he shares that Kafka was deliberately made for high throughput since many of the popular messaging systems at the time of its invention were single node, but his team needed to process large volumes of non-transactional data, such as application metrics, various logs, click streams, and IoT information.As regards the evolution of its features, in addition to others, Jun explains these two topics at great length:Consumer rebalancing protocol: The original "stop the world" approach to Kafka's consumer rebalancing—although revolutionary at the time of its launch, was eventually improved upon to take a more incremental approach.Cluster metadata: Moving from the external ZooKeeper to the built-in KRaft protocol allows for better scaling by a factor of ten. according to Jun, and it also means you only need to worry about running a single binary.The Kafka Internals course consists of eleven concise modules, each dense with detail—covering Kafka fundamentals in technical depth. The course also pairs with four hands-on exercise modules led by Senior Developer Advocate Danica Fine. EPISODE LINKSKafka Internals courseHow Apache Kafka Works: An Introduction to Kafka's InternalsCoding in Motion Workshop: Build a Streaming AppWatch the video version of this podcastKris Jenkins' TwitterStreaming Audio Playlist Join the Confluent CommunityLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperLive demo: Intro to Event-Driven Microservices with ConfluentUse PODCAST100 to get an additional $100 of free Confluent Cloud usage (details)   

We love a good Starlet, and they don't come much better than a turbocharged KP61 with some severe aero.Originally a drift school car back in Japan, Angel of Horsepower Techs KP61 Toyota Starlet has had more than a few changes over the years. After running the supercharged 4agze at 180HP Angel wanted more power so turned to a turbocharged Honda F20C engine he already had kicking around. The new F20C setup makes 650HP thanks to the Garrett 3582 Turbo strapped to it and One Ethanol fuel, but with stock internals, there are some more modifications planned to ensure reliability on track during Grid Life's Time Attack season next year.Want to learn how to EFI tune? Start with some free lessons right here: hpcdmy.co/efilessonWhile the car already had a TRD widebody kit, it was nothing compared to the APR Performance and custom rear diffuser now fitted which will be dialled in over the winter. This will be done with the help of Angels AEM Electronics package via more than a few hours spent comparing data and making logical adjustments based on it to ensure there is traction where and when it's needed on track.ECU wise the car retains the AEM Series 2 ECU that came with the F20C transplant from another project, and it is complemented by an AEM 3 port boost controller, CD-7 dash and IMU. The fuel system sports a solid Radium Engineering kit, Injector Dynamics ID2000's and a Horsepower Techs in house fuel rail. Needless to say, we're excited to see what Angel and this setup can do once dialled in and if he utilises parts mentioned like Turbosmarts new electronic wastegate and AEMs Infinity series ECUs.

Greg and Tim tackle E3 adding more partners, Nintendo Switch's continued dominance, and more! Time Stamps - 00:00:00 - Start 00:03:00 - Housekeeping Tonight, Me, Nick, SnowBikeMike, and Andy are headed to IGN for their Stop AAPI Hate Stream! At 7 pm PT, we'll join a cavalcade of stars to play some Fall Guys and raise some money for charity! Come watch on twitch.tv/kindafunnygames tonight! Next Tuesdays at 11AM PT right here on twitch.tv/kindafunnygames we are partnering with Razer to show off the RTX 30-series powered Blade laptops with Andy and the boys playing some Warzone Thank you to our Patreon Producers:  Black Jack & Tom Bock The Roper Report   - 00:07:21 - E3 Additions 00:17:50 - Meanwhile at Summer Game Fest, Geoff Says... 00:28:10 - Switch Sales Are Insane, Matt Purslow @ IGN 00:44:50 - Nintendo's making its own Dreams, Kat Bailey at IGN 00:50:12 -  Ad 00:51:49 - New Division Games, Deets! 00:59:10 - PlayStation Store Lawsuit, Robert Burnson @ Bloomberg 00:59:50 - PS5 Redesign Reportedly in the Works, but just the Internals, Alex Newhouse @ GameSpot 01:01:00 - Should we get hyped for Game Boost? Push Square 01:03:20 - Out today 01:05:40 - You‘re Wrong Tomorrow's Hosts: Greg and Jason Schreier