Podcasts about gaussian

Blackmagic leaks a 12K PYXIS camera just before NAB as the industry prepares for a major trade show. In this episode, hosts Addy Ghani and Joey Daoud dive into pre-NAB announcements, examining Strada's peer-to-peer media sharing tool, Adobe's new AI features, and how Gaussian splats are quietly becoming the next significant technology evolution. Plus, they unpack OpenAI's strategic move into open weights and what it means for enterprise AI adoption.

Jack Wang is the CEO of Kiri engine a powerful 3D scanning app for Android, iOS (and web browsers!). I have experienced Kiri myself and trust me, there some key features that makes it different from anything else on the market. Today we will dive into the world of Gaussian splatting with Jack discussing:Some of the current limitations of Gaussian splatting- The benefits of Gaussian Splatting compared to photogrammetry and the verticals benefitting from it- Some of the unique features that make Kiri engine the must have scanning app for Gaussian Splatting- A new path the team has taken after making an unexpected discovery Subscribe to XR AI Spotlight weekly newsletter

Sam Hodge discusses why NeRFs and gaussian splatting are changing VFX, real-time, photoreal volume capture in an AI game-changing way.

In the digital networked age, people's attention often overlooks local problems in favour of global ones, which don't necessarily impact them in their daily lives, or over which they don't have a say due to the skewed Pareto distribution of power in modern day societies. Puja Ohlhaver, in her recent research paper ‘Community currencies', proposes a dual-currency model that prices attention and influence in each community, with the ultimate goal of creating a Gaussian distribution of power, either locally, or globally through the dynamic interaction of multiple local communities. This model allows community members to stake their currency to earn non-transferable governance rights, creating a substrate for decentralised societal coordination that favours social innovation.Topics covered in this episode:Puja's backgroundWeb3 research‘Community currencies'Pareto vs. Gaussian distributionsGlobal vs. local power distributionsThe community currencies modelMeritocracy vs. influenceQuadratic fundingGovernance, bribery and the crisis of legitimacyExperimenting with community currenciesEpisode links:Puja Ohlhaver on X'Community Currencies' Research Paper'Decentralized Society' Research PaperSponsors:Gnosis: Gnosis builds decentralized infrastructure for the Ethereum ecosystem, since 2015. This year marks the launch of Gnosis Pay— the world's first Decentralized Payment Network. Get started today at - gnosis.ioChorus One: one of the largest node operators worldwide, trusted by 175,000+ accounts across more than 60 networks, Chorus One combines institutional-grade security with the highest yields at - chorus.oneThis episode is hosted by Friederike Ernst.

Niantic launched their Into the Scaniverse application on Quest 3 on February 26th, 2025 that features over 50,000 Gaussian Spats from 120 different countries. They originally launched the WebXR version on December 10th, 2024 at IntoTheScaniverse.com, which was built using Niantic Studio (be sure to check out their comprehensive history of Gaussian Splats by Kirsten M. Johnson released at the same time). Users can use the Scaniverse mobile app on Android or iOS to capture, render, geotag, and upload their own Gaussian Splats onto the Into the Scaniverse mapps that can be viewed on either mobile phone or XR devices. I had a chance to speak more about Into the Scaniverse with Joel Udwin, who is Niantic's Director of Product for Niantic's AR, Research, Developer Platforms, and Scaniverse. Gaussian Splats are only about 1 year and a half old as the original "3D Gaussian Splatting for Real-Time Radiance Field Rendering" paper was presented at SIGGRAPH in August 2023, but it represents a new rendering pipeline for volumetrically captured content. Niantic's Into the Scaniverse apps are able to process and render these splats locally on the phone or Quest devices, and they have a lot of plans for how they will continue to utilize and develop this as a core part of their technology infrastructure and enabling new mixed reality applications. https://www.youtube.com/watch?v=NR51MrAtUM4 This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

  On this episode of Unsupervised Learning Razib talks to Tade Souaiaia, a statistical geneticist at SUNY Downstate about his new preprint, Striking Departures from Polygenic Architecture in the Tails of Complex Traits. Souaiaia trained as a computational biologist at USC, but also has a background as a division I track and field athlete. Razib and Souaiaia discuss what “genetic architecture” means, and consider what we're finding when we look at extreme trait values in characteristics along a normal distribution. Though traits like height or risk for type II diabetes can be thought of as represented by an idealized Gaussian distribution, real molecular and cellular processes still underlie their phenotypic expression. Souaiaia talks about how genomics has resulted in an influx of data and allowed statistical geneticists with a theoretical bent to actually test some of the models that underpin our understanding of traits and examine how models like mutation-selection balance might differ from what we've long expected. After wading through the depths of genetic abstraction and how it intersects with the new age of big data, Razib and Souaiaia talk about race and sports, and whether there might be differences between groups in athletic ability. Souaiaia argues that the underlying historical track record is too variable to draw firm conclusions, while Razib argues that there are theoretical reasons that one should expect differences between groups at the tails and even around the memes.

Gaussian Splatting es una técnica que redefine la captura y visualización 3D, utilizando splats gaussianos para representar la realidad de forma eficiente y precisa. Una evolución que plantea interrogantes sobre el futuro de las nubes de puntos en el mundo BIM. ¿Un modelo 3D desde un vuelo dron con una calidad fotográfica? ¿Es el Gaussian Splatting el fin de las nubes de puntos? ¿Listos para descubrir cómo pintar la realidad en 3D de una forma sorprendente? Porque hoy nos zambullimos en el fascinante, caótico y, para qué negarlo, un poco mágico mundo del Gaussian Splatting, la tecnología que amenaza con convertirse en el fin de las nubes de puntos. ¡Bienvenido al episodio 175 de BIMrras!

Nalini AnantharamanGéométrie spectraleCollège de FranceAnnée 2023-2024Colloque - Géométries aléatoires et applications : Modèles d'images aléatoires et applications en mammographie digitaleIntervenante :Agnès DesolneuxCNRS, École normale supérieure Paris-SaclayRésuméIn this talk I will present several random image models that are else explicit (such as Gaussian models or Boolean models for instance), or more "implicit" (such as images generated by a neural network). I will discuss how these models are used to understand the detectability of some lesions in digital mammograms. I will also discuss another interest of such models, which is that they allow to perform virtual clinical trials.----Le terme « géométrie aléatoire » désigne tout processus permettant de construire de manière aléatoire un objet géométrique ou des familles d'objets géométriques. Un procédé simple consiste à assembler aléatoirement des éléments de base : sommets et arêtes dans le cas des graphes aléatoires, triangles ou carrés dans certains modèles de surfaces aléatoires, ou encore triangles, « pantalons » ou tétraèdres hyperboliques dans le cadre des géométries hyperboliques. La théorie des graphes aléatoires imprègne toutes les branches des mathématiques actuelles, des plus théoriques (théorie des groupes, algèbres d'opérateurs, etc.) aux plus appliquées (modélisation de réseaux de communication, par exemple). En mathématiques, l'approche probabiliste consiste à évaluer la probabilité qu'une propriété géométrique donnée apparaisse : lorsque l'on ne sait pas si un théorème est vrai, on peut tenter de démontrer qu'il l'est dans 99 % des cas.Une autre méthode classique pour générer des paysages aléatoires consiste à utiliser les séries de Fourier aléatoires, avec de nombreuses applications en théorie du signal ou en imagerie.En physique théorique, les géométries aléatoires sont au cœur de la théorie de la gravité quantique et d'autres théories des champs quantiques. Les différents aspects mathématiques s'y retrouvent curieusement entremêlés, par exemple, la combinatoire des quadrangulations ou des triangulations apparaît dans le calcul de certaines fonctions de partition.Ce colloque offrira un panorama non exhaustif des géométries aléatoires, couvrant des aspects allant des plus abstraits aux applications concrètes en imagerie et télécommunications.

Nalini AnantharamanGéométrie spectraleCollège de FranceAnnée 2023-2024Colloque - Géométries aléatoires et applications : Géométrie des excursions de champs aléatoires réguliers et inférence statistiqueIntervenante :Anne EstradeUniversité Paris CitéRésuméSome geometrical and topological features of the excursions of smooth random fields will be presented, such as their expected Lipschitz-Killing curvatures. The concerned random fields will be Gaussian or Gaussian based, but also shot-noise fields will be considered. Based on these features, one can statistically infer some informations on the underlying random field, in particular statistical tests (of Gaussianity, or of isotropy) and parameters estimations. A special focus on the two-dimensional case will be payed as it is the natural framework for image analysis.----Le terme « géométrie aléatoire » désigne tout processus permettant de construire de manière aléatoire un objet géométrique ou des familles d'objets géométriques. Un procédé simple consiste à assembler aléatoirement des éléments de base : sommets et arêtes dans le cas des graphes aléatoires, triangles ou carrés dans certains modèles de surfaces aléatoires, ou encore triangles, « pantalons » ou tétraèdres hyperboliques dans le cadre des géométries hyperboliques. La théorie des graphes aléatoires imprègne toutes les branches des mathématiques actuelles, des plus théoriques (théorie des groupes, algèbres d'opérateurs, etc.) aux plus appliquées (modélisation de réseaux de communication, par exemple). En mathématiques, l'approche probabiliste consiste à évaluer la probabilité qu'une propriété géométrique donnée apparaisse : lorsque l'on ne sait pas si un théorème est vrai, on peut tenter de démontrer qu'il l'est dans 99 % des cas.Une autre méthode classique pour générer des paysages aléatoires consiste à utiliser les séries de Fourier aléatoires, avec de nombreuses applications en théorie du signal ou en imagerie.En physique théorique, les géométries aléatoires sont au cœur de la théorie de la gravité quantique et d'autres théories des champs quantiques. Les différents aspects mathématiques s'y retrouvent curieusement entremêlés, par exemple, la combinatoire des quadrangulations ou des triangulations apparaît dans le calcul de certaines fonctions de partition.Ce colloque offrira un panorama non exhaustif des géométries aléatoires, couvrant des aspects allant des plus abstraits aux applications concrètes en imagerie et télécommunications.

Scopri come ricostruire oggetti, ambienti e persone in 3D partendo da foto o video con tre tecnologie all'avanguardia: fotogrammetria, NeRF e Gaussian Splatting. In questo video ti spiegherò, in modo semplice e pratico, come funzionano queste tecniche, quali sono le loro differenze e come utilizzarle per creare modelli tridimensionali realistici. Imparerai: - Cos'è la fotogrammetria e come puoi usarla con il tuo smartphone o fotocamera. - Come funziona il NeRF, una tecnologia basata sull'intelligenza artificiale per ricreare scene 3D incredibilmente realistiche. - Il Gaussian Splatting, il metodo più veloce e innovativo per generare nuvole di punti 3D. Se vuoi sapere come queste tecnologie stanno rivoluzionando settori come il gaming, il design 3D, la realtà virtuale e aumentata, questo video fa per te. Iscriviti al canale per non perderti tutorial, esempi pratici e approfondimenti sul 3D, l'AI e le nuove tecnologie creative.

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!My Intuitive Bayes Online Courses1:1 Mentorship with meOur theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!Visit our Patreon page to unlock exclusive Bayesian swag ;)Takeaways:Bayesian statistics offers a robust framework for econometric modeling.State space models provide a comprehensive way to understand time series data.Gaussian random walks serve as a foundational model in time series analysis.Innovations represent external shocks that can significantly impact forecasts.Understanding the assumptions behind models is key to effective forecasting.Complex models are not always better; simplicity can be powerful.Forecasting requires careful consideration of potential disruptions. Understanding observed and hidden states is crucial in modeling.Latent abilities can be modeled as Gaussian random walks.State space models can be highly flexible and diverse.Composability allows for the integration of different model components.Trends in time series should reflect real-world dynamics.Seasonality can be captured through Fourier bases.AR components help model residuals in time series data.Exogenous regression components can enhance state space models.Causal analysis in time series often involves interventions and counterfactuals.Time-varying regression allows for dynamic relationships between variables.Kalman filters were originally developed for tracking rockets in space.The Kalman filter iteratively updates beliefs based on new data.Missing data can be treated as hidden states in the Kalman filter framework.The Kalman filter is a practical application of Bayes' theorem in a sequential context.Understanding the dynamics of systems is crucial for effective modeling.The state space module in PyMC simplifies complex time series modeling tasks.Chapters:00:00 Introduction to Jesse Krabowski and Time Series Analysis04:33 Jesse's Journey into Bayesian Statistics10:51 Exploring State Space Models18:28 Understanding State Space Models and Their Components

Happy New Year's Eve! Before the clock strikes midnight I'm sharing a quick message of thanks and appreciation to all of my guests this year and to you, the listeners, who've downloaded episodes in 37 countries across the world.Our top listeners are from the U.S., Germany, Australia, Canada and the UK.Our Top podcast episode of 2024 is:Retail Therapy: My inner child said no to a $4500 Apple Vision ProFollowed by: Apple Vision Pro's paradigm shift with Ash Baccus-Clark, Speculative Researcher and Creative Strategist, ABC WorldwideLaunching JADU on mobile: a first-of-its-kind multiplayer AR fighting game, with Sarah Stumbo Huth, Gameplay Engineer, Jadu AROptimizing XR Development: The Power of Automated Testing with Shane Evans, Cofounder, GameDriverDECEMBER 2024 NEWSAndroid XR: The Gemini Era Comes to Headsets and Glasses - Google Android XR Resources Thread - @tom_krikorianHands on: Samsung's XR headset - Ben Lang, Road to VRInto the Scaniverse - YouTubeBuilding Into the Scaniverse in Niantic Studio - 8th Wall BlogImages & Ideas: A New Partnership With James Cameron's Lightstorm Vision - MetaAWE's The Future of VR Sports/Esports with Sonya Haskins - LinkedIn LivestreamQuest 2/3s/3 Software Update Issue - Meta Community ForumsNEW VR FITNESS WORKOUTS!  SynthRiders Experience - Barbie Dance'nDream DLCJane Fonda X Supernatural - YouTube trailerTaeBo Reboot - Billy Blanks Thanks for listening! Happy installing 2025. 

In this episode, Bill Bellows and Andrew Stotz explore the intersection of variation and quality through awareness of the "Paradigms of Variation.” In a progression from acceptability to desirability, Bill created this 4-part model to offer economic insights for differentiating “Zero Defect” quality from “Loss Function" quality," with the aim of avoiding confusion between precision and accuracy when desirability is the choice.   Learn how to decide which paradigm your quality management system fits into! TRANSCRIPT 0:00:02.5 Andrew Stotz: My name is Andrew Stotz and I'll be your host as we dive deeper into the teachings of Dr. W. Edwards Deming. Today I'm continuing my discussion with Bill Bellows, who has spent 31 years helping people apply Dr. Deming's ideas to become aware of how their thinking is holding them back from their biggest opportunities. This is episode 7, The Paradigms of Variation. Bill, take it away.   0:00:30.3 Bill Bellows: Thank you, Andrew, and welcome to our listeners, as well as viewers, if you have access to the viewing version. Yeah, so I went back and listened to Episode 6. I'm going out bike riding 2-3 hours a day, so I listened to the podcast, listened to other things, stop and write down. Let me go write that down. And, so, we're going to pick up today on some major themes. And, what I keep coming back to is, is I think the difference between acceptability and desirability is the difference between how most companies operate and how a company inspired by Dr. Deming would operate.   0:01:29.3 BB: And, I just think of, if there was no difference between the two, then... Well, lemme even back up. I mentioned last time we were talking about why my wife and I buy Toyotas. And, yes, we've had one terrible buy, which I continue to talk about. [laughter] And, it's fun because it's just a reminder that even a company like Toyota can deliver a really lousy product, which we were unfortunate to have purchased. And, we're not the only ones that, and they've rebounded and they've apologized, they've had issues. There's no doubt about that. They have issues, but they have notably been inspired by Dr. Deming.   0:02:30.6 BB: The one thing I brought up last time was relative on this thinking of acceptability, desirability, where acceptability is looking at things and saying it's a quality system of good and bad. It's acceptable, which is good and unacceptable is not good. And, that's how most organizations view quality. Again, the focus of this series is Misunderstanding Quality. Our previous series was broadly looking at implications for Dr. Deming's ideas. And, here our focus is quality. And, so what I'm trying to get across here is quality management, traditional quality management.   0:03:17.4 BB: In most organizations, in all organizations I've ever interacted with is acceptability basis, good parts and bad parts. It's a measurement system of it meets requirements, we ship it, if it meets requirements, we buy it. And, I'm not saying there's anything wrong with that, but I don't think a system focused on acceptability can explain... To me, it does not explain the incredible reliability I have personally experienced in Toyota products.   0:03:46.9 BB: Now, I'm working with a graduate student and I wanna pursue that as a research topic in the spring, 'cause for all I know, the reliability of components in all cars has improved. I don't know if it's, I only by Toyota, 'cause so this woman I've met recently and I'm mentoring her and we're working on a research project. And, I thought recently, I'd like... And, I'm not sure how to do this, but I just know, I think I've mentioned I worked at my father's gas station back in the '70s and I remember replacing water pumps and alternators and all this stuff. This was before Japanese cars were everywhere. There were Japanese cars, but not like you see today.   0:04:33.3 BB: And, so I'm just used to all those components being routinely replaced. And, all I know is I don't routinely replace anything but the battery and the tires and change the oil. I think that's about it. Everything else is pretty good. But, I do think the differentiation between Toyota and most other companies is their appreciation of desirability and how to manage desirability. And, that's why I keep coming back to this as a theme for these sessions. And, what I think is a differentiation between a Deming view of quality and all other views of quality. What I tried to say last time is I just give you indications of a focus on acceptability. It's a quality system which looks at things that are good or things that are bad. It's, last time we talked about category thinking. It's black and white thinking. If the parts are good, then the mindset, if they're good, then they fit.   0:05:38.4 BB: Well, with a focus on continuum thinking, then you have the understanding that there's variation in good. And, that leads to variation in fit and variation in performance. And, that's a sense of things are relatively good, not absolutely good, whereas black and white category thinking is acceptability. They're all good. And, if they're all good, then they should all fit. I was, when I was at Rocketdyne, met, and the one thing I wanted to point out is... Again, as I said in the past, so much of what I'm sharing with the audience and people I've met through these podcasts or people I'm mentoring, helping them bring these ideas to their respective organizations or their consultants, whatever it is.   0:06:29.0 BB: And, so I like to provide examples in here for things for them to go off and try. You at the end of each podcast encourage them to reach out to me, a number of them have, and from that I've learned a great deal. And, so one guy was... A guy I was working with at Rocketdyne, he was at a site that did final assembly of rocket engine components. And, so one thing I'd say is the people who... And for those listening, if you wanna find people in your organization that would really value the difference between an acceptability focus and a desirability focus, find the people that do assembly, find the people that put things together. 'Cause the ones that machine the holes, they think all the holes are good. People that make the tubes, all the tubes are good. But, find the people that are trying to put the tubes into the holes. Those are the people I loved working with because they were the ones that felt the difference every day.   0:07:31.1 BB: And, so I was in a workshop for a week or so. And there's two people ahead of me. They came from this final assembly operation of Rocketdyne. And, during a break, I was trying to clarify some of the things I had said and I used, I shared with them an example of how when we focused on not the tubes by themselves or the holes by themselves, that we focused on how well the tubes go into the holes, which has a lot to do with the clearance between them and the idea that nobody owns the clearance. One person owns one part, one owns another. And, what we realized is if we focused on the relationship, what a big difference it made. So I'm explaining it to him and he turns to me and he says, he's like, "Oh, my God," he says, "I've got hundreds of turbine blades and a bunch of turbine wheels and the blades slide into the wheel." And he says, "I can't get the blades onto the wheel."   0:08:31.0 BB: And I said, "But they're all good." He says, "They're all good." But he said, "Well, what you're now explaining to me is why they don't go together. Why I have this headache." So I said, "Well, do you know where the blades come from?" He says, "yeah". And I said, "Do you know where the wheels come from?" He says, "yeah". I said, "Well, why don't you call them up and talk to them?" He says, "There's no reason for a phone call 'cause all they're going to say is, "Why are you calling me? They're all good." So, he just walked away with his head exploding 'cause he's got all these things.   0:09:05.8 BB: And, so I use that for our listeners is if you want to find people that would really resonate with the difference between acceptable and desirable, talk to the people that have to put things together. There you will find... And, so my strategy was, get them smart. Now they have to be patient with the people upstream 'cause the people upstream are not deliberately doing what they're doing to them. So, what you don't want to do is have them get... You want their consciousness to go up but you now wanna use them to talk to the component people. Now you've got a conversation. Otherwise, the component people say, "Why are you talking to me? Everything I do is good."   0:09:51.6 BB: So, I just want to talk at this point, just to reinforce that I think there's something going on with Toyota that is very intentional about managing desirability when it makes sense using acceptability. So, it's a choice. And, so indications of a focus on desirability is when you look at options that are acceptable and you say, "Of all these apples, I want this one. It's the ripest. Of all these donuts, I want this one. It's got the most sprinkles. Of all these parking spots, I want this one. It's a little bit wider than the other. I want this surgeon. I want this professor for this course."   0:10:33.8 BB: All right. So, what we're saying "is of all the choices, I want this one". So, some new ideas I want to get into tonight are the Paradigms of Variation A, B, C, D, and E. Paradigm A we looked at in the past. That's just acceptability. Does it meet requirements or not? The quality focus is achieving zero defects. And tonight I want to get into B and C. The next time we'll look at D and E. In explaining these ideas recently to someone who listened to one of our previous podcasts and were focusing on, he started asking about decision making. And that got me thinking about, of course, I took years ago decision making with Kepner and Tregoe. And there they talk about decisions. We're gonna look, we're gonna go buy a car, go buy a house. We're gonna make a decision.   0:11:29.4 BB: And, once you decide on the decision, you then list the criteria of the decision. And you come up with all the things you want in this decision. And then you look at each of them and you say, "is it a must or a want"? And let's say you're looking at houses. It could be a lot of houses to go look at. What makes this focus on acceptability, it's musts and wants. And must is very much acceptability. So you say: "We're looking for a house that must be one story, it must be in the middle of the block. The house must be in the middle of the block. It must have four bedrooms, must have two bathrooms". So now when you're looking at all these houses, acceptability says "I'm only gonna look at the ones that meet those requirements". And, so now the strategy is to go from hundreds of options down to an order of magnitude less.   0:12:25.1 BB: Now we're going to get it down to maybe 20. Now you look at the wants. So you've got an original list of all the things, the criteria, and you look at each one and say, "is it a must, is it a want"? And what I've just said is the first screening is all the ones that pass the must get into the next category. Well, with the Kepner-Tregoe folks, they talk about must, which is acceptability, and the wants are about desirability.   0:12:51.4 BB: And then here it ties into Dr. Taguchi's mindset, and we'll look at Taguchi in a future session. Taguchi looks at a characteristic of quality, such as the diameter of a hole, the performance of an automobile, miles per gallon. And he says, in terms of desirability, there's three different targets. There is desirability, I want the smallest possible value. So, if you're buying a house, it could be, I want the lowest possible electric bills where zero is the goal. It's not gonna be zero, but I'm looking, of all the ones that pass the must, now I'm looking at all the houses, and I'm saying "I want the lowest possible electric bill". That's a Smaller-is-Best.   0:13:35.9 BB: Larger-is-Best is I want something which is as big as possible. It could be I want the most roof facing the sun, in case I put solar in. That's a Larger-is-Best characteristic, where Taguchi would say the ideal is infinity, but the bigger, the better, as opposed to Smaller-is-Better. And, the other characteristic is what Taguchi calls Nominal-is-Best, is I have an ideal single value in mind. And in each case, the reason I point that out is that desirability is about going past acceptability and saying amongst all the things that are acceptable, I want the smallest, I want the largest, or I want this. It is a preference for one of those.   0:14:19.4 BB: So, I thought... I was using that to explain to this friend the other day, and I thought that would be nice to tie in here. That desirability is a focus on of all the things that meet requirements, now I want to go one step further. That's just not enough. All right, so now let's get into Paradigms B and C. And I want to use an exercise we used in the first series. And, the idea for our audience is imagine a quality characteristic having a lower requirement, a minimum, otherwise known as the lower spec, the lower tolerance. So, there's a minimum value, and then there's a maximum value. And, when I do this in my classes, I say "let's say the quality characteristic is the outer diameter of a tube." And, then so what I'd like the audience to appreciate is we've got a min and a max.   0:15:18.9 BB: And, then imagine your job as listener is to make the decision as to who to buy from. And. let's say we've got two suppliers that are ready to provide us with their product, these tubes that we're gonna buy. And, your job as a listener is to make the decision as to who to buy from. Who are we going to buy from? And, so we go off and we tell them, "Here's the min, here's the max," and they come back. And, they each give us a distribution. And, so what I'd like the audience to think about is a distribution. Just think very simply of two normal distributions, two Gaussian distributions. And, let's say the first distribution goes all the way from the min to the max. It takes up the entire range.   0:16:08.5 AS: So wide and flat.   0:16:12.1 BB: Wide and flat. That's supplier one. And supplier two, let's say is maybe three quarters of the way over. It's incredibly uniform. It uses a very small fraction of the tolerance. So that's tall and narrow. That's distribution two as opposed to wide and flat. So, imagine we've got those two to buy from. But imagine also, and this is a highly idealized scenario. And, I use this and this is why I want to share it with our audience. Because it becomes a great way of diving into what I think is a lot of confusion about meeting requirements. And, so what I want you to imagine is that no matter who you buy from, they both promise that they will deliver at the same price per tube.   0:17:00.8 BB: So, no matter who you buy from, price-wise, they are identical. To which I'd say that's highly idealized, but that's a given. Criteria number two, the delivery rates are the same. So, we cannot differentiate on delivery. We cannot differentiate on price. The third condition we find out is that everything they deliver meets requirements, 100%. So, if there is any scrap and rework, they don't ship that to us. So, everything they deliver meets requirements. And, again, that's highly idealized.   0:17:41.6 BB: Number four is the distributions are in control. And, that means that the processes are predictable and stable. And, that's guaranteed. So, imagine these distributions day by day every order is the same shape, the same average, the same amount of variation. Also, it will never change. It will never change. And, the other thing I want to point out in this fourth point here is that your job as the buyer is to buy these. They are used as is within our organization. , 0:18:15.5 BB: And, the fifth point is that there's a min and a max. And, so I've been using this exercise for, gosh, going back to 1995, and I throw it out there and then I show them the distributions. I say "same price, same schedule, delivery rate, everything meets requirements, distributions never change shape or location. You're going to use as is. And there's the min, there's the max. Who do you buy from?" And, I give people not only do we buy from one or two, but I also say I'll give you a third option.   0:18:51.5 BB: The third option is it doesn't matter. It doesn't matter. So, what I find is that three quarters of the audience will take distribution two, the narrow one. And when I ask them, why do you like distribution two? They say, "because it has less variation". I then say, "From what?" Then they say, "From each other." And, that's what a standard deviation is, variation from each other. So roughly 75% plus and minus...   [overlapping conversation]   0:19:25.8 AS: When you say of each other, you're talking about each other curve or each other item in the...   0:19:31.3 BB: Each other tube.  So, the amount of variation from all the tubes are close together, so the variation from each other.   0:19:38.6 AS: Okay. Each item. Yeah, okay.   0:19:41.8 BB: Standard deviation is the average variation from the average value. So, when I ask them, why do you like two? Okay, and then I asked the ones who take the wide one in the middle, I say, "why do you like that one," and they say because... And, actually, we'll come back to that. This is pretty funny. They will take that, but a very small percent say it doesn't matter, and here's what's interesting, if I didn't show the distributions, if all I did was say there's two suppliers out there, the same price, same schedule, that guarantee zero defects, the results will never change. Here's the min, here's the max, I'm willing to bet if I didn't show the distributions, people would say "it doesn't matter, I'll take either one". But, as soon as I show them the distributions, they want the narrow one. And, I use this for our attendees, this is a great way to show people that they really don't believe in tolerances, 'cause as soon as you go past meeting requirements, what you're really saying is, there's a higher bar.   0:21:05.6 AS: Okay, so requirements would be... Or, tolerances would be the extremes of that flat, wide curve. And, any one of those outcomes meets the tolerance.   0:21:17.5 BB: Yes, and so for companies that are striving to meet requirements, why is it when I give you two distributions that meet requirements... Why is it when I show you the distributions, and I'm willing to bet if I don't show you the distributions and all you know is they're 100% good, then you say "well, it doesn't matter," Well then what changes when I show you the distributions?   0:21:43.6 AS: I know why I'd choose the narrow one.   0:21:48.1 BB: Go ahead.   0:21:49.1 AS: I know how damn hard it is to reduce variation and I forget about any tolerance of anything, if I have two companies that show me a wide distribution, and another one shows me a narrow one, and let's say it's accurate. I'm much more impressed with how a company can do the same exact output as another company, the same product that they're trying to deliver, but they are producing a much more narrow range of outcome, which could be that they just have automation in their production line and the other one has manual.   0:22:27.4 BB: And, I have seen that within Rocketdyne, I've seen processes do that. I have seen the wide become the narrow through automation. Yeah. Okay, so hold that thought then. So, what I do in my graduate classes is I show that... Not only do I give them two options, I give them four options. So, I throw in two other distributions, but really what it comes down to is the wide one versus the narrow one, and then the other two, I throw in there that usually aren't taken, they're distractions. All right, so what I'll do in a graduate class in quality management is to show that and get the results I just showed. If I present the same exercise and then say, "imagine the average value of distribution one, the middle of distribution one, imagine that is the ideal value".   0:23:24.7 AS: That, you're talking about the wide and flat.   0:23:28.4 BB: Yes. So, all I do is I go back to the entire exercise and now I add in a line at the average of the wide distribution, and then go through and ask one more time, who would you take.   0:23:46.3 AS: So, now the dilemma that the listener has is that now we have a, within limits, within tolerances, we have a wide but flat distribution that's centered on the middle point between the upper and lower tolerance.   0:24:06.4 BB: Yeah, yes.   0:24:08.8 AS: And, then we have... Go ahead.   0:24:11.7 BB: Well, yeah, that is distribution one, same as the first part, we went through this, and all I'm doing now is saying, "imagine the average value of the middle is said to be the ideal value".   0:24:29.4 AS: And, now you're gonna tell us that the narrow one is not on that central or ideal value.   0:24:36.2 BB: No, that is still where it is at the three-quarter point, all I've done is now said, this is desirability. I'm now saying "that is the ideal value, that is the target, that is the value we prefer". And, people still take the narrowest distribution number two.   0:24:58.8 AS: I wouldn't take the narrow one because I would think that the company would have to prove to me that they can shift that narrow curve.   0:25:06.6 BB: Well, okay, and I'm glad you brought that up because according to the explanation I gave of equal price, equal schedule, meets requirements. I deliberately put in the criteria that you have to use them as is. So, now I'm forcing people to choose between the narrowest one over there at the three-quarter point, and the wide one on target. And, there's no doubt if I gave them the option of taking the narrow distribution and sliding it over, they would. Every single person would do that. But, when I give you a choice of, okay, now what? So, two things here, one is, is it calling out the ideal of value, 'cause desirability is not just beyond acceptability, it is saying, "I desire this value, I want this parking spot, I want this apple, I want this value". And, that's something we've been alluding to earlier, but that's what I wanna call out today is that...   0:26:13.7 BB: So, in other words, when I presented the exercise of the two distributions, without calling out what's desirable, all I'm doing is saying they're both acceptable, which do you prefer? But, instead of saying it doesn't matter, I'd like the narrowest one, and it may well be what people are doing is exactly what you're saying is the narrowest one seems better and easily could be for what you explained.   0:26:40.8 BB: But, what's interesting is, even when I call out what's desirable as the value, people will take the narrowest distribution, and so now what I wanna add to our prior conversation is Paradigm A, acceptability, the Paradigm A response would be, it doesn't matter. Choosing the narrowest one, otherwise known as precision, we're very precisely hitting that value, small standard deviation, that's what I refer to as Paradigm B, piece-to -piece consistency. Paradigm C is desirability being on the ideal value, that's piece-to-target consistency. And, in Dr. Taguchi's work, what he's talking about is the impact downstream of not just looking at the tubes, but when you look at how the tubes are inserted into a hole, perhaps, then what he's saying is that the reason you would call out the desirable value is what you're saying is how this tube integrates in a bigger system matters, which is why I want this value.   0:27:54.2 AS: Okay, so let's go back, A, meet requirements, that's acceptability. Anything within those tolerances we can accept. B is a narrow distribution, what you called precision or piece-to -piece consistency. And what was C?   0:28:12.8 BB: C is, I'll take the wide distribution where the average value is on target, that's piece- to-target consistency. Otherwise known as accuracy.   0:28:27.3 AS: Okay. Target consistency, otherwise known as accuracy. All right, and then precision around D is precision around the ideal value.   0:28:37.7 BB: Well, for those that want to take the narrowest one and slide it over, what you're now doing is saying, "I'm gonna start with precision, and I'm going to focus on the ideal". Now, what you're doing is saying, "step one is precision, step two is accuracy".   0:28:56.4 AS: Okay. And step three or D?   0:29:00.9 BB: Paradigm D?   0:29:02.7 AS: Yeah.   0:29:02.7 BB: Is that what you're... Yeah. Paradigm D would be the ability to produce, to move the distribution as needed to different locations.   0:29:17.4 AS: The narrow distribution?   0:29:18.9 BB: Yes, and so I'll give you an example in terms of, let's say tennis, Paradigm A in tennis is just to get the ball across the net. I just wanna get it somewhere on the other side of the court, right. Now that may be okay if you and I are neighbors, but that doesn't get us into professional level. Paradigm B, is I can hit it consistently to one place on your side of the court. Now, I can't control that location, but boy, I can get that location every single time. Next thing you know, you know exactly where the ball is going, and that's Paradigm B.  Paradigm C is I can move it to where I want it to go, which you will eventually figure out, so I can control where it goes. Paradigm D is I can consistently hit any side of the court on the fly.   0:30:11.0 BB: So, Paradigm D is I can take that narrow distribution and move it around for different customers, different applications, and Dr. Taguchi refers to that as Technology Development, and what Taguchi is talking about is developing a technology which has incredible precision in providing your sales people the ability to move the next move it to accuracy and to sell that product by tuning it to different customers as you would in sports, move the ball around to the other side of the court. So now you're going to the point that you've got incredible precision, and now you've got “on demand accuracy,” that's Paradigm D.  Paradigm C is I can do one-size-fits-all which is, which may be all you need for the application.   0:31:06.9 AS: I wanna separate the Paradigm B, the narrow distribution and that's precision around some value versus Paradigm D is precision around the ideal value.   0:31:20.7 BB: And, the idea is that desirability is about an ideal value. And, so if we're talking piece-to-piece consistency, that means it's uniform, but I'm not paying attention to... I have a value in mind that I want. And that's the difference between Dr. Taguchi's work, I mean, it's the ability to be precise. Again, accuracy, desirability is I have an ideal value in mind. And acceptability is it doesn't really matter.  Precision is uniformity without accuracy. And so, if you are... What Dr. Taguchi is talking about is, is depending on how what you're delivering integrates, being consistent may cause the person downstream to consistently need a hammer to get the tube into the hole.   0:32:24.2 BB: So, it's consistent, but what you're now saying, what Taguchi is saying is, if you pay attention to where you are within requirement, which is desirability, then you can improve integration. And, that is my explanation for why Toyota's products have incredibly reliability, that they are focusing on integration, not just uniformity and precision by itself.   0:32:49.8 AS: I would love to put this in the context of a dart thrower. The Paradigm A meeting requirability or acceptability, they stand way behind and they throw and they hit the overall dart board.   0:33:04.3 BB: Dart board. It's on the board. Yes.   0:33:07.2 AS: And, the narrow distribution is, well, they hit the same spot over to the left, right towards the edge, they hit that spot consistently. And, then basically, I'm gonna jump to D just because I'm imagining that I'm just gonna ask the guy, Hey, can you just move over just a little bit, and I'mma move them over about a half a foot, and when I do, you're gonna start throwing that dart right at the same location, but over to the right, meaning at the target. The center of the dart...   0:33:43.9 BB: The bull's eye. Yeah. Yeah, well, that's... And you call that C or D?   0:33:47.6 AS: I call that D.   0:33:49.5 BB: No, I would say, let's call that C being on target, meaning that C is, for games of darts where the most points are being on the bull's eye, that's Paradigm C.   0:34:04.0 AS: So accuracy, yeah.   0:34:05.4 BB: Paradigm D would be a game in which the ideal value changes. So now, okay, now I watch the... When I play darts, I'm sure there's lots of darts games, but one game we used to play it in our cellar at home was baseball. So, the dart board is divided into has numbers one, two, three through, and you'd go to... There'd be a wedge number one, a wedge number two, a wedge number three, that's Paradigm D that I could hit the different wedges on demand. But that's what it is. So A is anywhere in. B is consistent, precision, but again, the idea is if you can move that, but now what we're talking about is, is there an impulse to move it or are we happy just being precise? What Taguchi's talking about is the value proposition of desirability is to take precision, take that uniformity and move it to the ideal value, and what you've just done and doing so, you're now focusing on not this characteristic in isolation, you're now focusing on how this characteristic meshes with another characteristic. And, it's not just one thing in isolation, one thing in isolation does not give you a highly reliable automobile.   0:35:38.9 AS: Is there anything you wanna add to that, or are you ready to sum it up?   0:35:45.0 BB: No, that's it. The big summation is, we've been building up to the contrast between acceptable and desirable. I just wanted to add some more fidelity. Desirable is I have a value in mind, which Dr. Taguchi referred to as a target. So, for people at home, in the kitchen, the target value could be exactly one cup of flour. We talked earlier about our daughter, when she worked in a coffee shop and then, and at home she'd give us these recipes for making coffee and it'd be dad, exactly this amount of coffee and exactly that. And, we had a scale, it wasn't just anywhere between. She'd say "dad, you have to get a scale." I mean she was... We started calling her the coffee snob, 'cause it was very, this amount, this amount. So, in the kitchen then it's about precisely one cup. Precisely one this. And that's desirability.   0:36:40.6 AS: And, I was just thinking, the best word for that is bull's eye!   0:36:48.3 BB: Yes.   0:36:48.8 AS: You hit it right on the spot.   0:36:50.6 BB: Yeah.   0:36:51.6 AS: Great. Well, Bill, on behalf of everyone at The Deming Institute, I wanna thank you again for this discussion. It was not only acceptable, it was desirable. For listeners, remember to go to deming.org to continue your journey. And, if you want to keep in touch with Bill, just find him on LinkedIn. He'll reply. This is your host, Andrew Stotz, and I leave you with one of my favorite quotes from Dr. Deming, "people are entitled to joy in work."

Recent advances in latent diffusion-based generative models for portrait image animation, such as Hallo, have achieved impressive results in short-duration video synthesis. In this paper, we present updates to Hallo, introducing several design enhancements to extend its capabilities. First, we extend the method to produce long-duration videos. To address substantial challenges such as appearance drift and temporal artifacts, we investigate augmentation strategies within the image space of conditional motion frames. Specifically, we introduce a patch-drop technique augmented with Gaussian noise to enhance visual consistency and temporal coherence over long duration. Second, we achieve 4K resolution portrait video generation. To accomplish this, we implement vector quantization of latent codes and apply temporal alignment techniques to maintain coherence across the temporal dimension. By integrating a high-quality decoder, we realize visual synthesis at 4K resolution. Third, we incorporate adjustable semantic textual labels for portrait expressions as conditional inputs. This extends beyond traditional audio cues to improve controllability and increase the diversity of the generated content. To the best of our knowledge, Hallo2, proposed in this paper, is the first method to achieve 4K resolution and generate hour-long, audio-driven portrait image animations enhanced with textual prompts. We have conducted extensive experiments to evaluate our method on publicly available datasets, including HDTF, CelebV, and our introduced"Wild"dataset. The experimental results demonstrate that our approach achieves state-of-the-art performance in long-duration portrait video animation, successfully generating rich and controllable content at 4K resolution for duration extending up to tens of minutes. Project page https://fudan-generative-vision.github.io/hallo2 2024: Jiahao Cui, Hui Li, Yao Yao, Hao Zhu, Hanlin Shang, Kaihui Cheng, Hang Zhou, Siyu Zhu, Jingdong Wang https://arxiv.org/pdf/2410.07718

Hassaan Raza is the cofounder and CEO of Tavus, a video API platform for digital twins. They've raised more than $28M in funding from investors such as Sequoia and Scale VP. Hassaan's favorite book: Go Like Hell (Author: A. J. Baime)(00:01) Introduction(00:38) Overview of AI in video generation(01:44) AI models used in video generation(03:35) Capturing intricate facial movements in real-time(06:46) Data capture and 3D modeling from basic video input(09:01) Explanation of neural radiance fields and Gaussian splatting(10:14) Capturing facial expressions for video generation(15:22) Temporal coherence in video generation(18:05) Challenges in conversational video, including lip-syncing and emotion alignment(20:38) Inference challenges in conversational video(22:47) Bottlenecks in the pipeline: LLMs and time-to-first-token(26:58) Multimodal models and trade-offs(27:36) Advice for founders running API businesses(30:04) Pitfalls to avoid in API businesses(32:15) Technological breakthroughs in AI(34:10) Rapid-fire round--------Where to find Prateek Joshi: Newsletter: https://prateekjoshi.substack.com Website: https://prateekj.com LinkedIn: https://www.linkedin.com/in/prateek-joshi-91047b19 Twitter: https://twitter.com/prateekvjoshi 

In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution. 2024: Yanbo Zhang, Benedikt Hartl, Hananel Hazan, Michael Levin https://arxiv.org/pdf/2410.02543

I interviewed Marcello Typrin Product Director at Reality Labs, at Meta Connect 2024 about the Hyperscape Demo. See more context in the rough transcript below. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

I interviewed Tipatat Chennavasin, General Partner and co-founder of Venture Reality Fund, at Meta Connect 2024 about why he has gone all-in with Gaussian Splatting by investing in the Gracia.ai and their viewer that's on the Quest store. We also talk about some of the news from Meta Connect, Quest 3S excitement, the merits of AAA games like Batman Arkham Shadow to bring more people into VR, and other some experiences and demos that caught his attention at Meta Connect 2024. See more context in the rough transcript below. This is a listener-supported podcast through the Voices of VR Patreon. Music: Fatality

Gaussian splatting or for short G splats is the latest method to capture objects or entire complex 3D scenes using only a video. It falls under the category of Radiance Fields much like Nerfs but it's actually quite different: they are faster to train, you can achieve an unprecedented level of realism BUT have they lived up to the hype? Today we will figure it out with Georgii Vysotskii, the co-founder and CEO of Gracia.ai a deep tech company specializing in the visualization of Gaussian splatting and redefining the way moments are captured and experienced…. Even in VR. We will talk about The story of Gracia and how they brought Splats to VR A type of Gaussian splat you might not have heard about The usecase unlocked by Gaussian splatting's fast rendering speed How they are tackling a key challenge when it comes to distribution *** Subscribe to XR AI Spotlight weekly newsletter

Domna Ladopoulou, a researcher in the Department of Statistical Science at UCL, is working on improving the efficiency and reliability of wind energy production through statistical and machine learning modelling approaches. Her research focuses on developing a probabilistic condition monitoring system for wind farms using SCADA data to detect faults and failures early. This system aims to enhance the sustainability of wind farms by reducing maintenance costs and improving overall reliability. Donna's methodology involves non-parametric probabilistic methods like Gaussian processes and probabilistic neural networks, which offer flexibility and computational efficiency. She emphasizes the importance of informed decision-making in sustainability and the potential for her research to be scaled globally, particularly in regions with high wind power reliance. Date of episode recording: 2024-05-30T00:00:00Z Duration: 00:17:34 Language of episode: English Presenter:Stephanie Dickinson Guests: Domna Ladopoulou Producer: Nathan Green

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!My Intuitive Bayes Online Courses1:1 Mentorship with meOur theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!Visit our Patreon page to unlock exclusive Bayesian swag ;)Takeaways:Bayesian statistics is a powerful framework for handling complex problems, making use of prior knowledge, and excelling with limited data.Bayesian statistics provides a framework for updating beliefs and making predictions based on prior knowledge and observed data.Bayesian methods allow for the explicit incorporation of prior assumptions, which can provide structure and improve the reliability of the analysis.There are several Bayesian frameworks available, such as PyMC, Stan, and Bambi, each with its own strengths and features.PyMC is a powerful library for Bayesian modeling that allows for flexible and efficient computation.For beginners, it is recommended to start with introductory courses or resources that provide a step-by-step approach to learning Bayesian statistics.PyTensor leverages GPU acceleration and complex graph optimizations to improve the performance and scalability of Bayesian models.ArviZ is a library for post-modeling workflows in Bayesian statistics, providing tools for model diagnostics and result visualization.Gaussian processes are versatile non-parametric models that can be used for spatial and temporal data analysis in Bayesian statistics.Chapters:00:00 Introduction to Bayesian Statistics07:32 Advantages of Bayesian Methods16:22 Incorporating Priors in Models23:26 Modeling Causal Relationships30:03 Introduction to PyMC, Stan, and Bambi34:30 Choosing the Right Bayesian Framework39:20 Getting Started with Bayesian Statistics44:39 Understanding Bayesian Statistics and PyMC49:01 Leveraging PyTensor for Improved Performance and Scalability01:02:37 Exploring Post-Modeling Workflows with ArviZ01:08:30 The Power of Gaussian Processes in Bayesian ModelingThank you to my Patrons for making this episode possible!Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna,...

If you see this in time, join our emergency LLM paper club on the Llama 3 paper!For everyone else, join our special AI in Action club on the Latent Space Discord for a special feature with the Cursor cofounders on Composer, their newest coding agent!Today, Meta is officially releasing the largest and most capable open model to date, Llama3-405B, a dense transformer trained on 15T tokens that beats GPT-4 on all major benchmarks:The 8B and 70B models from the April Llama 3 release have also received serious spec bumps, warranting the new label of Llama 3.1.If you are curious about the infra / hardware side, go check out our episode with Soumith Chintala, one of the AI infra leads at Meta. Today we have Thomas Scialom, who led Llama2 and now Llama3 post-training, so we spent most of our time on pre-training (synthetic data, data pipelines, scaling laws, etc) and post-training (RLHF vs instruction tuning, evals, tool calling).Synthetic data is all you needLlama3 was trained on 15T tokens, 7x more than Llama2 and with 4 times as much code and 30 different languages represented. But as Thomas beautifully put it:“My intuition is that the web is full of s**t in terms of text, and training on those tokens is a waste of compute.” “Llama 3 post-training doesn't have any human written answers there basically… It's just leveraging pure synthetic data from Llama 2.”While it is well speculated that the 8B and 70B were "offline distillations" of the 405B, there are a good deal more synthetic data elements to Llama 3.1 than the expected. The paper explicitly calls out:* SFT for Code: 3 approaches for synthetic data for the 405B bootstrapping itself with code execution feedback, programming language translation, and docs backtranslation.* SFT for Math: The Llama 3 paper credits the Let's Verify Step By Step authors, who we interviewed at ICLR:* SFT for Multilinguality: "To collect higher quality human annotations in non-English languages, we train a multilingual expert by branching off the pre-training run and continuing to pre-train on a data mix that consists of 90% multilingualtokens."* SFT for Long Context: "It is largely impractical to get humans to annotate such examples due to the tedious and time-consuming nature of reading lengthy contexts, so we predominantly rely on synthetic data to fill this gap. We use earlier versions of Llama 3 to generate synthetic data based on the key long-context use-cases: (possibly multi-turn) question-answering, summarization for long documents, and reasoning over code repositories, and describe them in greater detail below"* SFT for Tool Use: trained for Brave Search, Wolfram Alpha, and a Python Interpreter (a special new ipython role) for single, nested, parallel, and multiturn function calling.* RLHF: DPO preference data was used extensively on Llama 2 generations. This is something we partially covered in RLHF 201: humans are often better at judging between two options (i.e. which of two poems they prefer) than creating one (writing one from scratch). Similarly, models might not be great at creating text but they can be good at classifying their quality.Last but not least, Llama 3.1 received a license update explicitly allowing its use for synthetic data generation.Llama2 was also used as a classifier for all pre-training data that went into the model. It both labelled it by quality so that bad tokens were removed, but also used type (i.e. science, law, politics) to achieve a balanced data mix. Tokenizer size mattersThe tokens vocab of a model is the collection of all tokens that the model uses. Llama2 had a 34,000 tokens vocab, GPT-4 has 100,000, and 4o went up to 200,000. Llama3 went up 4x to 128,000 tokens. You can find the GPT-4 vocab list on Github.This is something that people gloss over, but there are many reason why a large vocab matters:* More tokens allow it to represent more concepts, and then be better at understanding the nuances.* The larger the tokenizer, the less tokens you need for the same amount of text, extending the perceived context size. In Llama3's case, that's ~30% more text due to the tokenizer upgrade. * With the same amount of compute you can train more knowledge into the model as you need fewer steps.The smaller the model, the larger the impact that the tokenizer size will have on it. You can listen at 55:24 for a deeper explanation.Dense models = 1 Expert MoEsMany people on X asked “why not MoE?”, and Thomas' answer was pretty clever: dense models are just MoEs with 1 expert :)[00:28:06]: I heard that question a lot, different aspects there. Why not MoE in the future? The other thing is, I think a dense model is just one specific variation of the model for an hyperparameter for an MOE with basically one expert. So it's just an hyperparameter we haven't optimized a lot yet, but we have some stuff ongoing and that's an hyperparameter we'll explore in the future.Basically… wait and see!Llama4Meta already started training Llama4 in June, and it sounds like one of the big focuses will be around agents. Thomas was one of the authors behind GAIA (listen to our interview with Thomas in our ICLR recap) and has been working on agent tooling for a while with things like Toolformer. Current models have “a gap of intelligence” when it comes to agentic workflows, as they are unable to plan without the user relying on prompting techniques and loops like ReAct, Chain of Thought, or frameworks like Autogen and Crew. That may be fixed soon?

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What progress have we made on automated auditing?, published by Lawrence Chan on July 6, 2024 on The AI Alignment Forum. One use case for model internals work is to perform automated auditing of models: https://www.alignmentforum.org/posts/cQwT8asti3kyA62zc/automating-auditing-an-ambitious-concrete-technical-research That is, given a specification of intended behavior, the attacker produces a model that doesn't satisfy the spec, and the auditor needs to determine how the model doesn't satisfy the spec. This is closely related to static backdoor detection: given a model M, determine if there exists a backdoor function that, for any input, transforms that input to one where M has different behavior.[1] There's some theoretical work (Goldwasser et al. 2022) arguing that for some model classes, static backdoor detection is impossible even given white-box model access -- specifically, they prove their results for random feature regression and (the very similar setting) of wide 1-layer ReLU networks. Relatedly, there's been some work looking at provably bounding model performance (Gross et al. 2024) -- if this succeeds on "real" models and "real" specification, then this would solve the automated auditing game. But the results so far are on toy transformers, and are quite weak in general (in part because the task is so difficult).[2] Probably the most relevant work is Halawi et al. 2024's Covert Malicious Finetuning (CMFT), where they demonstrate that it's possible to use finetuning to insert jailbreaks and extract harmful work, in ways that are hard to detect with ordinary harmlessness classifiers.[3] As this is machine learning, just because something is impossible in theory and difficult on toy models doesn't mean we can't do this in practice. It seems plausible to me that we've demonstrated non-zero empirical results in terms of automatically auditing model internals. So I'm curious: how much progress have we made on automated auditing empirically? What work exists in this area? What does the state-of-the-art in automated editing look like? 1. ^ Note that I'm not asking about mechanistically anomaly detection/dynamic backdoor detection; I'm aware that it's pretty easy to distinguish if a particular example is backdoored using baseline techniques like "fit a Gaussian density on activations and look at the log prob of the activations on each input" or "fit a linear probe on a handful of examples using logistic regression". 2. ^ I'm also aware of some of the work in the trojan detection space, including 2023 Trojan detection contest, where performance on extracting embedded triggers was little better than chance. 3. ^ That being said, it's plausible that dynamically detecting them given model internals is very easy. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Bayesian methods take the spotlight in this episode with Alex Andorra, co-founder of PyMC Labs, and Jon Krohn. Learn how Bayesian techniques handle tough problems, make the most of prior knowledge, and work wonders with limited data. Alex and Jon break down essentials like PyMC, PyStan, and NumPyro libraries, show how to boost model efficiency with PyTensor, and talk about using ArviZ for top-notch diagnostics and visualizations. Plus, get into advanced modeling with Gaussian Processes. This episode is brought to you by Crawlbase (https://crawlbase.com), the ultimate data crawling platform. Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information. In this episode you will learn: • Practical introduction to Bayesian statistics [04:54] • Definition and significance of epistemology [17:52] • Explanation of PyMC and Monte Carlo methods [27:57] • How to get started with Bayesian modeling and PyMC [34:26] • PyMC Labs and its consulting services [50:50] • ArviZ for post-modeling diagnostics and visualization [01:02:23] • Gaussian processes and their applications [01:09:02] Additional materials: www.superdatascience.com/793

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AXRP Episode 31 - Singular Learning Theory with Daniel Murfet, published by DanielFilan on May 7, 2024 on The AI Alignment Forum. What's going on with deep learning? What sorts of models get learned, and what do the learning dynamics? Singular learning theory is a theory of Bayesian statistics broad enough in scope to encompass deep neural networks that may help answer these questions. In this episode, I speak with Daniel Murfet about this research program and what it tells us. Topics we discuss: What is singular learning theory? Phase transitions Estimating the local learning coefficient Singular learning theory and generalization Singular learning theory vs other deep learning theory How singular learning theory hit AI alignment Payoffs of singular learning theory for AI alignment Does singular learning theory advance AI capabilities? Open problems in singular learning theory for AI alignment What is the singular fluctuation? How geometry relates to information Following Daniel Murfet's work In this transcript, to improve readability, first names are omitted from speaker tags. Filan: Hello, everybody. In this episode, I'll be speaking with Daniel Murfet, a researcher at the University of Melbourne studying singular learning theory. For links to what we're discussing, you can check the description of this episode and you can read the transcripts at axrp.net. All right, well, welcome to AXRP. Murfet: Yeah, thanks a lot. What is singular learning theory? Filan: Cool. So I guess we're going to be talking about singular learning theory a lot during this podcast. So, what is singular learning theory? Murfet: Singular learning theory is a subject in mathematics. You could think of it as a mathematical theory of Bayesian statistics that's sufficiently general with sufficiently weak hypotheses to actually say non-trivial things about neural networks, which has been a problem for some approaches that you might call classical statistical learning theory. This is a subject that's been developed by a Japanese mathematician, Sumio Watanabe, and his students and collaborators over the last 20 years. And we have been looking at it for three or four years now and trying to see what it can say about deep learning in the first instance and, more recently, alignment. Filan: Sure. So what's the difference between singular learning theory and classical statistical learning theory that makes it more relevant to deep learning? Murfet: The "singular" in singular learning theory refers to a certain property of the class of models. In statistical learning theory, you typically have several mathematical objects involved. One would be a space of parameters, and then for each parameter you have a probability distribution, the model, over some other space, and you have a true distribution, which you're attempting to model with that pair of parameters and models. And in regular statistical learning theory, you have some important hypotheses. Those hypotheses are, firstly, that the map from parameters to models is injective, and secondly (quite similarly, but a little bit distinct technically) is that if you vary the parameter infinitesimally, the probability distribution it parameterizes also changes. This is technically the non-degeneracy of the Fisher information metric. But together these two conditions basically say that changing the parameter changes the distribution changes the model. And so those two conditions together are in many of the major theorems that you'll see when you learn statistics, things like the Cramér-Rao bound, many other things; asymptotic normality, which describes the fact that as you take more samples, your model tends to concentrate in a way that looks like a Gaussian distribution around the most likely parameter. So these are sort of basic ingredients in understandi...

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!My Intuitive Bayes Online Courses1:1 Mentorship with meGPs are extremely powerful…. but hard to handle. One of the bottlenecks is learning the appropriate kernel. What if you could learn the structure of GP kernels automatically? Sounds really cool, but also a bit futuristic, doesn't it?Well, think again, because in this episode, Feras Saad will teach us how to do just that! Feras is an Assistant Professor in the Computer Science Department at Carnegie Mellon University. He received his PhD in Computer Science from MIT, and, most importantly for our conversation, he's the creator of AutoGP.jl, a Julia package for automatic Gaussian process modeling.Feras discusses the implementation of AutoGP, how it scales, what you can do with it, and how you can integrate its outputs in your models.Finally, Feras provides an overview of Sequential Monte Carlo and its usefulness in AutoGP, highlighting the ability of SMC to incorporate new data in a streaming fashion and explore multiple modes efficiently.Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !Thank you to my Patrons for making this episode possible!Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas, Luke Gorrie, Cory Kiser, Julio, Edvin Saveljev, Frederick Ayala, Jeffrey Powell and Gal Kampel.Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag ;)Takeaways:- AutoGP is a Julia package for automatic Gaussian process modeling that learns the

How long do you think Adam's crazy expensive Apple monitor will last? Dax tries to pass off all his front end tasks to Liz, Lambda fixed a streaming bug, Adam catches up on Cloudflare Dev week, Dax tries to make fun of Adam about Huberman, AI rizz and Gaussian splatting, and LEGO, Disney, and Universal all nailed it.Want to carry on the conversation? Join us in Discord.Home · tmux/tmux WikiVisual Studio Code ServerJosh's Online CoursesCopilot ProgrammerThe Pillars of the Earth (miniseries)Suno AIDax's AI BangersThe Cloudflare BlogAaron Francis on X: “@thdxr I'll wait, thanks.” / XCalifornia Workers' RightsAndrew Huberman Profile3D Gaussian SplattingTopics:(00:00) - The mystery project is... (00:42) - How long does a monitor last? (02:25) - Who wants to do front end dev really? (15:42) - Is AI just crypto bros take two? (34:44) - Would you chill at a relaxed job vs going to create more in a company? (38:15) - Lambda fixed the streaming bug (45:19) - Cloudflare Dev week (54:51) - How does Cloudflare pricing really work? (01:06:49) - React Miami teaser (01:08:33) - Was Twitter boring this week? (01:10:36) - Is working long hours in software dev bad? (01:18:50) - Andrew Huberman's methods of control (01:26:08) - AI has no rizz (01:30:47) - Gaussian splatting (01:36:58) - LEGO nailed it (01:41:31) - Disney nailed it (01:43:08) - Universal nailed it

Stephen Wolfram answers questions from his viewers about the history science and technology as part of an unscripted livestream series, also available on YouTube here: https://wolfr.am/youtube-sw-qa Questions include: Would an alien intelligence experiencing a different slice of the ruliad (a "ruster") close to ours likely experience black holes in a similar way? - ​Is rulial space bigger than branchial space? - Maybe it's a Gaussian distribution around a point in rulial space that makes human minds? - What do you think about NASA's recently released plans to build a Moon-based radio telescope? - ​How would the signal get back to Earth from the dark side of the Moon? - Why would so many nations be interested in the Moon? - Suppose we've just gotten lucky and developed our current level of technology during a period of unusual solar calm. How do we adapt if we expect solar storms to cause havoc with our electronics, say, every few decades? - Fiber optics have reduced our vulnerability from the days when landlines were all copper. Only the power grid remains. - What does the future look like for computational language? Will it be adopted on a larger scale? - How do you anticipate biotechnology shaping the future of biomaterials and tissue engineering? - How do you see the future of information consumption? Will it all be digital? Will physical books still be relevant? Will it even be reading, or simply data chips that are inserted into the brain? - Will we ever get to a point of other mammals evolving to the intelligence level of humans?

We introduce Chronos, a simple yet effective framework for pretrained probabilistic time series models. Chronos tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based language model architectures on these tokenized time series via the cross-entropy loss. We pretrained Chronos models based on the T5 family (ranging from 20M to 710M parameters) on a large collection of publicly available datasets, complemented by a synthetic dataset that we generated via Gaussian processes to improve generalization. In a comprehensive benchmark consisting of 42 datasets, and comprising both classical local models and deep learning methods, we show that Chronos models: (a) significantly outperform other methods on datasets that were part of the training corpus; and (b) have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. Our results demonstrate that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines. 2024: Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, Yuyang Wang https://arxiv.org/pdf/2403.07815v1.pdf

We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We disentangle the motion and appearance of a virtual human with explicit mesh geometry and implicit appearance modeling with Gaussian Splatting. The Gaussians are defined by barycentric coordinates and displacement on a triangle mesh as Phong surfaces. We extend lifted optimization to simultaneously optimize the parameters of the Gaussians while walking on the triangle mesh. SplattingAvatar is a hybrid representation of virtual humans where the mesh represents low-frequency motion and surface deformation, while the Gaussians take over the high-frequency geometry and detailed appearance. Unlike existing deformation methods that rely on an MLP-based linear blend skinning (LBS) field for motion, we control the rotation and translation of the Gaussians directly by mesh, which empowers its compatibility with various animation techniques, e.g., skeletal animation, blend shapes, and mesh editing. Trainable from monocular videos for both full-body and head avatars, SplattingAvatar shows state-of-the-art rendering quality across multiple datasets. 2024: Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, Zeyu Wang https://arxiv.org/pdf/2403.05087v1.pdf

Michael Rubloff is a leader in neural radiance fields (also known as. NeRF), a technology that turns 2D images into photorealistic three-dimensional scenes. He established the largest independent Discord server for radiance field creators radiancefields.com, to follow closely the rapid evolution of this technology.  He collaborated with leading companies like NVIDIA and Shutterstock and shared his expertise at prestigious events like SIGGRAPH 2023. Listen to this episode to learn: The differences between various types of Nerfs How to edit and clean NeRFs and Gaussian Splatting 2 ways to view your nerfS and splats in VR The latest exciting advancement in dynamic Nerfs *** CONNECT WITH MICHAEL

Show Notes: After graduation, Chris Ball spent his summer working in Cambridge before returning to Columbus, Ohio, where he began graduate school in physics at The Ohio State University. He worked with Professor Frank DeLuca, a world-renowned researcher in microwave spectroscopy. Chris' research focused on the microwave absorption of sulfur dioxide and its relationship to NASA's Microwave Limb Sounder instrument. Studying Interstellar Bands During his time at OSU, Chris collaborated with Professor Patrick Thaddeus from the Harvard Smithsonian Center for Astrophysics, who was looking to hire for a postdoc position. Chris moved back to Cambridge and worked in a lab in Somerville. He continued to do spectroscopy, but this time focused on long chains of carbon that don't occur naturally on Earth. These chains are unstable and are routinely observed in radio telescopes and optical telescopes. Chris and Thaddeus attempted to study the diffuse interstellar bands, which were optical features observed in telescope measurements that had never been explained over many years. They used laser spectroscopy to measure these bands and try to determine if any exotic carbon chains were responsible for them. Unfortunately, none of the exotic carbon chains were found, but the experience was rewarding. The Intersection of Science and Engineering After their first child was born, Chris and his family decided to move back to Columbus, Ohio, where he was offered a position at Battelle where his career began to focus on the intersection of science and engineering, specifically on developing sensor technologies and communications technologies. He worked on defense and security applications, such as detecting chemical and biological weapons, explosives, and narcotics. He also worked on pollution monitoring systems and handheld sensor technologies. Around  2015, Chris became disenchanted with Batel's strategic direction and started looking for alternatives. He found a similar job at Ohio State University's ElectroScience Laboratory, which focused on radar and communication systems. He left Batel, which coincided with his marriage falling apart. He moved offices, moved to an apartment, and started a consulting business. Working on the CubeSat Satellite at NASA Chris continued to focus on sensor and communication systems development. He was involved in a NASA program that built a CubeSat satellite, which was launched in 2018 from Wallops Island, Virginia, on a resupply mission to the International Space Station. The satellite went into orbit in July 2018. Chris discusses his exciting work in space, including developing sensors to detect toxic gasses and developing handheld infrared sensors for food and agricultural products. He is also working on an x-ray communication system, which uses X-rays as a carrier for wireless communications in space. In parallel with his work, he has a consulting company and has also discovered the joy of improv comedy, which he has been practicing for several months and now is part of an improv group called The Bunsen Burnouts. Interstellar Clouds and Molecules The discussion turns to interstellar clouds, and Chris explains what they are. There are many fundamental studies about the dynamics of molecules inside interstellar clouds and how they exist and might turn into stars in some regions. He also touches on the rotation of molecules, which is a fundamental discovery of quantum mechanics, and explains that, the transitions between rotational states in molecules are typically in the infrared part of the spectrum, while electronic transitions occur in the visible and ultraviolet part. However, molecules can also have bound atoms rotating, with quantized angular momentum and transitions corresponding to microwave frequencies. X ray Communications Research Chris talks about one of the projects he is proud of, X rays and the concept of wireless communications, which involve modulating a carrier frequency to transmit information. He explains that the idea of using X rays as a carrier and modulating them in some way came from discussions with NASA. NASA had a problem communicating with spacecraft during blackout periods when they enter the Earth's atmosphere. They developed a small X ray source that can be switched on and off quickly, allowing for about a gigahertz of bandwidth. This is better than current spaceborne optical systems, which can only transmit about a gigabyte of information per second. The team licensed this technology from NASA and applied its principles to X rays. X rays have significantly smaller wavelengths than optical systems, so they can propagate them much farther than optical systems. This could be advantageous for high data rate systems that can talk to Mars, as it would allow for interplanetary communication. Chris goes on to explain their process of research, feasibility of concepts, and demonstrating applicability.  Detecting Drugs and Toxic Chemicals  Chris has developed detectors for detecting drugs and toxic chemicals at extremely low concentrations and explains how these work. These detectors use microwave spectroscopy principles to measure gasses like formaldehyde in a low-pressure environment. The spectroscopic lines, which are sharp Gaussian distributions, are used to distinguish different gasses from each other and uniquely identify them. They achieve high sensitivity by making the lines taller and larger, and can be used in multipass configurations where the microwave beam passes through multiple times. This allows you to discriminate different gasses from each other and uniquely identify them like a fingerprint. Chris talks about a collaboration with his PhD advisor at Ohio State that led to the development of a mission adaptable chemical sensor funded by the Department of Defense. This sensor sucked in air and measured hundreds of different chemicals apart in a relatively short time. However, the technology is expensive due to the millimeter wave frequencies used in the microwave part of the system. The best available technologies cost around $60,000 for a transmitter and $50,000 for a receiver. This means that a $100,000 instrument is needed to buy the transmitter and receiver, along with all the electronics and pumps. Influential Harvard Professors and Courses Chris discusses their experiences at Harvard, focusing on the core curriculum courses and expository writing as the most valuable course he took. His advisor encouraged him to write a NASA fellowship proposal, which was well-written due to their expository writing skills. This experience has made him more valuable in various jobs, including red team reviews and proposal reviews for NASA and other funding agencies.  He also shares their experiences with math 22 and physics courses, and he mentions working at the high energy physics lab during their junior year and senior year, which was a valuable experience as they helped build a prototype muon detector system and perform measurements. Chris took advantage of opportunities to get involved with research while at Harvard, working at the high energy physics lab during the summer before his junior year and after his senior year. This experience allowed him to learn a lot about the science of expository writing and how to write effectively in academic settings. Timestamps:   01:03 Career path after Harvard graduation with a focus on physics research 06:04 Career progression from postdoc to industry to academia 10:39 Career changes, space research, and improv comedy 18:58 Interstellar clouds and molecular rotation 22:57 Wireless communication technologies and innovations 27:02 Using X-rays for high-speed communication in space 33:39 Developing infrared detectors for space applications with a focus on sensitivity and accuracy 39:16 Chemical sensing technology and its applications 45:36 Writing tips and research experiences at Harvard   Links: ElectroScience Laboratory: https://electroscience.osu.edu/ Page at OSU: https://electroscience.osu.edu/people/ball.51 Email address: ballc92@gmail.com

Get ready for 2024 and a brand new episode! We discuss non-parametric statistics in data analysis and AI modeling. Learn more about applications in user research methods, as well as the importance of key assumptions in statistics and data modeling that must not be overlooked, After you listen to the episode, be sure to check out the supplement material in Exploring non-parametric statistics.Welcome to 2024  (0:03)AI, privacy, and marketing in the tech industryOpenAI's GPT store launch. (The Verge)Google's changes to third-party cookies. (Gizmodo)Non-parametric statistics and its applications (6:49)A solution for modeling in environments where data knowledge is limited.Contrast non-parametric statistics with parametric statistics, plus their respective strengths and weaknesses.Assumptions in statistics and data modeling (9:48)The importance of understanding statistics in data science, particularly in modeling and machine learning. (Probability distributions, Wikipedia)Discussion about a common assumption of representing data with a normal distribution; oversimplifies complex real-world phenomena.The importance of understanding data assumptions when using statistical modelsStatistical distributions and their importance in data analysis (15:08)Discuss the importance of subject matter experts in evaluating data distributions, as assumptions about data shape can lead to missed power and incorrect modeling. Examples of different distributions used in various situations, such as Poisson for wait times and counts, and discrete distributions like uniform and Gaussian normal for continuous events.Consider the complexity of selecting the appropriate distribution for statistical analysis; understand the specific distribution and its properties.Non-parametric statistics and its applications in data analysis (19:31)Non-parametric statistics are more robust to outliers and can generalize across different datasets without requiring domain expertise or data massaging.Methods rely on rank ordering and have less statistical power compared to parametric methods, but are more flexible and can handle complex data sets better.Discussion about the usefulness and limitations, which require more data to detect meaningful changes compared to parametric tests.Non-parametric tests for comparing data sets (24:15)Non-parametric tests, including the K-S test and chi-square test, which can compare two sets of data without assuming a specific distribution.Can also be used for machine learning, classification, and regression tasks, even when the underlying datWhat did you think? Let us know.Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics: LinkedIn - Episode summaries, shares of cited articles, and more. YouTube - Was it something that we said? Good. Share your favorite quotes. Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

In today's episode of #TheRamblingsPod, I deep dive into a fascinating chat with Jacob Ennis. We're exploring Virtual Reality (VR) and Augmented Reality (AR) and how they're shaking up education, gaming, work, and marketing. We'll unpack different VR/AR gear, cool tech software, and Jacob's journey from teaching to a gig in software and sales at The Center of Simulation and Innovation. Plus, Jacob shares his personal VR adventures, and favorite games, and even touches on something called Gaussian splatting and what it's about. In our second VR episode in a short time, we lean into how VR is changing the game in various industries and its potential to revamp how we work and learn. If you're curious about tech or where VR and AR are headed, this chat is a must-listen. --- Send in a voice message: https://podcasters.spotify.com/pod/show/theramblings/message Support this podcast: https://podcasters.spotify.com/pod/show/theramblings/support

Learn about GMMs, a popular data science model! Want to support us? Become a premium subscriber to The Data Science Interview Prep Podcast: https://podcasters.spotify.com/pod/show/data-science-interview/subscribe

En este episodio, Santi Siri y Mauro Ordoñez hablarán sobre la revolución tecnológica detrás de herramienta para renderización de contenidos visuales: #GaussianSplatting. También hablarán sobre los avances del mundo de la tecnología. En esta edición: las regulaciones en #IA y #Cripto. Además, Gemini, lo nuevo de #Google: ¿cómo funciona esta inteligencia artificial? #SantiSiri #MauroOrdonez #Tecnología #InteligenciaArtificial #Diseño #Gemini #Google #FiloNews

What unites Newton's laws, the Penrose triangle, Gaussian elimination, the Pythagorean theorem, Halley's Comet, and the Fermi paradox? It should be obvious: they're all things named after their discoverers... or so it seems.   Story told by Garrett Tucker.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Neural uncertainty estimation for alignment, published by Charlie Steiner on December 5, 2023 on The AI Alignment Forum. Introduction Suppose you've built some AI model of human values. You input a situation, and it spits out a goodness rating. You might want to ask: "What are the error bars on this goodness rating?" In addition to it just being nice to know error bars, an uncertainty estimate can also be useful inside the AI: guiding active learning[1], correcting for the optimizer's curse[2], or doing out-of-distribution detection[3]. I recently got into the uncertainty estimation literature for neural networks (NNs) for a pet reason: I think it would be useful for alignment to quantify the domain of validity of an AI's latent features. If we point an AI at some concept in its world-model, optimizing for realizations of that concept can go wrong by pushing that concept outside its domain of validity. But just keep thoughts of alignment in your back pocket for now. This post is primarily a survey of the uncertainty estimation literature, interspersed with my own takes. The Bayesian neural network picture The Bayesian NN picture is the great granddaddy of basically every uncertainty estimation method for NNs, so it's appropriate to start here. The picture is simple. You start with a prior distribution over parameters. Your training data is evidence, and after training on it you get an updated distribution over parameters. Given an input, you calculate a distribution over outputs by propagating the input through the Bayesian neural network. This would all be very proper and irrelevant ("Sure, let me just update my 2trilliondimensional joint distribution over all the parameters of the model"), except for the fact that actually training NNs does kind of work this way. If you use a log likelihood loss and L2 regularization, the parameters that minimize loss will be at the peak of the distribution that a Bayesian NN would have, if your prior on the parameters was a Gaussian[4][5]. This is because of a bridge between the loss landscape and parameter uncertainty. Bayes's rule says P(parameters|dataset)=P(parameters)P(dataset|parameters)/P(dataset). Here P(parameters|dataset)is your posterior distribution you want to estimate, and P(parameters)P(dataset|parameters) is the exponential of the loss[6]. This lends itself to physics metaphors like "the distribution of parameters is a Boltzmann distribution sitting at the bottom of the loss basin." Empirically, calculating the uncertainty of a neural net by pretending it's adhering to the Bayesian NN picture works so well that one nice paper on ensemble methods[7] called it "ground truth." Of course to actually compute anything here you have to make approximations, and if you make the quick and dirty approximations (e.g. pretend you can find the shape of the loss basin from the Hessian) you get bad results[8], but people are doing clever things with Monte Carlo methods these days[9], and they find that better approximations to the Bayesian NN calculation get better results. But doing Monte Carlo traversal of the loss landscape is expensive. For a technique to apply at scale, it must impose only a small multiplier on cost to run the model, and if you want it to become ubiquitous the cost it imposes must be truly tiny. Ensembles A quite different approach to uncertainty is ensembles[10]. Just train a dozen-ish models, ask them for their recommendations, and estimate uncertainty from the spread. The dozen-times cost multiplier on everything is steep, but if you're querying the model a lot it's cheaper than Monte Carlo estimation of the loss landscape. Ensembling is theoretically straightforward. You don't need to pretend the model is trained to convergence, you don't need to train specifically for predictive loss, you don't even need...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] Remarks on the Convergence in Distribution of Random Neural Networks to Gaussian Processes in the Infinite Width Limit, published by Spencer Becker-Kahn on November 30, 2023 on The AI Alignment Forum. The linked note is something I "noticed" while going through different versions of this result in the literature. I think that this sort of mathematical work on neural networks is worthwhile and worth doing to a high standard but I have no reason to think that this particular work is of much consequence beyond filling in a gap in the literature. It's the kind of nonsense that someone who has done too much measure theory would think about. Abstract. We describe a direct proof of yet another version of the result that a sequence of fully-connected neural networks converges to a Gaussian process in the infinite-width limit. The convergence in distribution that we establish is the weak convergence of probability measures on the non-separable, non metrizable product space (Rd')Rd, i.e. the space of functions from Rd to Rd' with the topology whose convergent sequences correspond to pointwise convergence. The result itself is already implied by a stronger such theorem due to Boris Hanin, but the direct proof of our weaker result can afford to replace the more technical parts of Hanin's proof that are needed to establish tightness with a shorter and more abstract measure-theoretic argument. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Aras Pranckevičius reminisces about his formative years at Unity, detailing the transition from a Mac-only platform to a gaming revolution with the advent of the iPhone. He critiques Unity's package management and advocates for keeping core functionality within the main codebase. Aras favors text-based programming languages, given their clarity compared to visual nodes. He delves into 3D file formats, underscoring the resilience of simpler formats like OBJ and PLY, and prefers GLTF over the complex USD, which lacks a clear specification. Aras introduces Gaussian splatting as a more editable and understandable alternative to NeRF, outlining his work on optimizing splat data size. He acknowledges the transformative impact of AI on programming but expects it will take years for the dust to settle. Aras ends by recognizing the significant advancements of Blender in the open-source space and expresses cautious optimism for the future of 3D on the web with technologies like WebGPU…==== Have any comments or questions? Email the show Feedback@Buildingtheopenmetaverse.org   Want more information? Visit our website www.buildingtheopenmetaverse.org   And make sure you follow us on Linkedin for all of our show updates   https://www.linkedin.com/company/buildingtheopenmetaverse/ Building the Open Metaverse is a podcast hosted by Patrick Cozzi (Cesium) and Marc Petit (Epic Games) that invites a broad range of technical experts to share their insights on how the community is building the metaverse together. #BuildingTheOpenMetaversePodcast #MetaversePodcast #Metaverse

Chris Heinrich is the Co-Founder & CEO of Polycam, the leading 3D capture application that enables users to create high-quality 3D models from photos using any device. We talked about how he started Polycam, his vision for the company, AI and of course 3D Gaussian Splats. Episode 34 Timestamps: 00:00:00 Introduction 00:02:41 How was Polycam created? 00:12:00 Early days of Polycam 00:19:26 Vision for Polycam 00:22:23 3D as the new medium 00:24:09 3D Gaussian Splatting 00:31:28 Improving 3D Gaussians Splats on Polycam 00:38:37 Using AI in 3D scanning 00:45:40 ONE IMPORTANT Skill for Entrepreneurs Find out more about Chris and Polycam : https://poly.cam/ https://www.linkedin.com/in/chrispheinrich/ https://x.com/cpheinrich https://twitter.com/Polycam3D https://www.instagram.com/polycam3d/ Join our discord server where we connect and share assets: https://discord.gg/zwycgqezfD Bad Decisions Audio Podcast

Is The Bitcoin Bear Market Finally Over? This Indicator Shows It Is! We are joined by fellow bitcoiner, pleb and bitcoin rapper robbiep808x. we discuss his rabbit hole story and dive into how and why he makes the bitcoin content he does. On hopium we discuss which indicator is flashing green for the bulls and more! We dive into our update on what's going on with swan bitcoin, after last weeks reveal about ripple it seems swan is taking steps in the right direction with a big announcement or are they just placating bitcoiners? ✔ Special Guest: ► @robbiep808x ✔ Numbers: ► https://twitter.com/cooliganfields/status/1702066526780596440?s=52&t=CKH2brGypO5fEYTgQ-EFhQ ✔ Interview: ► https://www.youtube.com/watch?v=tvpOdef8CCc&list=PLz-0_5QIjz-_zB0lK9obSTDWnZnF_4dEZ ✔ REKT: ► https://x.com/aehw1/status/1702694312457232782?s=46&t=j3o89yN87GxWMG60Rt_wbg► https://x.com/citizenbitcoin/status/1702682992987025825?s=52&t=CKH2brGypO5fEYTgQ-EFhQ ✔ Bitcoin Hopium: ► https://twitter.com/thescalpingpro/status/1701654591765635450?s=52&t=CKH2brGypO5fEYTgQ-EFhQ ► https://learn.bybit.com/indicators/gaussian-channel-indicator/ ► https://twitter.com/brian_armstrong/status/1701806701853507886?s=52&t=CKH2brGypO5fEYTgQ-EFhQ ► https://twitter.com/jordanbpeterson/status/1702258366825877868?s=46&t=j3o89yN87GxWMG60Rt_wbg ► https://www.gemini.com/blog/weekly-market-update-friday-september-15-2023-franklin-templeton-bitcoin-etf ✔ BtcPayServer: ► https://github.com/btcpayserver/btcpayserver/releases ✔ Twitter Handles: @coinicarus @AEHW1 ✔ ShoutOuts: ► @BitkoYinowsky - our PlebUnderground Logo ► @WorldofRusty - Our YT backgrounds and segment transitions ► @luckyredfish - Outro Graphic ► @plebsTaverne - Intro video ► @robbieP808x - Outro music ✔ Links Mentioned: ► https://www.defendingbtc.com/ SUPPORT HODLONAUT ► https://timechainstats.com/ ✔ Check out our Sponsor, support Bitcoin ONLY Businesses: ► https://www.cryptocloaks.com/ For all of your 3D printed needs: Bitcoin Node Cases, Lightning Network Code Cases, BTC keychains, coasters, and a whole lot more , Printed By Bitcoiners for Bitcoiners! USE CODE PLEBUNDERGROUND FOR 5% OFF or use our affiliate link: https://www.cryptocloaks.com/aff/MobFfSr/ ► https://www.representltd.com/ It's your life...represent accordingly. Sweatpants, Tees, hoodies, a huge variety of Fresh Great Quality clothing by a Bitcoiner for Bitcoiners! USE CODE PLEBUNDERGROUND FOR 10% OFF ► https://cyphersafe.io/ We offer a full line of physical stainless steel and brass products to help you protect your bitcoin from various modes of failure. CypherSafe creates metal BIP39 / SLIP39 bitcoin seed word storage devices that backup your bitcoin wallet and protect them from physical disaster Timecodes: 0:00 - Intro 00:23 - Waltons Rap 03:46 - Robbies Rap 06:57 - Numbers 13:15 - Big Mac Vs. Quarter Pounder 15:36 - Fireside Chat 17:34 - Robbies RH story 19:57 - Why Make Bitcoin themed rap? 21:30 - Methods to come up with bitcoin rap material 29:43 - What's Next for Robbie P? 34:02 - REKT 35:03 - Swan/bitgo/fortress Update 43:00 - Fiat Concepts Being used in bitcoin 45:58 - TABconf - why you should have been there. 48:00 - Canadian Govt REKT! blaming grocers 50:14 - Hopium 50:48 - ETF Update 51:20 - Lightning development update 54:18 - WTF is the Gaussian channel? 57:50 - Time to scrap banks?

This is a recap of the top 10 posts on Hacker News on September 7th, 2023.This podcast was generated by wondercraft.ai(00:35): NSO group iPhone zero-click, zero-day exploit captured in the wildOriginal post: https://news.ycombinator.com/item?id=37425007&utm_source=wondercraft_ai(02:43): Kagi Small WebOriginal post: https://news.ycombinator.com/item?id=37420281&utm_source=wondercraft_ai(04:23): Mullvad on Tailscale: Privately browse the webOriginal post: https://news.ycombinator.com/item?id=37420053&utm_source=wondercraft_ai(06:25): North Korean campaign targeting security researchersOriginal post: https://news.ycombinator.com/item?id=37420831&utm_source=wondercraft_ai(08:21): Microsoft will assume liability for legal copyright risks of CopilotOriginal post: https://news.ycombinator.com/item?id=37420885&utm_source=wondercraft_ai(10:17): Tailscale Has Partnered with MullvadOriginal post: https://news.ycombinator.com/item?id=37420382&utm_source=wondercraft_ai(11:46): Gaussian splatting is pretty coolOriginal post: https://news.ycombinator.com/item?id=37415478&utm_source=wondercraft_ai(13:29): Exa Is DeprecatedOriginal post: https://news.ycombinator.com/item?id=37416430&utm_source=wondercraft_ai(15:15): Textual Web: TUIs for the WebOriginal post: https://news.ycombinator.com/item?id=37418424&utm_source=wondercraft_ai(17:07): Report says PR firm has been paying Rotten Tomatoes critics for positive reviewsOriginal post: https://news.ycombinator.com/item?id=37419427&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

In this episode, Bill and Andrew discuss the shades of variation: meeting requirements, accuracy, precision, and precision around variety. Is reducing variation to zero a good thing? Plus, Bill and Andrew share stories that offer practical ways to think about these concepts. TRANSCRIPT 0:00:02.4 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W Edwards Deming. Today, I'm continuing my discussion with Bill Bellows, who has spent 30 years helping people apply Dr. Deming's ideas to become aware of how their thinking is holding them back from their biggest opportunities. The topic for the day is The Paradigms of Variation. Bill, take it away.   0:00:28.1 Bill Bellows: Ooh.   0:00:28.1 AS: Exciting, exciting.   0:00:33.1 BB: Alright. So let me start off by saying this is episode number eight, and I wanna just make a couple comments about episode number seven, where we talked about "all straw" and "last straw" organizations also otherwise known as "me" or "we" organizations, or red pen or blue pen companies. And I just wanna burst a bubble and say neither one of them, neither organization exists, whether it's all or last or me or we. I view it as a... It's really a matter of which direction your organization is moving, it's a really simple model that I've seen get people to begin to appreciate what Deming's talking about, because I think that contrast is very much like a Deming organization versus a non-Deming organization. But instead of black-and-white thinking, there's really a continuum, and so I think... I just want to say at the beginning, it's really a question of which direction is your organization moving? Another thing I wanna throw out is... I don't think people know, I think absent an understanding of the System of Profound Knowledge, if you're in a last straw organization or a me organization, or a red pen company, I don't think you know that. I think if you become aware of Deming's work, you become aware of what could be. And I liken it to Dr. Deming saying, "How could they learn? How could they learn? The answer is frightening, how could they know?" So I think absent an understanding of The New Economics - Deming's work, I think it's hard to appreciate what you're missing.   0:02:11.4 BB: That you're being blamed for the grade, you're being blamed for the red beads. You're being blamed for the weather, if you're the weatherman. And the other thing that comes in mind there with that, "how could they know" is... There's a great video with Peter Senge, which he did a case with Dr. Deming, and there's a blog I wrote about it on the Deming Institute website if you just search for Peter Senge and my name. And you can find the blog as well as the link to the video. And in there Senge is talking about the present state of education systems and very much in this contrast of industrial and post-industrial, and he says, very much what it comes down to is, he says it's the water. He says, "We don't know what fish talk about, but you can be damn sure it's not the water." And likewise, I think people in a red pen company are not getting together. You and I talking about, "Andrew, this system sucks. I'm being blamed for the red beads," and I don't think we're the wiser. Now, if you turn me on to The New Economics. And we started listening to DemingNEXT and we became aware. But absent that, I think we're both frustrated, but we wouldn't know better. Alright, it's on the topic of variation.   0:03:30.8 AS: It's...   0:03:31.5 BB: Go ahead Andrew, you wanna say something?   0:03:32.4 AS: I was just gonna say that... That's where I think Dr. Deming's making the point of the difference between training and education. Education is the idea of bringing outside ideas into your mind, into your business, as opposed to training, which is trying to upgrade skills. And I had a little story of that when I was a head of research at an investment bank in Thailand. The whole job of a head of research is managing all these analysts who are writing research reports on company A, buy company A, sell company B for our institutional clients. And the job of a head of research is to try to manage that schedule. And you know that analysts are always gonna be interrupted and clients are gonna call, the market's gonna do this. So they're very rarely on time when they say that they're gonna finish something. So you're constantly scrambling for the morning meeting, because on the morning meeting you gotta have a story.   0:04:22.0 AS: And so that was just the job of a head of research. So I did that really well, managing them and, kind of, all that. And then I went to the number one investment bank, the number one broker in Thailand as the head of research. And I asked them, "So how often do you guys miss?" And they said, "Never." I said, "That's impossible." Because I've spent my whole career managing the flow of analysts. They said, "No, we never miss." When an analyst is gonna be on, they're always on. "And how do you do that?" "Well, we do a three-week-ahead schedule, everybody knows that you are held accountable for being that person on that day. And if you find out that you can't do it, you're gonna talk to someone else and rejigger it and say, hey, could you do Friday? And I'll do Monday the next week?" But they never miss. And I just thought, like the water, I never even knew I could go to a different level.   0:05:15.0 BB: Yeah.   0:05:16.8 AS: And then I went to a different level.   0:05:19.8 BB: Yeah, it's...it's the ability to step back. Alright, so on the topic of the paradigms of variation, I wanna throw out four words. Variety, variation, accuracy, and precision. A variety is, there's red beads and white beads, that's variety. There could be, eight different colors, that's variety, sizes of pants 32 waist, 32 length, that, to me that's variety. As opposed to variation is that a 32-inch waist or a gallon of gasoline, every time you go to get the gallon, you get a gallon of gasoline, it might not be exactly a gallon, that's variation. The reason I throw those out to begin with is that Dr. Deming is known in some circles back in the '80s, he was interviewed by somebody at the, I think at the BBC in England and an interview ends with him, with the interviewer saying, "Dr. Deming, if you could condense your philosophy down to a few words, what would it be?" And I thought, he's gonna say... He is just gonna reject that, that "I can't be condensed." No instead of that, he says, "Reduce variation." And I thought, "Oh, no... "   0:06:50.4 BB: So, and there are people alive and well today in the Deming community, who will quote that to me? "You know, Bill, Dr. Deming said, we gotta shrink variation to zero." And I said, "So, is he saying we all ought to be the same size? We ought to be the same skin color? Is he saying that he doesn't like diversity? What does that mean? And same religion?" I mean, you could look at religions as variety, and then you could say within each religion there's variation. So part of what I wanna get at today is what I think is confusion as to what he meant by shrinking variation to zero. So there's variety, variation. Accuracy is that when I get a gallon of gas, is it a gallon, or is it a couple ounces high, a couple ounces low? You go to the gas station, you'll see a sticker on the pump that says that it was calibrated to some standard, when you go to buy a pound of meat, are you getting a pound? Are you getting 15 ounces? And so the National Bureau of Standards is looking at accuracy, are all these things... Is every customer in the United States getting a gallon's worth of milk?   0:08:15.3 BB: Now, so that's accuracy. Precision is the idea that you get the same value each time, so I could go to the scale and it measures exactly a pound, exactly a pound, exactly a pound. But is that pound the same pound as the National Bureau of Standards pound? So I could be. 0:08:37.3 BB: Sorry about that. I could get the same value each time, and that's precision, but that's not to be confused with accuracy, so I just wanna throw those terms out. Relative to shrinking variation to zero, shrinking variation to zero which I, for the record, do not believe in. Dr. Deming would say anyone could accomplish anything if you don't count the cost. I think if you start to look at what is the benefit of having less variation versus the cost of that, then we can get to some point that makes sense economically as in The New Economics. But this idea of driving defects to zero, driving variation to zero without looking at cost.   0:09:24.1 BB: And you can look in The New Economics, we'll come back to this in a future episode. He definitely had in mind that you have to consider the cost, in fact, Dr. Deming would say, anyone could accomplish anything if you don't count the cost. But there's a... What I wanted to reference is a book by Peter Block called 'The Answer to How Is Yes' and what Block talks about is... Could be like, how...we get focused on, we're gonna go off and reduce variation, we're gonna go off and drive variation to zero or non-value added to zero. What Block talks about that I really appreciate, that I think Dr. Deming appreciate is, why? Why did... Let's step back a minute, and so part of what I wanna get at tonight in this paradigms of variation is the 'Why' piece. Okay. So the first example I wanna look at a variation is throwing darts okay? And hopefully that makes sense, you're throwing darts in a dart board and imagine meeting requirements is being on the dart board, so imagine it could be a foot in diameter.   0:10:29.4 BB: And in terms of meeting requirements, you wanna be on the dart board. So I throw it three times, and if you get three that are really close together, they may not be on the bullseye, and that says, I'm very precise, but if the three are not on the bullseye, then that's not very accurate. So again, throwing three and getting really, really consistent is one thing, but then how do I move that to the bullseye? So that's an idea that I could first focus on precision, and then often I find that if I could just slightly adjust my release or my arm, then maybe I could then move it over, so I wanna look at that.   0:11:14.7 AS: And moving over is accuracy or?   0:11:17.5 BB: Moving it over is accuracy.   0:11:19.2 AS: Okay.   0:11:19.5 BB: I mean, so the first thing could be, I'm just looking for three...   0:11:22.5 AS: Get on the board.   0:11:23.6 BB: I wanna be consistent.   0:11:25.9 AS: Yep.   0:11:26.6 BB: And then make the adjustment, 'cause I find often it's easier to make the adjustment, I think it's a lot of work to get consistency. So I just want to separate those out as two different strategies.   0:11:39.2 AS: Yeah, just go to the bar and start throwing darts and you'll see it's a lot of work. Yep that helps, that helps, that helps us to understand it.   0:11:45.9 BB: Alright, so next. Next I wanna talk about what I refer to as the Two Distributions Exercise, and so here's the context. Imagine that you are in the procurement organization, and your job is to make a decision as to who to buy a given product from. So your company goes out and gets quotes from four different suppliers, and they provide you with the information. And for simplicity, let's say what you're buying are these metal tubes and... Short metal tubes perhaps used in plumbing, they're a given length, a given diameter. And imagine these four suppliers come back to you. And again, you're the procurement person, "Who are we gonna buy from?" They come back and they say, they quote you the price, and they quote you exactly the same price. All four of them quote you exactly a dollar each, $10 each. It's like, "Holy cow, they're the same price."   0:12:46.2 BB: Imagine also, they quote the same delivery schedule. So you've got a plumbing supply, you need lots of these, they all tell you they're gonna give you the volume that you need. So I think, "Gosh, volume-wise that's the same, cost-wise, it's the same." Now imagine what they tell you is relative to meeting the diameter, let's say it's the outer diameter is really critical to how these things fit together. And they quote you and say, "All the outer diameters will meet requirements." They're gonna take care of the scrap and they're gonna get rid of the red beads. All the tubes they will send will meet requirements, guaranteed. And you're thinking, "I want that same schedule, same costs, same quality," now what? Well, now imagine they send you the distributions from the control charts and they tell you that these distributions, you're thinking, "Holy cow, these suppliers are using Cisco process control." And they provide you with the histograms, and they say, "These distributions will never change, shape or location." Holy cow.   0:13:49.6 BB: And then added onto that is that you're gonna use them as is. So you're not gonna take them and modify them, you're just gonna bring them into the inventory and send them off to the plumbers to use. So you're saying, "Okay, the process is in control, the level amount of variation, location is predictable, stable, forever. How could I go wrong?" And then the last thing they tell you is, procurement that, "Here's the lower requirement, here's the upper requirement, and here's the ideal value." And so then you end up with two distributions. If I was confusing, I meant to say two, not four [chuckle]   0:14:24.1 BB: Alright, so imagine you've got two suppliers and the one distribution goes from the lower spec to the upper spec. And let's say it's a normal Gaussian distribution and it starts at the low end, goes up, high in the center, then off to the other, and that's supplier A and then imagine the other supplier uses 10% of the variation, but is towards the upper spec so it's far more uniform, but it's off of the ideal value. And so I've been using those two distributions with people as an ideal scenario saying, "You're never gonna have all that information, let alone that's all the same." And very deliberately, what I want people to do is say, if it's the same price, same schedule, zero defects, guaranteed, distributions never change and you're looking at the lower spec, the upper spec, and you're saying, "Okay, so one distribution, it has more variation, but the average is right in the middle, which is the ideal value. And the other one is shifted towards the high end of the tolerance, but incredibly uniform," who do you choose?   0:15:38.3 AS: So it's a tall curve?   0:15:39.4 BB: It's a very tall curve, let's say it uses 10% of the variation, 10% of the tolerance and so I've been using that going on 30 years, and I'll have 30 people in the room and I'll ask them to write down on a three by five card, "Who would you buy from?" And I'll say, "Here are the choices you can buy from the, the one that's the widest, we'll call that supplier A and supplier B is the narrow one to the right, or You could say it doesn't matter." And what I find is incredibly consistent inside and outside of Rocketdyne and literally around the world is the majority of people will take the narrow distribution, to the right will call that supplier B, what I ask them, "Why do you like supplier B?" To a person they will say, "It's more consistent, there's less variation." And I say, "Less variation from what?" "Well, less variation from each other." Well Andrew, that's precision.   0:16:40.9 BB: And then I ask the others, and my find is three quarters of the room will take that distribution, the one which is precise. And for the ones who are focusing on the wider distribution, where the average is on target, I say, "Why do you like that one?" And they say, "Because it has less variation from the ideal value." Alright? And so I wanna throw that out is part of the confusion I find inside and outside of the Deming community, in the world of Six Sigma quality distribution B, using a smaller percent of the tolerance, is, has the higher process capability index. 'Cause what that index is doing is comparing the amount of variation, the width of the variation to the overall tolerance. And the idea that you're using a smaller portion is valued. And I said, "Okay, well that's not quite the same as what Dr. Taguchi is talking about. What Dr. Taguchi is talking about," and this one we'll get into in a later episode, "is the closer you are to the ideal value, what you're doing is affecting how this is used in a greater system, so if I'm at home cutting a piece of wood to a given length and I want it to be closer and closer to the ideal value, then what I'm gaining is making it easier to put that piece of wood, or whatever I'm making, together.   0:18:00.5 BB: And I find that people who preferred distribution B are really confused 'cause in a big way what they're saying is, "I don't care about where I am within, all I care about is using a small portion of the tolerance." And then when I press on that more and more, they say, "Well, I want fewer and fewer defects." I said, "Well, zero defects is guaranteed, so if you really believe in zero defects as the goal, then you should have said it doesn't matter." And so the reason I wanna talk about the paradigms of variation is that one: variation is one of the elements of the System of Profound Knowledge and it's not just the variation in the number of red beads, right?   0:18:58.0 BB: And not to dismiss that the variation of the red beads is caused by the system. But what I've tried to bring to these episodes interviews with you is what I learned from Dr. Taguchi is the variation in the white beads and what is the impact of the variation on the white beads. And if we ignore that, then what we're saying is, "As long as you meet print, that's all that matters at the end of the day." And I'd say if that's where you're going then, then you could do the same thing with Lean or Six Sigma operational excellence. What differentiates Dr. Deming's work, I believe in terms of his appreciation of variation as an element of Profound Knowledge, is what he learned from Dr. Taguchi. That the closer we are to the ideal value, that affects how the system, which is another element of Profound Knowledge, comes together.   0:19:53.8 BB: All right, so going back to those two examples, what I started to do, one is I was detecting that less variation, less, I was detecting within Rocketdyne and elsewhere that there was a far greater regard for less variation, less variation from each other than being on target. And I was just wanting to one; find out why does it matter if all you have to do is meet spec? Why does it matter? So relative to the paradigms of variation, and this was back into the mid '90s when I was working with some people in manufacturing and was greatly confused over this, and the confusion was, "Is it enough to meet print, Bill? You're not sure? And then we've got these capability indices. We want to use a small portion of the tolerance and then we've got this, "Bill you're telling we wanna be on target, help me understand that."   0:20:49.7 BB: Was what these guys were asking for. And the paradigms of variation that I come up with. And I described it, I said, "Well, let's look at it this way." I said, "There's this thing called... Let's call it paradigm A, and Paradigm A is meet print." All that matters at the end of the day, we wanna meet spec. So.   0:21:06.4 AS: When you say meet print, print is a kind of a word that maybe not everybody understands what that means.   0:21:12.7 BB: Thank you.   0:21:12.9 AS: What, that means spec?   0:21:13.6 BB: Meet the requirements.   0:21:14.6 AS: Meet the requirements.   0:21:15.6 BB: Meet the requirements. And so we want the meeting to start anywhere between here and here. And as long as we're in between... So "meeting requirements" such that everything is good, is paradigm A. And so if you went back to those... Looking at those two distributions, if you said it didn't matter which one to take, that would be the paradigm A answer. And that's rarely the case. And so what I was poking at with people is, "You tell me you're striving for zero defects, and then when I give you that information that there's zero defects, why does that not trigger you to say it doesn't matter?" Because there's something else going on. So then the idea that we want incredible uniformity, precision, that's what I refer to as paradigm B.   0:22:07.3 BB: And as I mentioned earlier, that is the dominant choice. We want narrow distributions. We want what people refer to as "piece to piece consistency" to be differentiated by the second most popular answer is being on the ideal value what Dr. Taguchi would call the target, which is what I refer to as paradigm C. So in explaining these three paradigms to these manufacturing folks, I said each of them has a goal. So the goal of paradigm A is to meet requirements, but they not only have a goal, they also have an approach. And their approach typically tends to be, "If you're slightly out measure again, if you're slightly in you're good. Can we change the requirements?" And so I thought as... The paradigm A solutions are all about playing with those lines, moving them in, moving them out.   0:23:01.1 BB: Paradigm B, which has a lot to do with, I find within Six Sigma quality, is we wanna have a given fraction of a percent of the tolerance. And these indices, the Cpk Cpk, Cp Cpk, and others, there'll be goals of, "It needs to be 1.33 or 2.0, or 1.67, and we wanna strive for Six Sigma quality." Well, the question I ask those people is, "How much money are we gonna spend to achieve Six Sigma quality? And is there a corresponding benefit?" And I don't get an answer. But so the paradigm B approach would be to take the distribution, and try to make it narrower, but narrow to the point that we're only using, 10% of the tolerance. And again, what bothers me about that is that it's not addressing what Taguchi's talking about, which is what we're doing at home.   0:24:04.8 BB: Whether it's baking something, we want the temperature to be close to 350 or, whatever it is we're doing. We're, looking for accuracy in how we're pulling something together, is we're looking for an ideal value. And there, what we're trying to do is, as I mentioned earlier, we're striving for, "Can we get precision and then can we make the adjustment to achieve accuracy?" And instead of just saying, "We wanna achieve some given value." To me, what I tell clients I work with and students in my classes is, "What is it gonna cost to achieve precision, to then focus on accuracy? How much money are we gonna spend on that? And what is the benefit?" And the benefit will be improvements downstream, which is looking at things as a system. And what we'll talk about in a future session, looking more at this is examples of things I've been involved with, that address this idea of not reducing variation to zero, but to me it's about managing variation and having the appropriate amount of variation, knowing that it could never be zero.   0:25:18.1 BB: But, does it...am I in a situation where meeting requirements is all I need to be. In the world of baseball there's a strike zone. You've got a batter coming up who can't hit the ball no matter what, and you say, "Well, it doesn't matter where it is. Just get it into the strike zone." The next batter comes up. And that batter is very determined to make... And you're trying to get the ball around the bat. Now it depends on where you are within the strike zone.   0:25:46.6 BB: Alright. So the other paradigm I wanna get into, and then we'll call it over, is, paradigm D. So there's A, is meet requirements, that's all that matters. B is, I'm looking for precision. C is, I'm looking for precision followed by accuracy. Paradigm D when I explained this to Dr. Taguchi in the late 1990s, and he said, I need to differentiate having one ideal value so I can be working in a place where all the tubes we make are one inch in outer diameter. And, so there's one ideal value, well, maybe what the company is doing is getting into variety and having different outer diameters. One inch, half inch, three-quarters of an inch. And in each case they're looking for accuracy, but accuracy around different values. And that's what Dr. Taguchi would refer to as... Well, he and I agreed to call it paradigm D, which is precision around an ideal value. But depending on your product line, you may have ideal values for different customers. And that's called variety. And so paradigm D is about precision coupled around varieties. So I just wanted to throw that out as well in our session.   0:27:16.7 AS: And the risk that you're highlighting is that somebody who's skilled in Six Sigma or some other tools will be patting themselves on the back, that they've got a very narrow distribution in that... And it's inside of spec and therefore they've done their job.   0:27:39.4 BB: Yes. Well...   0:27:40.1 AS: And what you're highlighting is that there is, there is an additional cost to the business or additional benefit if that narrow distribution could be moved to the target value?   0:27:58.2 BB: Well, here's what I've seen. I've seen organizations go from a really wide distribution where, in the assembly process, they need all those different sizes to put the puzzle together. And then somebody comes in and shrinks the variation to a fraction of that, not taking into account how they're used, and instead of going around and having all the different sizes to put the puzzle together, they can no longer do that. So what I'd say, I've seen plenty of examples where a given amount of variation that people are used to, that they're accommodating could be quite well until somebody comes along and gets rid of those other options.   0:28:48.2 BB: So I've seen variation reduction gone sour, a few times leading to some near catastrophic failures of a rocket engine because we're just looking at something in isolation. And, so I went to a very senior executive in that timeframe and I said... 'cause there's this big push in the company and we gotta reduce variation, "We gotta reduce variation." And I went to him and I said, "If we have a choice between shrinking the variation and doing nothing, I'd say do nothing." And he is like, "Well, what do you mean?" And I went through and explained this scenario with him and he said, "Oh, I've never seen anything like that." And I thought to myself, "You must have worked for companies that make the tubes, but don't use the tubes."   [laughter]   0:29:33.4 BB: I said. And so, this is why when I hear people talk about reducing variability, reducing cost, trying to make improvements, and again, we'll look at this in a whole nother episode, is my concern is are they thinking about that part in isolation? Are they thinking about how that fits into a greater system? So whether it's reducing the variation in the outer diameter, whether it's reducing the cost, if they're focusing on that as a KPI, and not looking at how that KPI fits into a greater system, I'd say I'd be nervous about that.   0:30:17.4 AS: One of the interesting examples I remember from when I was young and in maybe business school or whatever, was when Toyota came out with Lexus and they talked about how they spent a huge amount of time reducing variation in every part so that you had a much smoother and more quiet ride, and the reliability was better and better. And they talked about the pursuit of perfection was the tagline that they did. But it made sense to me that, many people would be... Many companies are satisfied with a certain amount of variation.   0:30:54.8 AS: When if they could get it more narrow around the desired outcome, then the knock on effects, particularly for a new company, maybe for an old company, and the knock on effects basically lead people to go, "Go back we want more variation," because you're screwing up everything downstream. But if you're building an operation where you can get more and more narrow distribution around the target output, the target desired output, then you're bringing benefit all the way down the line for the business. What have I got right and what have I got wrong out of that?   0:31:33.2 BB: Well, that's fantastic. And a couple things come to mind. I really appreciate that question. Andrew, if you were to do a Google search for Dr. Taguchi and Toyota, because this idea of being on target associated with what he referred to as the quality loss function, which again, will be a focus of another episode, I'd rather one, look at it as an integration loss function, just to reinforce the idea that being close to the ideal value is about improving integration. And that's it.   0:32:12.7 AS: When you say integration, what do you mean?   0:32:15.5 BB: Who's gonna use that tube? What are they gonna do with it?   0:32:18.1 AS: Okay. So downstream, integrating the process with the downstream.   0:32:20.5 BB: And so if I'm not looking at how the doctor fits into the system, how the tube fits into the system. So what I find is in the Taguchi community, people will say, Dr. Taguchi worked with Toyota back in the '50s and '60s. Dr. Taguchi and Deming met for the first time in the mid '50s in India. Dr. Taguchi was honored with the Deming prize in literature in 1960, and they would've met then. Don Wheeler in his books on Statistical Process Control, and inside the cover it will say, "In September 1960, a new definition of quality being on target with minimum variation." So there's all that. So what I've tried numerous times over the last 30 years is searching for documentation of Taguchi's influence on Toyota. I found nothing.   0:33:10.7 BB: And, so here I'm flying back from Japan, having gone there while Rocketdyne was owned by Boeing to explain these concepts to people at the Mitsubishi Heavy Industries, which is the largest aerospace company in Japan.   0:33:25.1 BB: There was a big partnership going on between Boeing, the division I worked for at Rocketdyne was part of Boeing. And, Boeing's, at that time, largest supplier in the world was MHI. So I was on a study team to go over there to... And I explained these ideas to them. They knew nothing about this. They were focusing on uniform... They were focusing on... Their quality system was precision, not accuracy.   0:33:47.6 BB: And I was explaining what we were doing with that. Well, flying home, I was sitting in business class, sitting next to me is a young engineer, flying out of Tokyo. He is Japanese. And now we started talking. Turns out he is a graduate of Cal Poly San Luis Obispo in California working for Toyota at the NUMMI plant. And I explained to him red pen and blue pen companies, he loved it. I explained to him the paradigms of variation. And he says, "Bill," he says, "I'm coming back from working with supplier to get them to focus on the ideal value." He says, "That is the thinking we use."   [laughter]   0:34:29.2 BB: He says they wanna change the tolerance. And I'm telling him, "No, you've got to hold that target value." So you can search the Internet, you won't find this. And so there's two data points I want to get before we close. So one is that the majority of the flight coming home was me explaining this stuff to him, and then afterwards maintaining a relationship with him and his boss and looking to see if I could learn more.   0:34:56.0 BB: But he was... For him to say, "That's exactly what we do." Well then I spent several years poking Dr. Taguchi about his loss function concepts and all, and he said, "No company in the United States uses the loss function." And I said, "Really?" He says, "No." He said, "The leading users in Japan are Toyota and Nippon Denso," now known as Denso, a major supplier to Toyota.   0:35:21.1 BB: And I said, "What do they do with it?" He says, well, he says, "Bill, they have a database of loss functions for how different things come together." He says, "They have a database for the impact of variation." And I said, "Really?" I said, "How do they use it?" He said, "They use it to guide their investments." That's what you're talking about, Andrew. But you won't find that on the Internet. I've not found that in any literature.   0:35:51.1 BB: So, those are two things that I hold there. I believe Toyota is using this somewhere deep in the organization as evidenced by this young guy. And my interest is to expand that appreciation within our community in The Deming Institute, that it is not about uniformity. It is not about precision. And, that improving precision could make things worse. [chuckle] If you're not focused on accuracy, then the question becomes, "Is every situation worth accuracy?" And the answer is, "No. You've got to look downstream."   0:36:29.6 AS: Okay. Now it's time for me to ask the question that was asked of Dr. Deming.   0:36:34.8 BB: Okay.   0:36:35.9 AS: Explain it in one short sentence. What do you think the key takeaway is from this excellent discussion?   0:36:44.8 BB: I think what's really important is the need to manage variation, which is the same thing as Akoff would say, the difference between managing actions and managing interactions. The idea is that how I accomplish my task depends upon how you're using it. And so for me to blindly meet a requirement from you not knowing how you use it, well, whether that's you asking me to clean the table and I don't know anything about the table, you saying, "I need you to meet these requirements."   0:37:21.2 BB: You saying, "I need this by tomorrow." And I say, "What do you mean by tomorrow, Andrew? Tomorrow at eight o'clock, tomorrow at nine o'clock?" And so I think what Deming's talking about is if I just blindly take a set of requirements and meet them in a way that I interpret without asking you for clarification, is not teamwork.   0:37:41.7 AS: Great.   0:37:44.1 BB: So I need to know how you're using this.   0:37:47.1 AS: And, that's a great lesson. And I think what it's telling us is the idea of communicating and cooperating and getting to the next level has to do with really understanding what the next process is doing with it, and how what you're delivering could be improved so that the improvement is measured by a benefit to the next and the next and the next profit process. Not as a loss to the next one, which is what you explained about if variation got reduced, all of a sudden people weren't built for handling that.   0:38:23.2 BB: Well, and let me throw one other thing out along those lines. And as a colleague of mine in Amsterdam says to people in the Lean community says, "How does Lean...how does implementation of Lean explain why we love Toyota products? How does it explain the reliability of the products? We buy nothing but Toyotas." Now, we've had bad luck with Toyotas, which people I met in business school classes told me, "You never buy anyone's first model even Toyota."   0:39:03.8 BB: So we will only buy Toyotas, but we'll never buy the first model year. And I'm buying it because I want it to start every single time. I don't want a car where I've gotta replace the water pump. And so for our listeners, if you wanna have customers revere your products for the reason, I think, many people revere Toyota products, I think what we're talking about tonight is a significant part of what makes those parts come together and those cars last so long.   0:39:41.3 AS: Bingo. Bill, on behalf of everyone at The Deming Institute, I want to thank you again for the discussion. And for listeners, remember to go to deming.org to continue your journey. This is your host, Andrew Stotz, and I'm gonna leave you with one of my favorite quotes from Dr. Deming, which is, "people are entitled to joy in work."

Episode 83.2: Misinformation, Blood Alcohol Content, Self-Driving Cars, Celebrating Life, European Nudity, and the Gaussian Curve

Episode 5. This episode about synthetic data is very real. The fundamentalists uncover the pros and cons of synthetic data; as well as reliable use cases and the best techniques for safe and effective use in AI. When even SAG-AFTRA and OpenAI make synthetic data a household word, you know this is an episode you can't miss.Show notesWhat is synthetic data? 0:03Definition is not a succinct one-liner, which is one of the key issues with assessing synthetic data generation.Using general information scraped from the web for ML is backfiring.Synthetic data generation and data recycling. 3:48OpenAI is running against the problem that they don't have enough data and the scale at which they're trying to operate.The poisoning effect that happens when trying to take your own data.Synthetic data generation is not a panacea. It is not an exact science. It's more of an art than a science.The pros and cons of using synthetic data. 6:46The pros and cons of using synthetic data to train AI models, and how it differs from traditional medical data.The importance of diversity in the training of AI models.Synthetic data is a nuanced field, taking away the complexity of building data that is representative of a solution.Differences between randomized and synthetic data. 9:52Differential privacy is a lot more difficult to execute than a lot of people are talking about.Anonymization is a huge piece of the application for the fairness bias, especially with larger deployments.The hardest part is capturing complex interrelationships. (i.e. Fukushima reactor testing wasn't high enough)The pros and cons of ChatGPT. 13:54Invalid use cases for synthetic data in more depth,Examples where humans cannot anonymize effectivelyCreating new data for where the company is right now before diving into the use cases; i.e. differential privacy.Mentally meaningful use cases for synthetic data. 16:38Meaningful use cases for synthetic data, using the power of synthetic data correctly to generate outcomes that are important to you.Pros and cons of using synthetic data in controlled environments.The fallacy of "fairness through awareness". 18:39Synthetic data is helpful for stress testing systems, edge case scenario thought experiments, simulation, stress testing system design, and scenario-based methodologies.The recent push to use synthetic data.Data augmentation and digital twin work. 21:26 Synthetic data as the only data is where the difficulties arise.Data augmentation is a better use case for synthetic data.Examples of digital twin methodology to createWhat did you think? Let us know.Good AI Needs Great Governance Define, manage, and automate your AI model governance lifecycle from policy to proof.Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics: LinkedIn - Episode summaries, shares of cited articles, and more. YouTube - Was it something that we said? Good. Share your favorite quotes. Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thoughts on Loss Landscapes and why Deep Learning works, published by beren on July 25, 2023 on LessWrong. Crossposted from my personal blog. Epistemic status: Pretty uncertain. I don't have an expert level understanding of current views in the science of deep learning about why optimization works but just read papers as an amateur. Some of the arguments I present here might be already either known or disproven. If so please let me know! There are essentially two fundamental questions in the science of deep learning: 1.) Why are models trainable? And 2.) Why do models generalize? The answer to both of these questions relates to the nature and basic geometry of the loss landscape which is ultimately determined by the computational architecture of the model. Here I present my personal and fairly idiosyncratic and speculative answers to these questions and present what I think are fairly novel answers for both of these questions. Let's get started. First let's think about the first question: Why are deep learning models trainable at all? A-priori, they shouldn't be. Deep neural networks are fundamentally solving incredibly high dimensional problems with extremely non-convex loss-landscape. The entirety of optimization theory should tell you that, in general, this isn't possible. You should get stuck in exponentially many local minima before getting close to any decent optimum, and also you should probably just explode because you are using SGD, the stupidest possible optimizer. So why does it, in fact, work? The fundamental reason is that we are very deliberately not in the general case. Much of the implicit work of deep learning is fiddling with architectures, hyperparameters, initializations, and so forth to ensure that, in the specific case, the loss landscape is benign. To even get SGD working in the first place, there are two fundamental problems which neural networks solve: First, vanilla SGD assumes unit variance. This can be shown straightforwardly from a simple Bayesian interpretation of SGD and is also why second order optimizers like natural gradients are useful. In effect, SGD fails whenever the variance and curvature of the loss landscape is either too high (SGD will explode) or too small (SGD will grind to a halt and take forever to traverse the landscape). This problem has many names such as 'vanishing/exploding gradients', 'poor conditioning', 'poor initialization', and so on. Many of the tricks to get neural networks to work address this initializing SGD in, and forcing it to stay in the regime of unit variance. Examples of this include: a.) Careful scaling of initializations by width to ensure that the post-activation distribution is as close to a unit variance Gaussian as possible. b.) Careful scaling of initialization by depth to ensure that the network stays close to an identity function at initialization and variance is preserved (around 1). This is necessary to prevent either exploding activations with depth or vanishing activations (rank collapse). c.) Explicit normalization at every layer (layer norm, batch norm) to force activations to a 0-mean unit variance gaussian, which then shapes the gradient distribution. d.) Gradient clipping to ensure that large updates do not happen which temporarily cuts off the effects of bad conditioning. e.) Weight decay which keeps weights close to a 0-mean unit variance Gaussian and hence SGD in the expected range. f.) Careful architecture design. We design neural networks to be as linear as possible while still having enough nonlinearities to represent useful functions. Innovations like residual connections allow for greater depth without exploding or vanishing as easily. g.) The Adam optimizer performs a highly approximate (but cheap) rescaling of the parameters with their empirical variance, thus approximatin...

IntroductionShekeib and Shohaib join the podcast as guests to talk about their experience with creating hot sauces using AI optimization.They created a special hot sauce, named "Counting Sauce," specifically for the podcast hosts.The Making of Counting SauceThis is a unique hot sauce that includes pineapple and mango flavors.The sauce was created as a token of appreciation for being featured on the podcast.Journey through Different Versions of the SauceThe hosts have tried versions 19, 20, 21, and they just received version 25.There will be a blind taste test to determine if they can tell the difference between the different iterations and compare them with other sauces to tell which is AI-created.Optimization ProcessThe process involves optimizing the amount of each ingredient.They use a Gaussian process regression model and an acquisition function called Expected Improvement for the optimization.Choice of IngredientsThe base hot sauce has five main ingredients: vinegar, pepper, jalapeno, and lime.After 25 iterations, the differences in taste become so minute it becomes hard to tell the difference.Subjective Taste TestingShekeib talks about how his taste tolerance changes after tasting hot sauces all day.They involved family and friends in the tasting process and asked for ratings on a scale of one to ten.The Learning Curve of the AIEarly on, the AI would try extreme variations like too much or too little salt.It learned quickly from feedback and adjusted accordingly.Strength of Bayesian OptimizationThe AI can learn mathematically from feedback and apply the learnings, making the optimization process quicker and more efficient.It was also able to tweak multiple ingredients simultaneously, unlike a human who might focus on one ingredient at a time.No Prior Experience in Hot Sauce MakingBoth brothers had no prior experience or generational knowledge in hot sauce making.The AI managed to create a decent hot sauce in just five iterations.Power of Bayesian Optimization with Human ExpertiseThe brothers emphasize the importance of having a human expert in the loop of Bayesian optimization.The AI simulates the intuition and experience of a human expert, but having a real human guide the process further enhances the results.Application Beyond Hot SauceThey discuss the potential of their Bayesian optimization process in other areas such as drug discovery.The process can be guided by human experts in the respective fields for even better results.

This episode is jam-packed with fencing, organ harvesting, the black market, and burking (with some fun Victorian medicine tidbits for our true crime). Episode information: Crossing Jordan S4 Ep9 Necessary Risks Follow us on Instagram: @insidethemorguepod Email us show suggestions: insidethemorguepod@gmail.com If you enjoy this podcast, support us! Music used from Pixabay.com: Crime Trap by Muzaproduction & Detective by SergeQuadrado Sources: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6923544/ https://forensicreader.com/fingerprints-on-clothes/ https://www.historic-uk.com/HistoryUK/HistoryofScotland/Burke-Hare-infamous-murderers-graverobbers/ https://tncourts.gov/sites/default/files/docs/fingerprints.pdf https://en.wikipedia.org/wiki/Gaussian_function#:~:text=In%20mathematics%2C%20a%20Gaussian%20function,characteristic%20symmetric%20%22bell%20curve%22%20shape https://www.suecoletta.com/fingerprints-points-type-and-classification/ https://www.who.int/news-room/fact-sheets/detail/schistosomiasis --- Support this podcast: https://podcasters.spotify.com/pod/show/insidethemorgue/support

The idea of grouping similar types of data together is the main idea behind clustering. Clustering supports the goals of Unsupervised Learning which is finding patterns in data without requiring labeled datasets. In this episode of the AI Today podcast hosts Kathleen Walch and Ron Schmelzer define the terms Clustering, Cluster Analysis, K-Means, and Gaussian Mixture Model, and explain how they relate to AI and why it's important to know about them. Continue reading AI Today Podcast: AI Glossary Series- Clustering, Cluster Analysis, K-Means, Gaussian Mixture Model at AI & Data Today.