Podcasts about frequentist

This episode is sponsored by Thuma. Thuma is a modern design company that specializes in timeless home essentials that are mindfully made with premium materials and intentional details. To get $100 towards your first bed purchase, go to http://thuma.co/eyeonai   In this episode of the Eye on AI podcast, Pedro Domingos, renowned AI researcher and author of The Master Algorithm, joins Craig Smith to explore the evolution of machine learning, the resurgence of Bayesian AI, and the future of artificial intelligence. Pedro unpacks the ongoing battle between Bayesian and Frequentist approaches, explaining why probability is one of the most misunderstood concepts in AI. He delves into Bayesian networks, their role in AI decision-making, and how they powered Google's ad system before deep learning. We also discuss how Bayesian learning is still outperforming humans in medical diagnosis, search & rescue, and predictive modeling, despite its computational challenges. The conversation shifts to deep learning's limitations, with Pedro revealing how neural networks might be just a disguised form of nearest-neighbor learning. He challenges conventional wisdom on AGI, AI regulation, and the scalability of deep learning, offering insights into why Bayesian reasoning and analogical learning might be the future of AI. We also dive into analogical learning—a field championed by Douglas Hofstadter—exploring its impact on pattern recognition, case-based reasoning, and support vector machines (SVMs). Pedro highlights how AI has cycled through different paradigms, from symbolic AI in the '80s to SVMs in the 2000s, and why the next big breakthrough may not come from neural networks at all. From theoretical AI debates to real-world applications, this episode offers a deep dive into the science behind AI learning methods, their limitations, and what's next for machine intelligence. Don't forget to like, subscribe, and hit the notification bell for more expert discussions on AI, technology, and the future of innovation!    Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction (02:55) The Five Tribes of Machine Learning Explained   (06:34) Bayesian vs. Frequentist: The Probability Debate   (08:27) What is Bayes' Theorem & How AI Uses It   (12:46) The Power & Limitations of Bayesian Networks   (16:43) How Bayesian Inference Works in AI   (18:56) The Rise & Fall of Bayesian Machine Learning   (20:31) Bayesian AI in Medical Diagnosis & Search and Rescue   (25:07) How Google Used Bayesian Networks for Ads   (28:56) The Role of Uncertainty in AI Decision-Making   (30:34) Why Bayesian Learning is Computationally Hard   (34:18) Analogical Learning – The Overlooked AI Paradigm   (38:09) Support Vector Machines vs. Neural Networks   (41:29) How SVMs Once Dominated Machine Learning   (45:30) The Future of AI – Bayesian, Neural, or Hybrid?   (50:38) Where AI is Heading Next  

It's every blogger's curse to return to the same arguments again and again. Matt Yglesias has to keep writing “maybe we should do popular things instead of unpopular ones”, Freddie de Boer has to keep writing “the way culture depicts mental illness is bad”, and for whatever reason, I keep getting in fights about whether you can have probabilities for non-repeating, hard-to-model events. For example: What is the probability that Joe Biden will win the 2024 election? What is the probability that people will land on Mars before 2050? What is the probability that AI will destroy humanity this century? The argument against: usually we use probability to represent an outcome from some well-behaved distribution. For example, if there are 400 white balls and 600 black balls in an urn, the probability of pulling out a white ball is 40%. If you pulled out 100 balls, close to 40 of them would be white. You can literally pull out the balls and do the experiment. In contrast, saying “there's a 45% probability people will land on Mars before 2050” seems to come out of nowhere. How do you know? If you were to say “the probability humans will land on Mars is exactly 45.11782%”, you would sound like a loon. But how is saying that it's 45% any better? With balls in an urn, the probability might very well be 45.11782%, and you can prove it. But with humanity landing on Mars, aren't you just making this number up? Since people on social media have been talking about this again, let's go over it one more depressing, fruitless time. https://www.astralcodexten.com/p/in-continued-defense-of-non-frequentist

We talked about: Rob's background Going from software engineering to Bayesian modeling Frequentist vs Bayesian modeling approach About integrals Probabilistic programming and samplers MCMC and Hakaru Language vs library Encoding dependencies and relationships into a model Stan, HMC (Hamiltonian Monte Carlo) , and NUTS Sources for learning about Bayesian modeling Reaching out to Rob Links: Book 1: https://bayesiancomputationbook.com/welcome.html Book/Course: https://xcelab.net/rm/statistical-rethinking/ Free ML Engineering course: http://mlzoomcamp.com Join DataTalks.Club: https://datatalks.club/slack.html Our events: https://datatalks.club/events.html

Whenever your marketing is being assessed by an analyst, they will use one of two approaches.  The first is called Multi-touch attribution, which takes a customer who's made a purchase decision, then puts weights on the touchpoints they had on various channels (Google calls their model ‘Data-driven attribution”) on the way to that point, to say which touchpoints were most influential.  The other approach they may use is Media Mix Modeling. From what previous podcast guest Kevin Hartman told me about MMM, it's a ‘tremendous undertaking.'  It involves collecting and analyzing historical data in different geographies at different times of the year: sales figures, both legacy and digital marketing channels, and external factors like economic indicators and even weather. It has its own jargon: Incrementality, ratios, betas, impact on objectives. Then there's the math. It uses regression methods, both linear and non-linear, Frequentist vs Bayesian statistics.  I get so overwhelmed with these modeling solutions, it's like the old Who's On First skit. I needed someone who would sort this out for me.  Our guest has been a consultant in the marketing and digital analytics space for 15 years. I'm currently focusing on helping clients quantify the impact of their marketing efforts using Marketing Mix Models, experimentation, and various attribution methodologies. He is so passionate, he started a newsletter called MMM Hub He graduated from Carnegie Mellon with a Masters degree in Information Technology, focused on Business Intelligence & Data Analytics. Jim is great at showcasing other people in the analytics community -He truly believes that all of us are smarter than any one of us. He, along with Simon Poulton, co-host the MeasureUp podcast.  He talked with me from his home in Pittsburgh. Let's meet Jim Gianoglio. Links to everything talked about in the show is found on the show's page at https://funnelreboot.com

Ramesh Johari is a professor at Stanford University focusing on data science methods and practice, as well as the design and operation of online markets and platforms. Beyond academia, Ramesh has advised some incredible startups, including Airbnb, Uber, Bumble, and Stitch Fix. Today we discuss:• What exactly a marketplace is, if you boil it down• What you need to get right to build a successful marketplace• How to optimize any marketplace• An easy litmus test to see if there's an opportunity to build a marketplace in the space• The role of data science in successful marketplaces• Ramesh's philosophy on experimentation and AI• Advice on implementing rating systems• Why learning isn't free—Brought to you by Sanity—The most customizable content layer to power your growth engine | Hex—Helping teams ask and answer data questions by working together | Eppo—Run reliable, impactful experiments—Find the full transcript at: https://www.lennyspodcast.com/marketplace-lessons-from-uber-airbnb-bumble-and-more-ramesh-johari-stanford-professor-startup/—Where to find Ramesh Johari:• LinkedIn: https://www.linkedin.com/in/rameshjohari/• Website: https://web.stanford.edu/~rjohari/• X: https://twitter.com/rameshjohari—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Ramesh's background(04:31) A brief overview of what a marketplace is(08:10) The role of data science in marketplaces(11:21) Common flaws of marketplaces(16:43) Why every founder is a marketplace founder(20:26) How Substack increased value to creators by driving demand(20:58) An example of overcommitting at eBay(22:24) An easy litmus test for marketplaces (25:52) Thoughts on employees vs. contractors(28:02) How to leverage data scientists to improve your marketplace(34:10) Correlation vs. causation(35:27) Decisions that should be made using data(39:29) Ramesh's philosophy on experimentation(41:06) How to find a balance between running experiments and finding new opportunities(44:11) Badging in marketplaces(46:04) The “superhost” badge at Airbnb(49:59) How marketplaces are like a game of Whac-A-Mole(52:41) How to shift an organization's focus from impact to learning(55:43) Frequentist vs. Bayesian A/B testing (57:50) The idea that learning is costly(1:01:55) The basics of rating systems(1:04:41) The problem with averaging(1:07:14) Double-blind reviews at Airbnb(1:08:55) How large language models are affecting data science(1:11:27) Lightning round—Referenced:• Riley Newman on LinkedIn: https://www.linkedin.com/in/rileynewman/• Upwork (formerly Odesk): https://www.upwork.com/• Ancient Agora: https://en.wikipedia.org/wiki/Ancient_Agora_of_Athens• Trajan's Market: https://en.wikipedia.org/wiki/Trajan%27s_Market• Kayak: https://www.kayak.com/• UrbanSitter: https://www.urbansitter.com/• Thumbtack: https://www.thumbtack.com/• Substack: https://substack.com/• Ebay: https://www.ebay.com/• Coase: “The Nature of the Firm”: https://en.wikipedia.org/wiki/The_Nature_of_the_Firm• Stitch Fix: https://www.stitchfix.com/• A/B Testing with Fat Tails: https://www.journals.uchicago.edu/doi/abs/10.1086/710607• The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon): https://www.lennyspodcast.com/the-ultimate-guide-to-ab-testing-ronny-kohavi-airbnb-microsoft-amazon/• Servaes Tholen on LinkedIn: https://www.linkedin.com/in/servaestholen/• Bayesian A/B Testing: A More Calculated Approach to an A/B Test: https://blog.hubspot.com/marketing/bayesian-ab-testing• Designing Informative Rating Systems: Evidence from an Online Labor Market: https://arxiv.org/abs/1810.13028• Reputation and Feedback Systems in Online Platform Markets: https://faculty.haas.berkeley.edu/stadelis/Annual_Review_Tadelis.pdf• How to Lie with Statistics: https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728• David Freedman's books on Amazon: https://www.amazon.com/stores/David-Freedman/author/B001IGLSGA• Four Thousand Weeks: Time Management for Mortals: https://www.amazon.com/Four-Thousand-Weeks-Management-Mortals/dp/0374159122• The Alpinist on Prime Video: https://www.amazon.com/Alpinist-Peter-Mortimer/dp/B09KYDWVVC• Only Murders in the Building on Hulu: https://www.hulu.com/series/only-murders-in-the-building-ef31c7e1-cd0f-4e07-848d-1cbfedb50ddf—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Without statistics, studies are just words. We need some way of translating data in to messages. More and more we read about studies which are using the mysterious Bayesian approach. How does this compare to the standard “frequentist” approach? What are the differences in data collection, analysis, messaging, and interpretation? […]

BeyondPlanck IV On end-to-end simulations in CMB analysis -- Bayesian versus frequentist statistics by M. Brilenkov et al. on Monday 12 September End-to-end simulations play a key role in the analysis of any high-sensitivity CMB experiment, providing high-fidelity systematic error propagation capabilities unmatched by any other means. In this paper, we address an important issue regarding such simulations, namely how to define the inputs in terms of sky model and instrument parameters. These may either be taken as a constrained realization derived from the data, or as a random realization independent from the data. We refer to these as Bayesian and frequentist simulations, respectively. We show that the two options lead to significantly different correlation structures, as frequentist simulations, contrary to Bayesian simulations, effectively include cosmic variance, but exclude realization-specific correlations from non-linear degeneracies. Consequently, they quantify fundamentally different types of uncertainties, and we argue that they therefore also have different and complementary scientific uses, even if this dichotomy is not absolute. Before BeyondPlanck, most pipelines have used a mix of constrained and random inputs, and used the same hybrid simulations for all applications, even though the statistical justification for this is not always evident. BeyondPlanck represents the first end-to-end CMB simulation framework that is able to generate both types of simulations, and these new capabilities have brought this topic to the forefront. The Bayesian BeyondPlanck simulations and their uses are described extensively in a suite of companion papers. In this paper we consider one important applications of the corresponding frequentist simulations, namely code validation. That is, we generate a set of 1-year LFI 30 GHz frequentist simulations with known inputs, and use these to validate the core low-level BeyondPlanck algorithms; gain estimation, correlated noise estimation, and mapmaking. arXiv: http://arxiv.org/abs/http://arxiv.org/abs/2209.04437v1

BeyondPlanck IV On end-to-end simulations in CMB analysis -- Bayesian versus frequentist statistics by M. Brilenkov et al. on Monday 12 September End-to-end simulations play a key role in the analysis of any high-sensitivity CMB experiment, providing high-fidelity systematic error propagation capabilities unmatched by any other means. In this paper, we address an important issue regarding such simulations, namely how to define the inputs in terms of sky model and instrument parameters. These may either be taken as a constrained realization derived from the data, or as a random realization independent from the data. We refer to these as Bayesian and frequentist simulations, respectively. We show that the two options lead to significantly different correlation structures, as frequentist simulations, contrary to Bayesian simulations, effectively include cosmic variance, but exclude realization-specific correlations from non-linear degeneracies. Consequently, they quantify fundamentally different types of uncertainties, and we argue that they therefore also have different and complementary scientific uses, even if this dichotomy is not absolute. Before BeyondPlanck, most pipelines have used a mix of constrained and random inputs, and used the same hybrid simulations for all applications, even though the statistical justification for this is not always evident. BeyondPlanck represents the first end-to-end CMB simulation framework that is able to generate both types of simulations, and these new capabilities have brought this topic to the forefront. The Bayesian BeyondPlanck simulations and their uses are described extensively in a suite of companion papers. In this paper we consider one important applications of the corresponding frequentist simulations, namely code validation. That is, we generate a set of 1-year LFI 30 GHz frequentist simulations with known inputs, and use these to validate the core low-level BeyondPlanck algorithms; gain estimation, correlated noise estimation, and mapmaking. arXiv: http://arxiv.org/abs/http://arxiv.org/abs/2209.04437v1

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Mountain Troll, published by lsusr on June 11, 2022 on LessWrong. It was a sane world. A Rational world. A world where every developmentally normal teenager was taught Bayesian probability. Saundra's math class was dressed in their finest robes. Her teacher, Mr Waze, had invited the monk Ryokan to come speak. It was supposed to be a formality. Monks rarely came down from their mountain hermitages. The purpose of inviting monks to speak was to show respect for how much one does not know. And yet, monk Ryokan had come down to teach a regular high school class of students. Saundra ran to grab monk Ryokan a chair. All the chairs were the same—even Mr Waze's. How could she show respect to the mountain monk? Saundra's eyes darted from chair to chair, looking for the cleanest or least worn chair. While she hesitated, Ryokan sat on the floor in front of the classroom. The students pushed their chairs and desks to the walls of the classroom so they could sit in a circle with Ryokan. "The students have just completed their course on Bayesian probability," said Mr Waze. "I see[1]," said Ryokan. "The students also learned the history of Bayesian probability," said Mr Waze. "I see," said Ryokan. There was an awkward pause. The students waited for the monk to speak. The monk did not speak. "What do you think of Bayesian probability?" said Saundra. "I am a Frequentist," said Ryokan. Mr Waze stumbled. The class gasped. A few students screamed. "It is true that trolling is a core virtue of rationality," said Mr Waze, "but one must be careful not to go too far." Ryokan shrugged. Saundra raised her hand. "You may speak. You need not raise your hand. Rationalism does not privilege one voice above all others," said Ryokan. Saundra's voice quivered. "Why are you a Frequentist?" she said. "Why are you a Bayesian?" said Ryokan. Ryokan kept his face still but he failed to conceal the twinkle in his eye. Saundra glanced at Mr Waze. She forced herself to look away. "May I ask you a question?" said Ryokan. Saundra nodded. "With what probability do you believe in Bayesianism?" said Ryokan. Saundra thought about the question. Obviously not 1 because no Bayesian believes anything with a confidence of 1. But her confidence was still high. "Ninety-nine percent," said Saundra, "Zero point nine nine." "Why?" said Ryokan, "Did you use Bayes' Equation? What was your prior probability before your teacher taught you Bayesianism?" "I notice I am confused," said Saundra. "The most important question a Rationalist can ask herself is 'Why do I think I know what I think I know?'" said Ryokan. "You believes Bayesianism with a confidence of P(A|B)=0.99 where A represents the belief 'Bayesianism is true' and B represents the observation 'your teacher taught you Bayesianism'. A Bayesian belives A|B with a confidence P(A|B) because P(A|B)=P(A)P(B|A)P(B). But that just turns one variable P(A) into three variables P(B|A),P(A),P(B)." Saundra spotted the trap. "I think I see where this is going," said Saundra, "You're going to ask me where I got values for the three numbers P(B|A),P(A),P(B)." Ryokan smiled. "My prior probability P(A) was very small because I didn't know what Bayesian probability was. Therefore P(B|A)P(B) must be very large." said Saundra. Ryokan nodded. "But if P(B|A)P(B) is very large then that means I trust what my teacher says. And a good Rationalist always questions what her teacher says," said Saundra. "Which is why trolling is a fundamental ingredient to Rationalist pedagogy. If teachers never trolled their students then students would get lazy and believe everything that came out of their teachers' mouths," said Ryokan. "Are you trolling me right now? Are you really a Frequentist?" said Saundra. "Is your teacher really a Bayesian?" said Ryokan. Thanks for listening. To help us out with The N...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Mountain Troll, published by lsusr on June 11, 2022 on LessWrong. It was a sane world. A Rational world. A world where every developmentally normal teenager was taught Bayesian probability. Saundra's math class was dressed in their finest robes. Her teacher, Mr Waze, had invited the monk Ryokan to come speak. It was supposed to be a formality. Monks rarely came down from their mountain hermitages. The purpose of inviting monks to speak was to show respect for how much one does not know. And yet, monk Ryokan had come down to teach a regular high school class of students. Saundra ran to grab monk Ryokan a chair. All the chairs were the same—even Mr Waze's. How could she show respect to the mountain monk? Saundra's eyes darted from chair to chair, looking for the cleanest or least worn chair. While she hesitated, Ryokan sat on the floor in front of the classroom. The students pushed their chairs and desks to the walls of the classroom so they could sit in a circle with Ryokan. "The students have just completed their course on Bayesian probability," said Mr Waze. "I see[1]," said Ryokan. "The students also learned the history of Bayesian probability," said Mr Waze. "I see," said Ryokan. There was an awkward pause. The students waited for the monk to speak. The monk did not speak. "What do you think of Bayesian probability?" said Saundra. "I am a Frequentist," said Ryokan. Mr Waze stumbled. The class gasped. A few students screamed. "It is true that trolling is a core virtue of rationality," said Mr Waze, "but one must be careful not to go too far." Ryokan shrugged. Saundra raised her hand. "You may speak. You need not raise your hand. Rationalism does not privilege one voice above all others," said Ryokan. Saundra's voice quivered. "Why are you a Frequentist?" she said. "Why are you a Bayesian?" said Ryokan. Ryokan kept his face still but he failed to conceal the twinkle in his eye. Saundra glanced at Mr Waze. She forced herself to look away. "May I ask you a question?" said Ryokan. Saundra nodded. "With what probability do you believe in Bayesianism?" said Ryokan. Saundra thought about the question. Obviously not 1 because no Bayesian believes anything with a confidence of 1. But her confidence was still high. "Ninety-nine percent," said Saundra, "Zero point nine nine." "Why?" said Ryokan, "Did you use Bayes' Equation? What was your prior probability before your teacher taught you Bayesianism?" "I notice I am confused," said Saundra. "The most important question a Rationalist can ask herself is 'Why do I think I know what I think I know?'" said Ryokan. "You believes Bayesianism with a confidence of P(A|B)=0.99 where A represents the belief 'Bayesianism is true' and B represents the observation 'your teacher taught you Bayesianism'. A Bayesian belives A|B with a confidence P(A|B) because P(A|B)=P(A)P(B|A)P(B). But that just turns one variable P(A) into three variables P(B|A),P(A),P(B)." Saundra spotted the trap. "I think I see where this is going," said Saundra, "You're going to ask me where I got values for the three numbers P(B|A),P(A),P(B)." Ryokan smiled. "My prior probability P(A) was very small because I didn't know what Bayesian probability was. Therefore P(B|A)P(B) must be very large." said Saundra. Ryokan nodded. "But if P(B|A)P(B) is very large then that means I trust what my teacher says. And a good Rationalist always questions what her teacher says," said Saundra. "Which is why trolling is a fundamental ingredient to Rationalist pedagogy. If teachers never trolled their students then students would get lazy and believe everything that came out of their teachers' mouths," said Ryokan. "Are you trolling me right now? Are you really a Frequentist?" said Saundra. "Is your teacher really a Bayesian?" said Ryokan. Thanks for listening. To help us out with The N...

In this episode founding Editor-in-Chief of the Harvard Data Science Review and Professor of Statistics at Harvard University, Prof. Xiao-Li Meng, joins Jon Krohn to dive into data trade-offs that abound, and shares his view on the paradoxical downside of having lots of data. In this episode you will learn: • What the Harvard Data Science Review is and why Xiao-Li founded it [5:31] • The difference between data science and statistics [17:56] • The concept of 'data minding' [22:27] • The concept of 'data confession' [30:31] • Why there's no “free lunch” with data, and the tricky trade-offs that abound [35:20] • The surprising paradoxical downside of having lots of data [43:23] • What the Bayesian, Frequentist, and Fiduciary schools of statistics are, and when each of them is most useful in data science [55:47] Additional materials: www.superdatascience.com/581

We talked about: Juan's background Typical problems in marketing that are solved with ML Attribution model Media Mix Model – detecting uplift and channel saturation Changes to privacy regulations and its effect on user tracking User retention and churn prevention A/B testing to detect uplift Statistical approach vs machine learning (setting a benchmark) Does retraining MMM models often improve efficiency? Attribution model baselines Choosing a decay rate for channels (Bayesian linear regression) Learning resource suggestions Bayesian approach vs Frequentist approach Suggestions for creating a marketing department Most challenging problems in marketing The importance of knowing marketing domain knowledge for data scientists Juan's blog and other learning resources Finding Juan online Links: Juan's PyData talk on uplift modeling: https://youtube.com/watch?v=VWjsi-5yc3w Juan's website: https://juanitorduz.github.io Introduction to Algorithmic Marketing book: https://algorithmic-marketing.online Preventing churn like a bandit: https://www.youtube.com/watch?v=n1uqeBNUlRM MLOps Zoomcamp: https://github.com/DataTalksClub/mlops-zoomcamp Join DataTalks.Club: https://datatalks.club/slack.html Our events: https://datatalks.club/events.html

In this episode, Martin and Jahed sit down with Dr. Shaun Conway of IXO Protocol, to talk about the future of measuring, verifying, and delivering social impact with new mechanisms and designs enabled by blockchains. In the conversation, Dr. Conway leads us through his history working as a medical doctor, then transitioning to HIV / AIDS impact projects in his native South Africa, to working with the WHO, UNICEF, and other NGOs on policy measures. This experience led him to conceive of an impact marketplace long before the current ReFi space brought together off-chain impact with on-chain data. We cover the possibilities for better incentive design, governance structure, and real-life impact measurement through the IXO protocol. This episode will be particularly insightful for those working in NGOs, sustainability and social impact projects, and social impact entrepreneurs. Here are the show notes: Impact Alpha Bonds The Tokenized Impact Economy The Origins of IXO Risk Adjusted Bonding Curves Statistics: Are you Bayesian or Frequentist? Podcast with Chimple / IXO UBS Impact Bonds - Optimus Foundation Nature 2.0 - Ocean Protocol MIT Connection Science

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Moses and the Class Struggle, published by lsusr on April 1, 2022 on LessWrong. "

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Moses and the Class Struggle, published by lsusr on April 1, 2022 on LessWrong. "

April 1, 2022 RJ podcast: Frequentist vs Bayesian statistics. Editor-in-Chief Dr. Sue Yom hosts a discussion of Bayesian statistics with Dr. David Sher, Statistics for the People Editor and Professor of Radiation Oncology at the University of Texas Southwestern, and Dr. Gareth Price, Senior Lecturer in the Division of Cancer Sciences at The University of Manchester and supervising author of the new Red Journal review, "Understanding the Differences Between Bayesian and Frequentist Statistics."

We talked about: Jakob's background The importance of A/B tests Statistical noise A/B test example A/B tests vs expert opinion Traffic splitting, A/A tests, and designing experiments Noisy vs stable metrics – test duration and business cycles Z-tests, T-tests, and time series A/B test crash course advice Frequentist approach vs Bayesian approach A/B/C/D tests Pizza dough Links: Jakob's LinkedIn: https://www.linkedin.com/in/jakob-graff-a6113a3a/ Product Analyst role at Inkitt: https://jobs.lever.co/inkitt/d2b0427a-f37f-4002-975d-28bd60b56d70 Join DataTalks.Club: https://datatalks.club/slack.html Our events: https://datatalks.club/events.html

We all want to know how to improve the number of conversions we get on our websites. A lot of the time, we try to do that by rearranging the layout or switch up page elements. Then we go to our dashboards, see if the number went up or down. You've probably spotted the flaw in this. This is circular reasoning, a change in customer behavior can't be proven by "looking back at data, trying to decide whether or not it was some sort of change that we made" As conversion expert, Matt Gershoff puts it.  What's the right way to do this? It's simple, reverse the order of events and start off with your hunch about what you should change, run an experiment on your customers, proving or disproving if the thing you believe causes their behaviour actually has that effect.  This discipline is called CRO, which stands for Conversion Rate Optimization. There's someone who's superbly qualified to talk about this and I'm lucky to have known her for the past few years.  Deborah unknowingly ran her first optimization study in school at the age of 8, when she put her classmates through a science experiment where they looked at pieces of construction paper tacked on a bristol board. Little Deborah grew up to earn a master's of science degree, specializing in eye tracking technology. Today, Deborah applies her specialized skillset to Conversion Rate Optimization (CRO). She founded a well known resource website where she has published hundreds of client A/B test case studies. She also has a certificate in graphic design, giving her the blend of left and right-brain thinking that's just right for working in CRO.   Some things to listen for: Marketers who run paid search & paid social will want to listen to what she has to say about mixing traffic from various channels together.  She explains what minimum traffic constraints we're under for A/B tests, and why with small volumes we're better to put 2-pages into a test where only 1 thing has changed, versus testing multiple pages or multiple variables at the same time  She has some tips on how we can maintain objectivity as we run our tests and as we present the results to our leadership.  Let's go hear from Deborah O'Malley. People/Products/Concepts Mentioned in Show Deborah's Five Step Process: 1. Practice Know Your Audience (KYA) and research the performance of pages via site analytics and heatmaps like CrazyEgg and HotJar.  2. Form a SMART experiment hypothesis. Here is a single-sentence version:  "Because I observed and received feedback on [what is causing users to convert at X rate], I believe that [the change to be tested]  For [targeted segment or all users] will result in [specific lift in conversion rate]." Ensure you have sufficient traffic / time for the experiment, using one of these calculators: Evan's Awesome A/B Tools CXL Shinyapps 3. Pick a Tool to run the experiment. Here are some common ones: Google Optimize VWO Optimizely Adobe Target 4. Run the experiment. 5. Implement the winning page and monitor for expected results.  How A/B pages are commonly named: "A" is your Control (original version) and "B' is your Variant (includes change you're testing). Here is a more in-depth Conversion Rate Optimization glossary Note the different frameworks that tools use:  Frequentist vs Bayesian frameworks. Depending on the statistical framework used, the timeframe needed to give results will change. Connect with Deborah on LinkedIn and Twitter Deborah's websites include GuessTheTest.com and Convert Experts Episode Reboot Run an A/A Test, here is Deborah's article on why you should and what to consider. for more details, please visit https://funnelreboot.com/episode-65-conversion-rate-optimization-with-deborah-omalley/

Welcome to the new series on products for Product Managers: A/B Testing! In this episode, Matt and Moshe are joined by Rommil Santiago to discuss the use of A/B testing tools to experiment different product solutions.Rommil is a Product Lead - Experimentation, Personalization, and Analytics Platform at Loblaw Digital, and the Founder and Owner of Experiment Nation.Experiment Nation is a community of conversion rate optimizers, product managers, and data analysts from all over the world who are connected by their passion for experimentation.As product managers, we strive to build the best solution for our customers, and A/B testing tools play a big role in validating our hypotheses on what works best for the users. Some things to look for when choosing an A/B Testing product:Which statistical approach the tool is taking, Frequentist, Bayesian, Sequential, or a hybrid one?How well is the tool integrated with other systems, mainly data analytics?Does the tool have front end and/or back end testing?What interface does the tool have to set up the tests?How do we communicate the test results with our team?And so much more! Come join Matt, Moshe and Rommil as they discuss the different scenarios and what they expect to see in the upcoming episodes discussing specific A/B Testing tools. *You can connect with Rommil at:LinkedIn: https://www.linkedin.com/in/rommil/ Experiment Nation website: https://experimentnation.com/ Experiment Nation on LinkedIn: https://www.linkedin.com/company/experiment-nation/ *You can find the podcast's page, and connect with Matt and Moshe on Linkedin: linkedin.com/company/product-for-product-podcast *Matt - www.linkedin.com/in/mattgreenanalytics *Moshe - www.linkedin.com/in/mikanovsky *Note: any views mentioned in the podcast are the sole views of our hosts and guests, and do not represent the products mentioned in any way. Please leave us a review and feedback ⭐️⭐️⭐️⭐️⭐️

 Chelsea is a full time faculty member teaching undergraduate Data Science and Computer Science, and earned her PhD this year in Computational and Data Science. She also does casual statistical consulting at the Chatistician which aims to empower people to do their own statistics well. In her free time, Chelsea makes educational and just-for-fun statistics memes, TikToks and art. 

Jakob and Todd discuss the philosophy of statistics. Frequentist and Bayesian approaches. Fisher, Neyman, and Pearson and statistical methods for evaluating hypotheses. Deborah Mayo and statistical inference as severe testing. Proper and improper uses of p-values. The pitfalls of data dredging and p-hacking. Conditions under which prior probabilities make Bayesian approaches particularly useful. The utility of Bayesian concepts like priors, posteriors, updating, and loss functions in machine learning. Bayes' Theorem versus Bayesianism as a statistical philosophy. An algorithmic ‘method of methods' for when to apply various statistical tools as an AI-complete problem. Important test cases in statistics like the Higgs Boson observation, the Eddington experiment for General Relativity, and the causal link between smoking and cancer. The problem of induction. Inferring the direction of causation for correlated variables. Karl Popper, falsification, and the impossibility of confirmation. What counts as evidence. Randomness as a limitation on knowledge and as a feature of reality itself. The ontological status and nature of a probability distribution, of classical values and as a quantum property.

We discuss a previous Big Data Bowl finalist paper `Expected Hypothetical Completion Probability` (https://arxiv.org/abs/1910.12337) with authors Sameer Deshpande (@skdeshpande91) and Kathy Evans (@CausalKathy). Sameer is a postdoctoral associate at MIT. Prior to that, he completed his Ph.D. at the Wharton School of the University of Pennsylvania. He is broadly interested in Bayesian methods and causal inference. He is a long-suffering but unapologetic fan of America's Team. He's also a fan of the Dallas Mavericks. Kathy is the Director of Strategic Research for the Toronto Raptors. She completed her Ph.D. in Biostatistics at Harvard University. She doesn't have an opinion on Frequentist vs Bayesian or R vs Python, but will get very upset if Rise of Skywalker is your favorite Star Wars movie. For additional references mentioned in the show: Big Data Bowl notebooks: https://www.kaggle.com/c/nfl-big-data-bowl-2021/notebooks BART: https://arxiv.org/abs/0806.3286 XBART: Accelerated Bayesian Additive Regression Trees https://jingyuhe.com/xbart.html Matthew Reyers (@Stats_By_Matt) thesis: https://twitter.com/Stats_By_Matt/status/1296570171687989249?s=20

Today's guest is Brian Blais, a professor of Science at Bryant University. His approach to teaching statistical inference includes taking the Bayesian approach instead of delegating it to an advanced or elective topic. We talk about the Bayesian vs Frequentist debate, how to navigate the disconnect between them, and the role of imagination when discovering truth.

This week we open with a critical take on our current system of disseminating scientific research, specifically focusing on the prevalence of -- and dependence on -- medical writers. In the second half of the episode, we interview Dr. Allen Pannell of the Haslam College of Business at the University of Tennessee on using a Frequentist approach vs a Bayesian approach in the context of a single clinical trial. Frequentist vs Bayesian: doi.org/10.1136/bmjopen-2018-024256 Dr. Pannell's breast cancer support group: https://www.breastconnect.org/ Dr. Pannell's research project: http://tinyurl.com/peerErPrHer2 Back us on Patreon! www.patreon.com/plenarysession

Bayesian vs. Frequentist. False Positive vs. False Negative. Truth vs. Uncertainty. It's the world of A/B testing! In this bonus mini-episode, Moe sat down with Chad Sanderson from Subway to discuss some of the pitfalls of A/B testing -- the nuances that may seem subtle, but are anything but trivial when it comes to planning and running a test.

In this episode, i'm joined by Clare Gollnick, CTO of Terbium Labs, to discuss her thoughts on the “reproducibility crisis” currently haunting the scientific landscape. For a little background, a “Nature” survey in 2016 showed that "more than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments." Clare gives us her take on the situation, and how it applies to data science, along with some great nuggets about the philosophy of data and a few interesting use cases as well. We also cover her thoughts on Bayesian vs Frequentist techniques and while we’re at it, the Vim vs Emacs debate. No, actually I’m just kidding on that last one. But this was indeed a very fun conversation that I think you’ll enjoy! For the complete show notes, visit twimlai.com/talk/121.

Daniel Lakens (Eindhoven University of Technology) drops in to talk statistical inference with James and Dan. Here’s what they cover: How did Daniel get into statistical inference? Are we overdoing the Frequentist vs. Bayes debate? What situations better suit Bayesian inference? The over advertising of Bayesian inference Study design is underrated The limits of p-values Why not report both p-values and Bayes factors? The “perfect t-test” script and the difference between Student’s and Welch’s t-tests The two-one sided test Frequentist and Bayesian approaches for stopping procedures Why James and Dan started the podcast The worst bits of advice that Daniel has heard about statistical inference Dan discuss a new preprint on Bayes factors in psychiatry Statistical power Excel isn’t all bad… The importance of accessible software We ask Daniel about his research workflow - how does he get stuff done? Using blog posts as a way of gauging interest in a topic Chris Chambers’ new book: The seven deadly sins of psychology Even more names for methodological terrorists Links Daniel on Twitter - @lakens Daniel’s course - https://www.coursera.org/learn/statistical-inferences Daniel’s blog - http://daniellakens.blogspot.no TOSTER - http://daniellakens.blogspot.no/2016/12/tost-equivalence-testing-r-package.html Dan’s preprint on Bayesian alternatives for psychiatry research - https://osf.io/sgpe9/ Understanding the new statistics - https://www.amazon.com/Understanding-New-Statistics-Meta-Analysis-Multivariate/dp/041587968X Daniel’s effect size paper - http://journal.frontiersin.org/article/10.3389/fpsyg.2013.00863/full The seven deadly sins of Psychology - http://press.princeton.edu/titles/10970.html Special Guest: Daniel Lakens.

In this episode, Hilary and Roger discuss the new direction for the journal Biostatistics, the recent fracas over ggplot2 and base graphics in R, and whether collecting more data is always better than collecting less (fewer?) data. Also, Hilary and Roger respond to some listener questions and more free advertising. If you have questions you’d like us to answer, you can send them to nssdeviations@gmail.com or tweet us at @NSSDeviations. Show notes: Jeff Leek on why he doesn’t use ggplot2 (http://goo.gl/am6I3r) David Robinson on why he uses ggplot2 (http://goo.gl/UpqO3c) Nathan Yau’s post comparing ggplot2 and base graphics (http://goo.gl/6iEB2I) Biostatistics Medium post (https://goo.gl/YuhFgB) Photoviz (http://goo.gl/tXNdCA) PigeonAir (https://twitter.com/PigeonAir) I just want to plot() (https://goo.gl/jqlg0G) Hilary and Rush Limbaugh (https://goo.gl/vvCfkl) Deep learning training set (http://imgur.com/a/K4RWn) NSSD Patreon page! (https://goo.gl/TbihVq)

Doing some science, and want to know if you might have found something? Or maybe you've just accomplished the scientific equivalent of going fishing and reeling in an old boot? Frequentist p-values can help you distinguish between "eh" and "oooh interesting". Also, there's a lot of physics in this episode, nerds.

This week we are joined by Professor Matheus Grasselli. Matheus is the Deputy Director of the Fields Institute for Research in Mathematical Sciences, and an associate professor at McMaster University, where he is the co-director of PhiMac, the Financial Mathematics Laboratory. He also writes a blog on Quantitative Finance for the Fields Institute, where he discusses his work, and thoughts on economic modelling, complexity theory, and probability. Matheus has been working with Prof. Steve Keen to help give a mathematicians viewpoint on his ground-breaking monetary economic models of the capitalist system. We discuss the current state of neoclassical macroeconomic modelling, complexity and emergence, Wynn Godley and his stock-flow consistent models, Hyman Minsky and Ponzi finance, black swans and fragility, Bayesian vs Freqentist statistics, Poker, and Samuel Beckett. You can check out his blog here: http://fieldsfinance.blogspot.co.uk/