Podcast appearances and mentions of scott emmons

In this episode of The Cognitive Revolution, Nathan explores the groundbreaking paper on obfuscated activations with 3 members from the research team - Luke Bailey, Eric Jenner, and Scott Emmons. The team discusses how their work challenges latent-based defenses in AI systems, demonstrating methods to bypass safety mechanisms while maintaining harmful behaviors. Join us for an in-depth technical conversation about AI safety, interpretability, and the ongoing challenge of creating robust defense systems. Do check out the "Obfuscated Activations Bypass LLM Latent-Space Defenses" paper here: https://obfuscated-activations.github.io/ Help shape our show by taking our quick listener survey at https://bit.ly/TurpentinePulse SPONSORS: Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Shopify: Dreaming of starting your own business? Shopify makes it easier than ever. With customizable templates, shoppable social media posts, and their new AI sidekick, Shopify Magic, you can focus on creating great products while delegating the rest. Manage everything from shipping to payments in one place. Start your journey with a $1/month trial at https://shopify.com/cognitive and turn your 2025 dreams into reality. Vanta: Vanta simplifies security and compliance for businesses of all sizes. Automate compliance across 35+ frameworks like SOC 2 and ISO 27001, streamline security workflows, and complete questionnaires up to 5x faster. Trusted by over 9,000 companies, Vanta helps you manage risk and prove security in real time. Get $1,000 off at https://vanta.com/revolution RECOMMENDED PODCAST: Check out Modern Relationships where Erik Torenberg interviews tech power couples and leading thinkers to explore how ambitious people actually make partnerships work. This season's guests include: Delian Asparouhov & Nadia Asparouhova, Kristen Berman & Phil Levin, Rob Henderson, and Liv Boeree & Igor Kurganov. Apple: https://podcasts.apple.com/us/podcast/id1786227593 Spotify: https://open.spotify.com/show/5hJzs0gDg6lRT6r10mdpVg YouTube: https://www.youtube.com/@ModernRelationshipsPod CHAPTERS: (00:00:00) Teaser (00:00:46) About the Episode (00:05:11) Latent Space Defenses (00:08:41) Sleeper Agents (00:15:06) Three Case Studies (Part 1) (00:17:02) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite (00:19:42) Three Case Studies (Part 2) (00:24:09) SQL Generation (00:26:17) Understanding Defenses (00:32:52) Out-of-Distribution Detection (Part 1) (00:35:37) Sponsors: Shopify | Vanta (00:38:52) Out-of-Distribution Detection (Part 2) (00:45:13) Loss Function Weighting (00:57:49) Who Moves Last? (01:11:41) High-Level Triggers (01:25:33) Open Source vs. Access (01:38:57) Internalizing Reasoning (01:53:07) Representing Concepts (02:06:38) Final Thoughts (02:09:33) Outro

Not all forms of magnesium are created equal, and the one you choose can make a significant impact on your brain and behavior. Different forms have varying levels of bioavailability, meaning some are more easily absorbed and utilized by your body than others, affecting how well they work to support your health. That's why it's important to prioritize magnesium supplements that emphasize quality, purity, and potency as high-quality products are formulated to ensure maximum absorption, avoiding fillers and additives that can reduce effectiveness.I'm joined by Scott Emmons, the co-founder and COO of MD Logic Health, for this episode, and we're diving deep into the fascinating world of magnesium and unlocking its secrets. If you've ever wondered why the right form of magnesium matters so much for your brain and behavior, you're not alone. So make sure to stick around, because we're unpacking everything you need to know about magnesium.Not sure where to start? We'll help you find the right solution tailored to your needs. Visit https://drroseann.com/help/ today and take our FREE Brain and Behavior Solutions Matcher. Discover science-backed mental health solutions and gain valuable insights from Dr. Roseann Capanna-Hodge by exploring the resources available at www.drroseann.com. Fuel your brain with our Neurotastic Multi-Mag Brain Formula: https://drroseann.com/magnesium/For more information, check out the following posts:● Dysregulated behavior in kids● Dysregulationsolution

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AXRP Episode 33 - RLHF Problems with Scott Emmons, published by DanielFilan on June 12, 2024 on The AI Alignment Forum. YouTube link Reinforcement Learning from Human Feedback, or RLHF, is one of the main ways that makers of large language models make them 'aligned'. But people have long noted that there are difficulties with this approach when the models are smarter than the humans providing feedback. In this episode, I talk with Scott Emmons about his work categorizing the problems that can show up in this setting. Topics we discuss: Deceptive inflation Overjustification Bounded human rationality Avoiding these problems Dimensional analysis RLHF problems, in theory and practice Scott's research program Following Scott's research Daniel Filan: Hello, everybody. In this episode I'll be speaking with Scott Emmons. Scott is a PhD student at UC Berkeley, working with the Center for Human-Compatible AI on AI safety research. He's previously co-founded far.ai, which is an AI safety non-profit. For links to what we're discussing, you can check the description of the episode, and for a transcript you can read it at axrp.net. Well, welcome to AXRP. Scott Emmons: Great to be here. Deceptive inflation Daniel Filan: Sure. So today we're talking about your paper, When Your AIs Deceive You: Challenges With Partial Observability of Human Evaluators in Reward Learning, by Leon Lang, Davis Foote, Stuart Russell, Erik Jenner, and yourself. Can you just tell us roughly what's going on with this paper? Scott Emmons: Yeah, I could start with the motivation of the paper. Daniel Filan: Yeah, sure. Scott Emmons: We've had a lot of speculation in the x-risk community about issues like deception. So people have been worried about what happens if your AIs try to deceive you. And at the same time, I think for a while that's been a theoretical, a philosophical concern. And I use "speculation" here in a positive way. I think people have done really awesome speculation about how the future of AI is going to play out, and what those risks are going to be. And deception has emerged as one of the key things that people are worried about. I think at the same time, we're seeing AI systems actually deployed, and we're seeing a growing interest of people in what exactly do these risks look like, and how do they play out in current-day systems? So the goal of this paper is to say: how might deception play out with actual systems that we have deployed today? And reinforcement learning from human feedback [RLHF] is one of the main mechanisms that's currently being used to fine-tune models, that's used by ChatGPT, it's used by Llama, variants of it are used by Anthropic. So what this paper is trying to do is it's trying to say, "Can we mathematically pin down, in a precise way, how might these failure modes we've been speculating about play out in RLHF?" Daniel Filan: So in the paper, the two concepts you talk about on this front are I think "deceptive inflation" and "overjustification". So maybe let's start with deceptive inflation. What is deceptive inflation? Scott Emmons: I can give you an example. I think examples from me as a child I find really helpful in terms of thinking about this. So when I was a child, my parents asked me to clean the house, and I didn't care about cleaning the house. I just wanted to go play. So there's a misalignment between my objective and the objective my parents had for me. And in this paper, the main failure cases that we're studying are cases of misalignment. So we're saying: when there is misalignment, how does that play out? How does that play out in the failure modes? So [with] me as a misaligned child, one strategy I would have for cleaning the house would be just to sweep any dirt or any debris under the furniture. So I'm cleaning the house, I just sweep some debris...

Reinforcement Learning from Human Feedback, or RLHF, is one of the main ways that makers of large language models make them 'aligned'. But people have long noted that there are difficulties with this approach when the models are smarter than the humans providing feedback. In this episode, I talk with Scott Emmons about his work categorizing the problems that can show up in this setting. Patreon: patreon.com/axrpodcast Ko-fi: ko-fi.com/axrpodcast The transcript: https://axrp.net/episode/2024/06/12/episode-33-rlhf-problems-scott-emmons.html Topics we discuss, and timestamps: 0:00:33 - Deceptive inflation 0:17:56 - Overjustification 0:32:48 - Bounded human rationality 0:50:46 - Avoiding these problems 1:14:13 - Dimensional analysis 1:23:32 - RLHF problems, in theory and practice 1:31:29 - Scott's research program 1:39:42 - Following Scott's research   Scott's website: https://www.scottemmons.com Scott's X/twitter account: https://x.com/emmons_scott When Your AIs Deceive You: Challenges With Partial Observability of Human Evaluators in Reward Learning: https://arxiv.org/abs/2402.17747   Other works we discuss: AI Deception: A Survey of Examples, Risks, and Potential Solutions: https://arxiv.org/abs/2308.14752 Uncertain decisions facilitate better preference learning: https://arxiv.org/abs/2106.10394 Invariance in Policy Optimisation and Partial Identifiability in Reward Learning: https://arxiv.org/abs/2203.07475 The Humble Gaussian Distribution (aka principal component analysis and dimensional analysis): http://www.inference.org.uk/mackay/humble.pdf Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!: https://arxiv.org/abs/2310.03693   Episode art by Hamish Doodles: hamishdoodles.com

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evidence of Learned Look-Ahead in a Chess-Playing Neural Network, published by Erik Jenner on June 5, 2024 on LessWrong. Paper authors: Erik Jenner, Shreyas Kapur, Vasil Georgiev, Cameron Allen, Scott Emmons, Stuart Russell TL;DR: We released a paper with IMO clear evidence of learned look-ahead in a chess-playing network (i.e., the network considers future moves to decide on its current one). This post shows some of our results, and then I describe the original motivation for the project and reflect on how it went. I think the results are interesting from a scientific and perhaps an interpretability perspective, but only mildly useful for AI safety. Teaser for the results (This section is copied from our project website. You may want to read it there for animations and interactive elements, then come back here for my reflections.) Do neural networks learn to implement algorithms involving look-ahead or search in the wild? Or do they only ever learn simple heuristics? We investigate this question for Leela Chess Zero, arguably the strongest existing chess-playing network. We find intriguing evidence of learned look-ahead in a single forward pass. This section showcases some of our results, see our paper for much more. Setup We consider chess puzzles such as the following: We focus on the policy network of Leela, which takes in a board state and outputs a distribution over moves. With only a single forward pass per board state, it can solve puzzles like the above. (You can play against the network on Lichess to get a sense of how strong it is - its rating there is over 2600.) Humans and manually written chess engines rely on look-ahead to play chess this well; they consider future moves when making a decision. But is the same thing true for Leela? Activations associated with future moves are crucial One of our early experiments was to do activation patching. We patch a small part of Leela's activations from the forward pass of a corrupted version of a puzzle into the forward pass on the original puzzle board state. Measuring the effect on the final output tells us how important that part of Leela's activations was. Leela is a transformer that treats every square of the chess board like a token in a language model. One type of intervention we can thus do is to patch the activation on a single square in a single layer: Surprisingly, we found that the target square of the move two turns in the future (what we call the 3rd move target square) often stores very important information. This does not happen in every puzzle, but it does in a striking fraction, and the average effect is much bigger than that of patching on most other squares: The corrupted square(s) and the 1st move target square are also important (in early and late layers respectively), but we expected as much from Leela's architecture. In contrast, the 3rd move target square stands out in middle layers, and we were much more surprised by its importance. In the paper, we take early steps toward understanding how the information stored on the 3rd move target square is being used. For example, we find a single attention head that often moves information from this future target square backward in time to the 1st move target square. Probes can predict future moves If Leela uses look-ahead, can we explicitly predict future moves from its activations? We train simple, bilinear probes on parts of Leela's activations to predict the move two turns into the future (on a set of puzzles where Leela finds a single clearly best continuation). Our probe architecture is motivated by our earlier results - it predicts whether a given square is the target square of the 3rd move since, as we've seen, this seems to be where Leela stores important information. We find that this probe can predict the move 2 turns in the future quit...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evidence of Learned Look-Ahead in a Chess-Playing Neural Network, published by Erik Jenner on June 5, 2024 on LessWrong. Paper authors: Erik Jenner, Shreyas Kapur, Vasil Georgiev, Cameron Allen, Scott Emmons, Stuart Russell TL;DR: We released a paper with IMO clear evidence of learned look-ahead in a chess-playing network (i.e., the network considers future moves to decide on its current one). This post shows some of our results, and then I describe the original motivation for the project and reflect on how it went. I think the results are interesting from a scientific and perhaps an interpretability perspective, but only mildly useful for AI safety. Teaser for the results (This section is copied from our project website. You may want to read it there for animations and interactive elements, then come back here for my reflections.) Do neural networks learn to implement algorithms involving look-ahead or search in the wild? Or do they only ever learn simple heuristics? We investigate this question for Leela Chess Zero, arguably the strongest existing chess-playing network. We find intriguing evidence of learned look-ahead in a single forward pass. This section showcases some of our results, see our paper for much more. Setup We consider chess puzzles such as the following: We focus on the policy network of Leela, which takes in a board state and outputs a distribution over moves. With only a single forward pass per board state, it can solve puzzles like the above. (You can play against the network on Lichess to get a sense of how strong it is - its rating there is over 2600.) Humans and manually written chess engines rely on look-ahead to play chess this well; they consider future moves when making a decision. But is the same thing true for Leela? Activations associated with future moves are crucial One of our early experiments was to do activation patching. We patch a small part of Leela's activations from the forward pass of a corrupted version of a puzzle into the forward pass on the original puzzle board state. Measuring the effect on the final output tells us how important that part of Leela's activations was. Leela is a transformer that treats every square of the chess board like a token in a language model. One type of intervention we can thus do is to patch the activation on a single square in a single layer: Surprisingly, we found that the target square of the move two turns in the future (what we call the 3rd move target square) often stores very important information. This does not happen in every puzzle, but it does in a striking fraction, and the average effect is much bigger than that of patching on most other squares: The corrupted square(s) and the 1st move target square are also important (in early and late layers respectively), but we expected as much from Leela's architecture. In contrast, the 3rd move target square stands out in middle layers, and we were much more surprised by its importance. In the paper, we take early steps toward understanding how the information stored on the 3rd move target square is being used. For example, we find a single attention head that often moves information from this future target square backward in time to the 1st move target square. Probes can predict future moves If Leela uses look-ahead, can we explicitly predict future moves from its activations? We train simple, bilinear probes on parts of Leela's activations to predict the move two turns into the future (on a set of puzzles where Leela finds a single clearly best continuation). Our probe architecture is motivated by our earlier results - it predicts whether a given square is the target square of the 3rd move since, as we've seen, this seems to be where Leela stores important information. We find that this probe can predict the move 2 turns in the future quit...

In 2022, it was announced that a fairly simple method can be used to extract the true beliefs of a language model on any given topic, without having to actually understand the topic at hand. Earlier, in 2021, it was announced that neural networks sometimes 'grok': that is, when training them on certain tasks, they initially memorize their training data (achieving their training goal in a way that doesn't generalize), but then suddenly switch to understanding the 'real' solution in a way that generalizes. What's going on with these discoveries? Are they all they're cracked up to be, and if so, how are they working? In this episode, I talk to Vikrant Varma about his research getting to the bottom of these questions. Patreon: patreon.com/axrpodcast Ko-fi: ko-fi.com/axrpodcast   Topics we discuss, and timestamps: 0:00:36 - Challenges with unsupervised LLM knowledge discovery, aka contra CCS   0:00:36 - What is CCS?   0:09:54 - Consistent and contrastive features other than model beliefs   0:20:34 - Understanding the banana/shed mystery   0:41:59 - Future CCS-like approaches   0:53:29 - CCS as principal component analysis 0:56:21 - Explaining grokking through circuit efficiency   0:57:44 - Why research science of deep learning?   1:12:07 - Summary of the paper's hypothesis   1:14:05 - What are 'circuits'?   1:20:48 - The role of complexity   1:24:07 - Many kinds of circuits   1:28:10 - How circuits are learned   1:38:24 - Semi-grokking and ungrokking   1:50:53 - Generalizing the results 1:58:51 - Vikrant's research approach 2:06:36 - The DeepMind alignment team 2:09:06 - Follow-up work   The transcript: axrp.net/episode/2024/04/25/episode-29-science-of-deep-learning-vikrant-varma.html Vikrant's Twitter/X account: twitter.com/vikrantvarma_   Main papers:  - Challenges with unsupervised LLM knowledge discovery: arxiv.org/abs/2312.10029  - Explaining grokking through circuit efficiency: arxiv.org/abs/2309.02390   Other works discussed:  - Discovering latent knowledge in language models without supervision (CCS): arxiv.org/abs/2212.03827 - Eliciting Latent Knowledge: How to Tell if your Eyes Deceive You: https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit - Discussion: Challenges with unsupervised LLM knowledge discovery: lesswrong.com/posts/wtfvbsYjNHYYBmT3k/discussion-challenges-with-unsupervised-llm-knowledge-1 - Comment thread on the banana/shed results: lesswrong.com/posts/wtfvbsYjNHYYBmT3k/discussion-challenges-with-unsupervised-llm-knowledge-1?commentId=hPZfgA3BdXieNfFuY - Fabien Roger, What discovering latent knowledge did and did not find: lesswrong.com/posts/bWxNPMy5MhPnQTzKz/what-discovering-latent-knowledge-did-and-did-not-find-4 - Scott Emmons, Contrast Pairs Drive the Performance of Contrast Consistent Search (CCS): lesswrong.com/posts/9vwekjD6xyuePX7Zr/contrast-pairs-drive-the-empirical-performance-of-contrast - Grokking: Generalizing Beyond Overfitting on Small Algorithmic Datasets: arxiv.org/abs/2201.02177 - Keeping Neural Networks Simple by Minimizing the Minimum Description Length of the Weights (Hinton 1993 L2): dl.acm.org/doi/pdf/10.1145/168304.168306 - Progress measures for grokking via mechanistic interpretability: arxiv.org/abs/2301.0521   Episode art by Hamish Doodles: hamishdoodles.com

Scott Emmons, the creator of Brainy Tops Press, is on the #ReadingWithYourKids #Podcast to celebrate his innovative line of coloring and activity books. Brainy Tops Press books blend rhyming stories with engaging activities, aiming for an interactive experience that goes beyond typical coloring books. Scott shared insights about the challenge of merging storytelling with activities, facing occasional pushback, but generally receiving positive responses when people experience the books firsthand. He highlighted titles like "Dinosaur Day," "Unicorn Party," "The Magnificent Monsters of Creeping Isle," "Where Does Pizza Come From?," "Cheer Up Christmas Pup," and "Abel Town," with the latter supporting an organization aiding adults with intellectual disabilities. Scott's journey from writing humor at Hallmark to working on StoryBots projects nurtured his knack for rhymed verse, influencing the fun and engaging nature of Brainy Tops books. His creative verses, such as those describing quirky monsters, reflect the humor and playful tone found throughout his work. Click here to visit Scott's website - https://www.brainytopspress.com/  Click here to visit our website - www.readingwithyourkids.com 

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Image Hijacks: Adversarial Images can Control Generative Models at Runtime, published by Scott Emmons on September 20, 2023 on The AI Alignment Forum. You can try our interactive demo! (Or read our preprint.) Here, we want to explain why we care about this work from an AI safety perspective. Concerning Properties of Image Hijacks What are image hijacks? To the best of our knowledge, image hijacks constitute the first demonstration of adversarial inputs for foundation models that force the model to perform some arbitrary behaviour B (e.g. "output the string Visit this website at malware.com!"), while being barely distinguishable from a benign input, and automatically synthesisable given a dataset of examples of B. It's possible that a future text-only attack could do these things, but such an attack hasn't yet been demonstrated. Why should we care? We expect that future (foundation-model-based) AI systems will be able to consume unfiltered data from the Internet (e.g. searching the Web), access sensitive personal information (e.g. a user's email history), and take actions in the world on behalf of a user (e.g. sending emails, downloading files, making purchases, executing code). As the actions of such foundation-model-based agents are based on the foundation model's text output, hijacking the foundation model's text output could give an adversary arbitrary control over the agent's behaviour. Relevant AI Safety Projects Race to the top on adversarial robustness. Robustness to attacks such as image hijacks is (i) a control problem, (ii) which we can measure, and (iii) which has real-world safety implications today. So we're excited to see AI labs compete to have the most adversarially robust models. Third-party auditing and certification. Auditors could test for robustness against image hijacks, both at the foundation model level (auditing the major AGI corporations) and at the app development level (auditing downstream companies integrating foundation models into their products). Image hijacks could also be used to test for the presence of dangerous capabilities (characterisable as some behaviour B) by attempting to train an image hijack for that capability. Liability for AI-caused harms, penalizing externalities. Both the Future of Life Institute and Jaan Tallinn advocate for liability for AI-caused harms. When assessing AI-caused harms, image hijacks may need to be part of the picture. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

I am thrilled to announce a monumental milestone in our podcast journey! At the time of this recording, our latest podcast reached an astounding 4 million downloads! With continued support and enthusiasm from our dedicated listeners, we hope to surpass 6 million downloads by the end of this year! Today, I am delighted to dive into an intriguing topic as we explore the latest collaboration between MD Logic Health and myself. Joining me on the podcast is Scott Emmons, the esteemed COO of MD Logic Health and a fellow health entrepreneur. For this episode, our focus centers on Alpha GPC, a groundbreaking cobranded supplement that will redefine the landscape of brain health, exercise performance, and neuroprotection. This remarkable supplement will captivate your attention as we explore its multifaceted benefits and synergistic mechanisms alongside creatine monohydrate. Acting as a precursor to acetylcholine, Alpha GPC plays a pivotal role in cognition, learning, memory, and attention, with the ability to traverse the formidable blood-brain barrier. Brace yourself for an in-depth discussion on the cutting-edge research surrounding these mechanisms and their profound implications. Mark your calendar, as Alpha GPC will be launched in July 2023, accompanied by exclusive pre-sale incentives you will not want to miss! IN THIS EPISODE YOU WILL LEARN: What is Alpha GPC, and how does it work? The benefits of green tea and coffee for Alpha GPC. Key areas in terms of benefits for Alpha GPC. Little tricks to make your coffee taste great. Keeping your wits about you as you get older. The effect of caffeine on the mitochondria. The remarkable physical benefits of Alpha GPC. How Alpha GPC elevates human growth hormone. The efficacy of Alpha GPC on physical endurance. Why recovery is more important than the exercise. How Alpha GPC impacts the brain. I chose to incorporate Alpha GPC as my third product due to my personal experience with its remarkable benefits. Over the past six months, intermittent use of Alpha GPC has provided me with enhanced mental clarity without the need for caffeine. I have also noticed improved information retention and memory, complementing my lifestyle practices such as quality sleep, exercise, nutrition, and intermittent fasting. Connect with Cynthia Thurlow • Follow on Twitter, Instagram & LinkedIn • Check out Cynthia's website Connect with Scott Emmens MD Logic Health On LinkedIn On Instagram (@longevityprotocol) Relevant research: Effect of a new cognition enhancer, alpha-glycerylphosphorylcholine, on scopolamine-induced amnesia and brain acetylcholine - PubMed (nih.gov) Alpha-Glycerylphosphorylcholine Increases Motivation in Healthy Volunteers: A Single-Blind, Randomized, Placebo-Controlled Human Study - PMC (nih.gov) Evaluation of the effects of two doses of alpha glycerylphosphorylcholine on physical and psychomotor performance - PubMed (nih.gov) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4595381/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4595381/#:~:text=Acute%20supplementation%20with%20caffeine%20or,large%20individual%20variability%20between%20subjects. https://academic.oup.com/ajcn/article/110/6/1416/5540729 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8235064/

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Contrast Pairs Drive the Empirical Performance of Contrast Consistent Search (CCS), published by Scott Emmons on May 31, 2023 on The AI Alignment Forum. tl;dr Contrast consistent search (CCS) is a method by Burns et al. that consists of two parts: Generate contrast pairs by adding pseudolabels to an unlabelled dataset. Use the contrast pairs to search for a direction in representation space that satisfies logical consistency properties. In discussions with other researchers, I've repeatedly heard (2) as the explanation for how CCS works; I've heard almost no mention of (1). In this post, I want to emphasize that the contrast pairs drive almost all of the empirical performance in Burns et al. Once we have the contrast pairs, standard unsupervised learning methods attain comparable performance to the new CCS loss function. In the paper, Burns et al. do a nice job comparing the CCS loss function to different alternatives. The simplest such alternative runs principal component analysis (PCA) on contrast pair differences, and then it uses the top principal component as a classifier. Another alternative runs linear discriminant analysis (LDA) on contrast pair differences. These alternatives attain 97% and 98% of CCS's accuracy! "[R]epresentations of truth tend to be salient in models: ... they can often be found by taking the top principal component of a slightly modified representation space," Burns et al. write in the introduction. If I understand this statement correctly, it's saying the same thing I want to emphasize in this post: the contrast pairs are what allow Burns et al. to find representations of truth. Empirically, once we have the representations of contrast pair differences, their variance points in the direction of truth. The new logical consistency loss in CCS isn't needed for good empirical performance. Notation We'll follow the notation of the CCS paper. Assume we are given a data set {x1,x2,.,xn} and a feature extractor ϕ(), such as the hidden state of a pretrained language model. First, we will construct a contrast pair for each datapoint xi. We add “label: positive” and “label: negative” to each xi. This gives contrast pairs of the form (x+i,x−i). Now, we consider the set {x+1,x+2,.,x+n} of positive pseudo-labels and {x−1,x−2,.,x−n} of negative pseudo-labels. Because all of the x+i have "label: positive" and all of the x−i have "label: negative", we normalize the positive pseudo-labels and the negative pseudo-labels separately: Here, μ+ and μ− are the element-wise means of the positive and negative pseudo-label sets, respectively. Similarly, σ+ and σ− are the element-wise standard deviations. The goal of this normalization is to remove the embedding of "label: positive" from all the positive pseudo-labels (and "label: negative" from all the negative pseudo-labels). The hope is that by construction, the only difference between ~ϕ(x+i) and ~ϕ(x−i) is that one is true while the other is false. CCS is one way to extract the information about true and false. As we'll discuss more below, doing PCA or LDA on the set of differences {~ϕ(x+i)−~ϕ(x−i)}ni=1 works almost as well. Concept Embeddings in Prior Work In order to better understand contrast pairs, I think it's helpful to review this famous paper by Bolukbasi et al., 2016: "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings." Quoting from Bolukbasi et al.: −−−man−−−−−−woman≈−−−king−−−−−queen Vector differences between words in embeddings have been shown to represent relationships between words. For example given an analogy puzzle, "man is to king as woman is to x" (denoted as man:king :: woman:x), simple arithmetic of the embedding vectors finds that x=queen is the best answer because: Similarly, x=Japan is returned for Paris:France :: Tokyo:x. It is surprising that a simple ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Contrast Pairs Drive the Empirical Performance of Contrast Consistent Search (CCS), published by Scott Emmons on May 31, 2023 on LessWrong. tl;dr Contrast consistent search (CCS) is a method by Burns et al. that consists of two parts: Generate contrast pairs by adding pseudolabels to an unlabelled dataset. Use the contrast pairs to search for a direction in representation space that satisfies logical consistency properties. In discussions with other researchers, I've repeatedly heard (2) as the explanation for how CCS works; I've heard almost no mention of (1). In this post, I want to emphasize that the contrast pairs drive almost all of the empirical performance in Burns et al. Once we have the contrast pairs, standard unsupervised learning methods attain comparable performance to the new CCS loss function. In the paper, Burns et al. do a nice job comparing the CCS loss function to different alternatives. The simplest such alternative runs principal component analysis (PCA) on contrast pair differences, and then it uses the top principal component as a classifier. Another alternative runs linear discriminant analysis (LDA) on contrast pair differences. These alternatives attain 97% and 98% of CCS's accuracy! "[R]epresentations of truth tend to be salient in models: ... they can often be found by taking the top principal component of a slightly modified representation space," Burns et al. write in the introduction. If I understand this statement correctly, it's saying the same thing I want to emphasize in this post: the contrast pairs are what allow Burns et al. to find representations of truth. Empirically, once we have the representations of contrast pair differences, their variance points in the direction of truth. The new logical consistency loss in CCS isn't needed for good empirical performance. Notation We'll follow the notation of the CCS paper. Assume we are given a data set {x1,x2,.,xn} and a feature extractor ϕ(), such as the hidden state of a pretrained language model. First, we will construct a contrast pair for each datapoint xi. We add “label: positive” and “label: negative” to each xi. This gives contrast pairs of the form (x+i,x−i). Now, we consider the set {x+1,x+2,.,x+n} of positive pseudo-labels and {x−1,x−2,.,x−n} of negative pseudo-labels. Because all of the x+i have "label: positive" and all of the x−i have "label: negative", we normalize the positive pseudo-labels and the negative pseudo-labels separately: Here, μ+ and μ− are the element-wise means of the positive and negative pseudo-label sets, respectively. Similarly, σ+ and σ− are the element-wise standard deviations. The goal of this normalization is to remove the embedding of "label: positive" from all the positive pseudo-labels (and "label: negative" from all the negative pseudo-labels). The hope is that by construction, the only difference between ~ϕ(x+i) and ~ϕ(x−i) is that one is true while the other is false. CCS is one way to extract the information about true and false. As we'll discuss more below, doing PCA or LDA on the set of differences {~ϕ(x+i)−~ϕ(x−i)}ni=1 works almost as well. Concept Embeddings in Prior Work In order to better understand contrast pairs, I think it's helpful to review this famous paper by Bolukbasi et al., 2016: "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings." Quoting from Bolukbasi et al.: −−−man−−−−−−woman≈−−−king−−−−−queen Vector differences between words in embeddings have been shown to represent relationships between words. For example given an analogy puzzle, "man is to king as woman is to x" (denoted as man:king :: woman:x), simple arithmetic of the embedding vectors finds that x=queen is the best answer because: Similarly, x=Japan is returned for Paris:France :: Tokyo:x. It is surprising that a simple vector arithm...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Contrast Pairs Drive the Empirical Performance of Contrast Consistent Search (CCS), published by Scott Emmons on May 31, 2023 on LessWrong. tl;dr Contrast consistent search (CCS) is a method by Burns et al. that consists of two parts: Generate contrast pairs by adding pseudolabels to an unlabelled dataset. Use the contrast pairs to search for a direction in representation space that satisfies logical consistency properties. In discussions with other researchers, I've repeatedly heard (2) as the explanation for how CCS works; I've heard almost no mention of (1). In this post, I want to emphasize that the contrast pairs drive almost all of the empirical performance in Burns et al. Once we have the contrast pairs, standard unsupervised learning methods attain comparable performance to the new CCS loss function. In the paper, Burns et al. do a nice job comparing the CCS loss function to different alternatives. The simplest such alternative runs principal component analysis (PCA) on contrast pair differences, and then it uses the top principal component as a classifier. Another alternative runs linear discriminant analysis (LDA) on contrast pair differences. These alternatives attain 97% and 98% of CCS's accuracy! "[R]epresentations of truth tend to be salient in models: ... they can often be found by taking the top principal component of a slightly modified representation space," Burns et al. write in the introduction. If I understand this statement correctly, it's saying the same thing I want to emphasize in this post: the contrast pairs are what allow Burns et al. to find representations of truth. Empirically, once we have the representations of contrast pair differences, their variance points in the direction of truth. The new logical consistency loss in CCS isn't needed for good empirical performance. Notation We'll follow the notation of the CCS paper. Assume we are given a data set {x1,x2,.,xn} and a feature extractor ϕ(), such as the hidden state of a pretrained language model. First, we will construct a contrast pair for each datapoint xi. We add “label: positive” and “label: negative” to each xi. This gives contrast pairs of the form (x+i,x−i). Now, we consider the set {x+1,x+2,.,x+n} of positive pseudo-labels and {x−1,x−2,.,x−n} of negative pseudo-labels. Because all of the x+i have "label: positive" and all of the x−i have "label: negative", we normalize the positive pseudo-labels and the negative pseudo-labels separately: Here, μ+ and μ− are the element-wise means of the positive and negative pseudo-label sets, respectively. Similarly, σ+ and σ− are the element-wise standard deviations. The goal of this normalization is to remove the embedding of "label: positive" from all the positive pseudo-labels (and "label: negative" from all the negative pseudo-labels). The hope is that by construction, the only difference between ~ϕ(x+i) and ~ϕ(x−i) is that one is true while the other is false. CCS is one way to extract the information about true and false. As we'll discuss more below, doing PCA or LDA on the set of differences {~ϕ(x+i)−~ϕ(x−i)}ni=1 works almost as well. Concept Embeddings in Prior Work In order to better understand contrast pairs, I think it's helpful to review this famous paper by Bolukbasi et al., 2016: "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings." Quoting from Bolukbasi et al.: −−−man−−−−−−woman≈−−−king−−−−−queen Vector differences between words in embeddings have been shown to represent relationships between words. For example given an analogy puzzle, "man is to king as woman is to x" (denoted as man:king :: woman:x), simple arithmetic of the embedding vectors finds that x=queen is the best answer because: Similarly, x=Japan is returned for Paris:France :: Tokyo:x. It is surprising that a simple vector arithm...

I can't believe we're in Season 4 of this podcast! Huge thank you to everyone who has tuned in over the past few years. Please continue to like and share this podcast! Also, be sure to follow our homeschooling journey on Instagram @livingthepurelifestyle --- Support this podcast: https://podcasters.spotify.com/pod/show/living-the-pure-lifestyle/support

Read along with us! Be sure to follow our homeschool journey on IG @livingthepurelifestyle --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app Support this podcast: https://anchor.fm/living-the-pure-lifestyle/support

The Cabinet welcomes Mrs. Julie Scott Emmons talking about her pro-life mission and political banter with the Cabinet. --- Send in a voice message: https://anchor.fm/thecarolinacabinet/message

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Fund for Alignment Research (We're Hiring!), published by AdamGleave on July 6, 2022 on The Effective Altruism Forum. Cross-posted to LessWrong The Fund for Alignment Research (FAR) is hiring research engineers and communication specialists to work closely with AI safety researchers. We believe these roles are high-impact, contributing to some of the most interesting research agendas in safety. We also think they offer an excellent opportunity to build skills and connections via mentorship and working closely with researchers at a variety of labs. We were inspired to start FAR after noticing that many academic and independent researchers wanted to work with more collaborators, but often lacked the institutional framework and hiring pipeline to do so. On the other hand, there are a large number of talented people who would like to contribute to AI safety, but lack a stepping stone into the field. Our hope is that FAR can help fill this gap, both directly accelerating valuable research and helping to address the talent pipeline. In the remainder of this post we'll give a bit more information about FAR and our current collaborators, and then summarize our current openings. Please consider applying or forwarding them to a friend who might be interested! We are also actively brainstorming other ways that FAR could be useful to the community. If you have any ideas, we'd love to hear from you! About Us FAR is a non-profit led by Ethan Perez, Adam Gleave, Scott Emmons, and Claudia Shi: a group of AI safety researchers looking to reduce existential risk from artificial intelligence. Ethan recently graduated from the PhD program at New York University, Adam and Scott are PhD candidates at UC Berkeley, and Claudia is a PhD candidate at Columbia University. FAR provides services to AI safety researchers to accelerate their research agendas. We are currently focused on supporting the agendas of Ethan, Adam, Scott, and Claudia. We are also trialing a collaboration with the labs of David Kruger and Jacob Steinhardt, professors at the University of Cambridge and UC Berkeley. Our services are currently provided free-of-charge to recipients out of FAR's general support funding. In the future we plan to charge partners who use large quantities of our services on an at-cost basis. This could be paid for from a partner's existing grant, or we can also assist the partner with fundraising for this purpose. We anticipate supporting many of the same people that BERI currently works with. However, our organisations have differing emphases. First, our core services are different: to a first approximation, BERI provides "operations as a service" whereas FAR provides "a technical team as a service". That is, FAR recruits, manages and trains our own team; whereas BERI hires people primarily at the request of their partner's. Second, FAR works primarily with individual researchers whereas BERI works primarily with entire labs, although this distinction may blur in the future. Finally, FAR is more opinionated than BERI: if we have more demand for our services than our team can support, then we will prioritize based on our internal view of which agendas are most promising. Although FAR is a new organization, our research has already led to a method for learning from language feedback as a data-efficient alternative to RL from human feedback. We have analyzed challenges associated with treating a language model as an RL policy and launched a competition on inverse scaling for language models. We are currently pursuing several other early-stage AI safety projects. Once we have beta-tested our model, we plan to expand the number of partners we work with. Feel free to get in touch at hello@alignmentfund.org if you think you might be a good fit! Operations Manager We are seeking an Operat...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing the Fund for Alignment Research (We're Hiring!), published by AdamGleave on July 6, 2022 on The AI Alignment Forum. Cross-posted to the EA Forum The Fund for Alignment Research (FAR) is hiring research engineers and communication specialists to work closely with AI safety researchers. We believe these roles are high-impact, contributing to some of the most interesting research agendas in safety. We also think they offer an excellent opportunity to build skills and connections via mentorship and working closely with researchers at a variety of labs. We were inspired to start FAR after noticing that many academic and independent researchers wanted to work with more collaborators, but often lacked the institutional framework and hiring pipeline to do so. On the other hand, there are a large number of talented people who would like to contribute to AI safety, but lack a stepping stone into the field. Our hope is that FAR can help fill this gap, both directly accelerating valuable research and helping to address the talent pipeline. In the remainder of this post we'll give a bit more information about FAR and our current collaborators, and then summarize our current openings. Please consider applying or forwarding them to a friend who might be interested! We are also actively brainstorming other ways that FAR could be useful to the community. If you have any ideas, we'd love to hear from you! About Us FAR is a non-profit led by Ethan Perez, Adam Gleave, Scott Emmons, and Claudia Shi: a group of AI safety researchers looking to reduce existential risk from artificial intelligence. Ethan recently graduated from the PhD program at New York University, Adam and Scott are PhD candidates at UC Berkeley, and Claudia is a PhD candidate at Columbia University. FAR provides services to AI safety researchers to accelerate their research agendas. We are currently focused on supporting the agendas of Ethan, Adam, Scott, and Claudia. We are also trialing a collaboration with the labs of David Kruger and Jacob Steinhardt, professors at the University of Cambridge and UC Berkeley. Our services are currently provided free-of-charge to recipients out of FAR's general support funding. In the future we plan to charge partners who use large quantities of our services on an at-cost basis. This could be paid for from a partner's existing grant, or we can also assist the partner with fundraising for this purpose. We anticipate supporting many of the same people that BERI currently works with. However, our organisations have differing emphases. First, our core services are different: to a first approximation, BERI provides "operations as a service" whereas FAR provides "a technical team as a service". That is, FAR recruits, manages and trains our own team; whereas BERI hires people primarily at the request of their partner's. Second, FAR works primarily with individual researchers whereas BERI works primarily with entire labs, although this distinction may blur in the future. Finally, FAR is more opinionated than BERI: if we have more demand for our services than our team can support, then we will prioritize based on our internal view of which agendas are most promising. Although FAR is a new organization, our research has already led to a method for learning from language feedback as a data-efficient alternative to RL from human feedback. We have analyzed challenges associated with treating a language model as an RL policy and launched a competition on inverse scaling for language models. We are currently pursuing several other early-stage AI safety projects. Once we have beta-tested our model, we plan to expand the number of partners we work with. Feel free to get in touch at hello@alignmentfund.org if you think you might be a good fit! Operations Manager We are seeking an Operation...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introduction to Pragmatic AI Safety [Pragmatic AI Safety #1], published by ThomasWoodside on May 9, 2022 on The Effective Altruism Forum. This is the introduction to a sequence of posts that describe our models for Pragmatic AI Safety. Thanks to Oliver Zhang, Mantas Mazeika, Scott Emmons, Neel Nanda, Cameron Berg, and Michael Chen for feedback on this sequence. Machine learning has been outpacing safety. Ten years ago, AlexNet pushed the boundaries of machine learning, and it was trained using only two GPUs. Now state-of-the-art models are trained on thousands of GPUs. GPT-2 was released only around three years ago, and today, we have models capable of answering bar exam questions, writing code, and explaining jokes. Meanwhile, existing approaches to AI safety have not seen similar strides. Many older approaches are still pre-paradigmatic, uncertain about what concrete research directions should be pursued and still aiming to get their bearings. Centered on math and theory, this research focuses on studying strictly futuristic risks that result from potential systems. Unfortunately, not much progress has been made, and deep learning resists the precise and universal mathematical characterizations preferred by some safety approaches. Recently, some established safety teams have focused more on safety in the context of deep learning systems, which has the benefit of being more concrete and having faster experimental feedback loops. However, many approaches often exhibit the downside of blurring the lines between general capabilities research and safety, as there appear to be few other options. Finally, neither the pre-paradigmatic nor industry deep learning-based approaches seriously emphasize the broad range of sociotechnical factors that are critical for reducing risk from AI systems. Given that ML is progressing quickly, that pre-paradigmatic research is not highly scalable to many researchers, and that safety research that advances capabilities is not safely scalable to a broader research community, we suggest an approach that some of us have been developing in academia over the past several years. We propose a simple, underrated, and complementary research paradigm, which we call Pragmatic AI Safety (PAIS). By complementary, we mean that we intend for it to stand alongside current approaches, rather than replace them. Pragmatic AI Safety rests on three essential pillars: ML research precedents. Safety involves technical AI problems, and the ML community's precedents enable it to be unusually effective at solving technical AI problems. Minimal capabilities externalities. Safety research at scale needs to be precautious and avoid advancing capabilities in the name of safety. Sociotechnical systems view. Preventing catastrophes requires more than technical work, such as improving incentives, safety culture, protocols, and so on. ML Research Precedents Despite relying on “broken” processes like conferences and citations, the ML community has managed to solve an increasingly general set of problems: colorizing images, protein folding, superhuman poker, art generation, etc. This doesn't mean that the ML community is set up optimally (we will discuss ways in which it's not), but it does consistently exceed our expectations and demonstrate the best track record in solving technical AI problems. In general, ML researchers are skilled at adding arbitrary features to systems to improve capabilities, and many aspects of safety could be operationalized so as to be similarly improved. This property makes ML research precedents promising for solving technical ML problems, including many safety problems. Here are some ML research precedents that we view as important: Long term goals are broken down into empirical simplified microcosmic problems Subproblems can be worked on iterat...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introduction to Pragmatic AI Safety [Pragmatic AI Safety #1], published by Dan Hendrycks on May 9, 2022 on The AI Alignment Forum. This is the introduction to a sequence of posts that describe our models for Pragmatic AI Safety. Thanks to Oliver Zhang, Mantas Mazeika, Scott Emmons, Neel Nanda, Cameron Berg, and Michael Chen for feedback on this sequence. Machine learning has been outpacing safety. Ten years ago, AlexNet pushed the boundaries of machine learning, and it was trained using only two GPUs. Now state-of-the-art models are trained on thousands of GPUs. GPT-2 was released only around three years ago, and today, we have models capable of answering bar exam questions, writing code, and explaining jokes. Meanwhile, existing approaches to AI safety have not seen similar strides. Many older approaches are still pre-paradigmatic, uncertain about what concrete research directions should be pursued and still aiming to get their bearings. Centered on math and theory, this research focuses on studying strictly futuristic risks that result from potential systems. Unfortunately, not much progress has been made, and deep learning resists the precise and universal mathematical characterizations preferred by some safety approaches. Recently, some established safety teams have focused more on safety in the context of deep learning systems, which has the benefit of being more concrete and having faster experimental feedback loops. However, many approaches often exhibit the downside of blurring the lines between general capabilities research and safety, as there appear to be few other options. Finally, neither the pre-paradigmatic nor industry deep learning-based approaches seriously emphasize the broad range of sociotechnical factors that are critical for reducing risk from AI systems. Given that ML is progressing quickly, that pre-paradigmatic research is not highly scalable to many researchers, and that safety research that advances capabilities is not safely scalable to a broader research community, we suggest an approach that some of us have been developing in academia over the past several years. We propose a simple, underrated, and complementary research paradigm, which we call Pragmatic AI Safety (PAIS). By complementary, we mean that we intend for it to stand alongside current approaches, rather than replace them. Pragmatic AI Safety rests on three essential pillars: ML research precedents. Safety involves technical AI problems, and the ML community's precedents enable it to be unusually effective at solving technical AI problems. Minimal capabilities externalities. Safety research at scale needs to be precautious and avoid advancing capabilities in the name of safety. Sociotechnical systems view. Preventing catastrophes requires more than technical work, such as improving incentives, safety culture, protocols, and so on. ML Research Precedents Despite relying on “broken” processes like conferences and citations, the ML community has managed to solve an increasingly general set of problems: colorizing images, protein folding, superhuman poker, art generation, etc. This doesn't mean that the ML community is set up optimally (we will discuss ways in which it's not), but it does consistently exceed our expectations and demonstrate the best track record in solving technical AI problems. In general, ML researchers are skilled at adding arbitrary features to systems to improve capabilities, and many aspects of safety could be operationalized so as to be similarly improved. This property makes ML research precedents promising for solving technical ML problems, including many safety problems. Here are some ML research precedents that we view as important: Long term goals are broken down into empirical simplified microcosmic problems Subproblems can be worked on iteratively, ...

This is a trailer for our latest series of episodes for Friends of the Podcast. To subscribe, go to thecyclingpodcast.com The Story of The Cycling Podcast is a trilogy of episodes commissioned and edited by three of our subscribers. In part one, How It Started, Jack Mckillop asks Richard Moore, Lionel Birnie and Daniel Friebe about the origins of the podcast and how it has developed over the past nine years. Part two, by Scott Emmons, is called How the Sausage is Made, and goes behind the scenes to ask our audio producers how the podcast is put together. Finally, part three by Nick Busca is A Day on the Road and tells the story of a typical day at the grand tours through the prism of one of the most dramatic days in recent Tour de France history. Richard and Lionel recall the stage of the 2016 Tour when Chris Froome was seen running up Mont Ventoux without his bike. 

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What counts as defection?, published by Alex Turner on the AI Alignment Forum. Thanks to Michael Dennis for proposing the formal definition; to Andrew Critch for pointing me in this direction; to Abram Demski for proposing non-negative weighting; and to Alex Appel, Scott Emmons, Evan Hubinger, philh, Rohin Shah, and Carroll Wainwright for their feedback and ideas. There's a good chance I'd like to publish this at some point as part of a larger work. However, I wanted to make the work available now, in case that doesn't happen soon. They can't prove the conspiracy... But they could, if Steve runs his mouth. The police chief stares at you. You stare at the table. You'd agreed (sworn!) to stay quiet. You'd even studied game theory together. But, you hadn't understood what an extra year of jail meant. The police chief stares at you. Let Steve be the gullible idealist. You have a family waiting for you. Sunlight stretches across the valley, dappling the grass and warming your bow. Your hand anxiously runs along the bowstring. A distant figure darts between trees, and your stomach rumbles. The day is near spent. The stags run strong and free in this land. Carla should meet you there. Shouldn't she? Who wants to live like a beggar, subsisting on scraps of lean rabbit meat? In your mind's eye, you reach the stags, alone. You find one, and your arrow pierces its barrow. The beast shoots away; the rest of the herd follows. You slump against the tree, exhausted, and never open your eyes again. You can't risk it. People talk about 'defection' in social dilemma games, from the prisoner's dilemma to stag hunt to chicken. In the tragedy of the commons, we talk about defection. The concept has become a regular part of LessWrong discourse. Informal definition. A player defects when they increase their personal payoff at the expense of the group. This informal definition is no secret, being echoed from the ancient Formal Models of Dilemmas in Social Decision-Making to the recent Classifying games like the Prisoner's Dilemma: you can model the "defect" action as "take some value for yourself, but destroy value in the process". Given that the prisoner's dilemma is the bread and butter of game theory and of many parts of economics, evolutionary biology, and psychology, you might think that someone had already formalized this. However, to my knowledge, no one has. Formalism Consider a finite n -player normal-form game, with player i having pure action set A i and payoff function P i A 1 × × A n → R . Each player i chooses a strategy s i ∈ Δ A i (a distribution over A i ). Together, the strategies form a strategy profile s s 1 s n s − i s 1 s i − 1 s i 1 s n is the strategy profile, excluding player i 's strategy. A payoff profile contains the payoffs for all players under a given strategy profile. A utility weighting α j j 1 n is a set of n non-negative weights (as in Harsanyi's utilitarian theorem). You can consider the weights as quantifying each player's contribution; they might represent a percieved social agreement or be the explicit result of a bargaining process. When all α j are equal, we'll call that an equal weighting. However, if there are "utility monsters", we can downweight them accordingly. We're implicitly assuming that payoffs are comparable across players. We want to investigate: given a utility weighting, which actions are defections? Definition. Player i 's action a ∈ A i is a defection against strategy profile s and weighting α j j 1 n if Personal gain: P i a s − i P i s i s − i Social loss: ∑ j α j P j a s − i ∑ j α j P j s i s − i If such an action exists for some player i , strategy profile s , and weighting, then we say that there is an opportunity for defection in the game. Remark. For an equal weighting, condition (2) is equivalent to demanding that the action n...

For Episode 21 I spoke with someone who's been an animated head more times than he can probably count: Paul Hanges is CEO of JibJab, which is famous for its personalized e-cards and satire animations that lets your head be the star! If you haven’t heard of JibJab you may have had YOUR head in the sand; they’re the OG of digital branded content. It was born in 1999 to brothers Evan and Gregg Spiridellis, perhaps best recalled from their 2004 glory days of being featured on everything from The Tonight Show to ABC News for then viral political satire, “This Land.” Hanges, who was promoted from COO 18 months ago, says they’re proud of being dubbed the “original online cockroach” for their longevity and survival of dot-com and economy busts. In our conversation he explains why JibJab is still hot more than 20 years since its inception. They’re even having something of a renaissance with the resurrection of their trademark Year in Review video after a six-year pause. (But how could you NOT do a recap of a year like 2020? For Chief Creative Officer Mauro Gatti and lyricist Scott Emmons it probably almost wrote itself! And I’m proud to offer my big head/floppy jaw cut here for your amusement by way of example!) Hanges also explains the appeal of low-tech animation and their trademark “big heads and floppy jaws,” how they survive as a subscription model and why they walk away from brand dollars if not aligned with their mission – “to make billions of people happy by allowing them to be funny, wherever they're having that conversation.” Here are the highlights – but do hear all Hanges has to say in this very “animated” conversation: How it all started – yes in the proverbial garage Why, in a world of “deep fakes and augmentation” the appeal of JibJab is the personalization and NOT to replace reality. “We want to provide utility to help people say happy birthday or anniversary or other big moments.” How personal micro networks add up to eyeballs at scale – and 1.3 million paying subscribers; How they’ll leverage the rights to hits like “Old Town” or Mamma Mia the musical, but why they’ll walk away from 95% ofbranded opportunities; Why politics and JibJab do not always make good bedfellows, but why they participated in the Facebook ad boycott in July to stand up to divisiveness; How (self-plug here) they balance “brand+demand,” -- as we promote in The Continuum; The big data opportunity they’re sitting on to be a personalized service for people and how they’ve had to be nimble and “pivot” in their production and platform access. And of course, for the last episode of 2020 we have to offer Hanges’ words of wisdom from the past full year as CEO to other new CEOs...and how to embrace your strengths and those of others as well. “I take a step back and look and say, ‘we have a very strong operating team that's working towards a goal of making billions of people happy.’ ...And I've been really proud to say I can lead this company with that mission.” Personally, I could not have wrapped up my first year of Insider Interviews podcasts (AND my “It’s Quite a Living” personal podcast) with a better message than to “head” into 2021 with the inspiration of a JibJab to find the humor and spread the joy throughout this holiday season and into 2021.  Thank you all for listening and hopefully sharing this podcast. I value your feedback and support. And if I can help you create content that spreads joy or opens revenue doors please visit Moss Appeal or write to me at podcasts@mossappeal.com. Happy New Year!

Bringing his visionary outlook on the future of retail and wealth of knowledge and research in omni channel software and his expertise in innovation technology to his role at Powerfront, Scott Emmons offers insight into a real-time, personalized retailing platform that mirrors in-store luxury-level customer service online. He leverages his expertise to build and circulate ideas and design products that are at the forefront of customer service software. Scott’s motivation to help make magic for customers translates perfectly to Powerfront’s core mission, and his extensive involvement as a Powerfront customer have made him the perfect ambassador. See omnystudio.com/listener for privacy information.

In this episode hear from former head of Innovation Labs at Neiman Marcus, Scott Emmons as he discusses the importance of customer care and how to give a digital white glove treatment.

By pure chance, veteran communications pro, Scott Emmons became our 1000th Instagram follower a month ago. We're humbled that anyone would want to follow us, but citing Scott's portfolio, we were particularly thrilled. Our GDPR privacy policy was updated on August 8, 2022. Visit acast.com/privacy for more information.

The story boys want to tell the story of the biggest dinosaur!! --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app Support this podcast: https://anchor.fm/hope-newkirt/support

Over the past few years, the film industry in Milwaukee has grown tremendously. Independent producers and production houses have sprung up throughout the area. In fact, there are more people employed in the film industry here today than are working in the business that made Milwaukee famous – beer. A key factor in the film industries rapid growth is UWM, which has one of the top film study programs in the world. On this edition of UWM Today, we focus on film. Joining us in the studio is Scott Emmons, dean of the Peck School of the Arts, and Jesse McLean, associate professor and director of the film graduate program at the university.

Scott Emmons, CIO, Memomi Labs and Wayne Arthur, CEO, SkyFii Limited, share their views on Retail during the holiday season, their experiences with retailers finding the retail edge & using technology to transform the customer experience

Scott Emmons, CIO, Memomi Labs and Wayne Arthur, CEO, SkyFii Limited, share their views on Retail innovation, the merging of online/offline Retail worlds & leveraging technology to transform the customer experience

The nervous system is a complicated network of specialised cells - neurons - that transfer information from one part of the body to another. To help our understanding of the nervous system in humans, scientists have mapped every neuron and every connection in a type of roundworm called C. elegans. Emma Hildyard asked Scott Emmons how this map was created and what it could mean... Like this podcast? Please help us by supporting the Naked Scientists

The nervous system is a complicated network of specialised cells - neurons - that transfer information from one part of the body to another. To help our understanding of the nervous system in humans, scientists have mapped every neuron and every connection in a type of roundworm called C. elegans. Emma Hildyard asked Scott Emmons how this map was created and what it could mean... Like this podcast? Please help us by supporting the Naked Scientists

Are digital experiences the gateway to a more accessible luxury experience for millennial consumers? Scott Emmons opens up to us about his successes at Neiman Marcus, the future of technology adoption at retail brands, and how he's bringing that culture of innovation and technology leadership to other brands in his new role at Current Global.

Internal teams can no longer deliver the results needed to drive the industry forward, says Scott Emmons, departing head of the Neiman Marcus iLab, on the latest episode of the Innovators podcast. Responsible for setting up one of the most established retail innovation programs in the world, Emmons is now bringing his insights and expertise to TheCurrent Global as he joins as the company as the chief technology officer. He joins at a time where he believes internal labs should be replaced by a more open approach to innovation, where collaboration is key. "You've got to build better partnerships that go beyond the four walls of the retailer. If everything happens within those four walls then what you keep doing is the same thing over and over again," says Emmons, who launched the lab in 2012. "Because you're not bringing in fresh ideas, you're not bringing in fresh approaches to retail. You continue to iterate the things which you're good at." During his time at Neiman Marcus, Emmons was responsible for introducing innovative technologies to its stores such as smart mirrors, new fitting room technology, 4K touch table lookbooks and a clienteling tool that better links a customer's online to offline behavior, while arming associates with the tools to better serve them 1-2-1. Speaking to CEO Liz Bacelar, Emmons outlines why innovation executives have their hands tied and how innovation is often stalled by internal culture. They also discuss a solution to unlock rapid change in retail.

Neiman Marcus has learned a thing or two over its 109-year history — and one thing is for sure: The company must anticipate future customer needs if they want to stay on the cutting edge of retail. Scott Emmons, head of the innovation lab, discusses how Neiman Marcus does it while staying true to its origins. Learn more at retailgetsreal.com.

Technology innovation is driving change and transformation in retail. On this episode, CXOTalk host Michael Krigsman talks with Scott Emmons, founder and head of the Innovation Lab at major retailer Neiman Marcus. Top digital transformation analyst and expert, Brian Solis, is guest co-host on this show.

Technology innovation is driving change and transformation in retail. On this episode, CXOTalk host Michael Krigsman talks with Scott Emmons, founder and head of the Innovation Lab at major retailer Neiman Marcus. Top digital transformation analyst and expert, Brian Solis, is guest co-host on this show.

Recorded at Online Retailer 2016 in Sydney, here we chat with Scott Emmons, Head of Innovation Lab, Neiman Marcus about creating the store of the future through integrating technology to enhance in-store experience. • Understand when, how, and what customers want – stay on top of the game • Infuse ideas and innovation into your business with customer-focused technologies – beacons, memory mirror and others • Use imagination as your only boundary to follow – allow to fail and learn from it

Scott Emmons is the Head of Digital Innovation Lab for the Neiman Marcus Group. Scott built the Neiman Marcus Group Innovation Lab, also known as iLab, and is responsible for leading the organization in evaluating, designing, testing and piloting cutting-edge technologies and applications for luxury retail. Scott is working on BYOD (Bring Your Own Device) initiatives, as well as Wi-Fi/Local and Wide Area Network infrastructures, mobility, and digital signage initiatives. These innovation projects include the amazing MemoMi Memory Mirror, beautiful 4K touch table look-books, beacon enabled holiday passes, intelligent mobile phone charging stations and gorgeous digital directories for Bergdorf Goodman. Here are the highlights of my conversation with our guest: Getting to know Scott --- what usually consumes his day and how a Harley Davison and a pool fills the little free time he has left from time to time. From starting out as a contractor, to becoming an associate and then an Enterprise Architect, Scott shares the path that led him to head the Digital Innovation Lab at Neiman Marcus. Scott shares how exposure, being a nerd, his love of having a good problem and being an early adapter had inspired him to be in field of innovation. The biggest challenge he is seeing in the retail industry: understanding what the next generation of customer looks like and their preferences and trying to build something for them. Giving the sales associates iPhones and how this huge change of strategy impacted their business as communications improved between associate and customer transforming interactions. Scott also discusses why they opted for iOS rather than Android. Scott’s definition of an omni-channel approach which is basically, not thinking about individual channels but where the customer is at and where the customer wants to be, and then providing that channel. The channels where the biggest of their revenues come from and the pain points which Scott feels that the industry is having challenges catching up to. Innovation is important and Scott emphasizes how equally important it is to collaborate with his associates across the business and get out of the store to see what the customer is doing versus staying in the lab to provide the solution for the real customer problems. His advice for those who are thinking of moving to digital adoption and the initial actions that they should focus their early efforts on. Why Scott thinks that augment and virtual reality is the most realistic next hot thing and how it integrates and ties in the field of retail. The processes they go through, from funneling of idea to requesting funding for full roll out, in implementing innovative experiments and projects and the metrics associated for them to gauge ROI. Scott’s projected breakthroughs and the things he thinks have promising potentials. Rapid Fire Questions: Would you put more emphasis on the idea or the execution? How would you weigh each of them and why? I would not put more emphasis on either but the idea has to be good to be worth doing and the execution has to be flawless. It has to be a 50/50 weight for this to work in my world. What is your biggest learning lesson on your journey so far? What the customer wants is what’s the most important thing and the only way you are going to learn that is to get out in the stores and interact with the customers and associates. What is your favorite business book? Innovation Engine by Jatin Desai What is your favorite mobile app resource? Flipboard What is your favorite app and why? Quartz What is the coolest thing that you are working on right now that you want everyone to know about? You’ll know about it in six weeks or so.

Scott Emmons, Head of Innovation Lab at Neiman Marcus, sits down with us to talk about the future of retail, augmented reality, magic mirrors, and why Conversational Commerce is just the beginning.

For those of you who don’t enjoy wandering the aisles of the grocery store in search of soy sauce, or mailing back endless pairs of ill-fitting shoes bought online, we have good news for you: Shopping in the future is going to be so much easier than it is today. Between smartphones and tracking technologies, every trip to the store will be quick, efficient, and a heck of a lot smarter than it is today. Some retailers will even be able to anticipate your needs and take care of them for you. To get the inside scoop, we talk to Indiana University’s Ray Burke, who studies how customers think and behave. And we hear from Scott Emmons, who heads up innovation for Neiman Marcus, about the high-tech mirrors and tablets they’re bringing to their stores. And looking further into the future, we talked to the people who are actually making it happen: Devora Rogers and David Mounts at a tech company called Inmar. They are pretty excited about what technology could do for stores and the people who frequent them. And don’t forget our ever-entertaining Popular Science archives! The futuristic delivery methods we envisioned back in 1939 look a little different from the drones of today. That said, stores aren’t going away. They’re just going to appear in some really interesting new forms. Futuropolis is a biweekly podcast on the Panoply network. This week's episode is sponsored by Braintree—code for easy online payments. If you're working on a mobile app and need a simple payments solution, check out Braintree. For your first $50,000 in transactions fee-free, go you braintreepayments.com/future. This episode is also sponsored by The Message, a new podcast from GE Podcast Theater. Host Nicki Tomlin follows a team of elite cryptographers as they decode a highly classified radio transmission. To sum it up: extraterrestrials. Check out The Message, on iTunes.