Podcasts about safety research

Val and Faith are joined in the studio by Dr Lauren Pearson, Research Fellow & Equity Lead, Sustainable Mobility and Safety Research at Monash University. Lauren joined us on an earlier program to discuss the Petal Project and her report Unlocking young women's access to bike-riding, relesed late last year. Lauren gives us a a quick recap of the project and its findings about the complexity of travel for young women. You can check out this episode to hear more. Today we loook at the second part of the program which was workshops and interviews with participants and the solutions and interventions particpants themselves came up with for many of the issues raised during the course of research. During the program we also discussed a specific discussion on Late Night Live which you can find here.  

Val and Faith are joined in the studio by Dr Lauren Pearson, Research Fellow & Equity Lead, Sustainable Mobility and Safety Research at Monash University. We share our bike moments before taking a look at some local news. Over the New Year the Road Safety Action Plan Phase 2 was released. Developed by Victoria's Road Safety Partners, the plan includes some subtle changes in language and otherwise that may prove significant for active transport in Melbourne and Victoria.  The Age has included in a recent story on Victoria's road toll over the past 15 years a handy infographic that can be filtered by suburb and shows the impact of t he road toll on different road users, pedestrians, cyclsists and others, over the last. fifteen years. Take a look at Victoria's Road toll by the numbers. Discussion turns to the Petal Project and the report Unlocking young women's access to bike-riding relesed late last year. Lauren explains how the project came about, the complexity of travel for young women, of their choices with repect to different modes and how considerations of these issues hasn't traditionally been a part of the provision of transport infrastrcuture. With so much left to unpack when we run out of time we decide we will need to have Lauren back on the program soon to look at the solutions and interventions particpants in the project came up with for many of the issues raised during the course of research. 

On January 16, 2024, FDNY members experienced an extreme fire event while operating at a residential building fire at 2162 Valentine Avenue in the Bronx. A fire in an apartment on the third floor of a six-story H-type New Law tenement extended into the public hallway, up the interior public stairwell, and blew fifteen feet out the roof bulkhead door.   In this episode, host Battalion Chief Jason Cascone discusses this fascinating operation with Lt. Brian Currid and FF Rob Camaj from Ladder 33–the first due truck– and special guest Dr. Dan Madrzykowski, Senior Director of Research for the Fire Safety Research Institute (FSRI), part of UL Research Institutes.  An analysis of the fire concluded that the extreme fire behavior was caused by a combination of basic fire dynamics and combustible paint in the public hallway. A chain of openings—that included a failed fire-apartment window, the fire-apartment door and the roof-bulkhead door—created a low-intake, high-exhaust flow path. 

Safety is a multi-discipline endeavor. Jessica Cicchino, Senior VP of Research from the IIHS-HLDI, was our guide through the many ways that her organization brings together professionals from a variety of specialties and backgrounds to perform valuable research on the motor vehicles that traverse our roadways and the human behaviors that influence vehicle operation. The IIHS stays on top of the latest technology trends to provide consumers with information for an informed purchase. Sarah and Eric cherished the opportunity to speak with and relate to Jessica and her organization's work because in one way or another we are all working to improve safety for our communities.  Date of Recording: Monday, May 20, 2024 https://www.linkedin.com/in/jessica-cicchino-a3b3005/

Creatine May Have Brain and Muscle Benefits: Initially popular among athletes for muscle growth and performance, creatine is now recognized for its potential positive effects on brain function. Naturally occurring in the body, especially in muscles, supplementation can significantly boost its levels. Enhancing Brain Function: Studies suggest that creatine supplementation increases brain creatine levels, which may counter mental fatigue and enhance brain energy metabolism. It's particularly noted for improving memory, especially in older adults, and may also boost intelligence and reasoning abilities. Specific Benefits and Safety: Research indicates notable benefits of creatine supplementation for older adults, women, and those under metabolic stress. Concerns regarding kidney health have been largely dispelled, with studies suggesting safety in young adults and chronic renal disease patients. Optimal Use and Dosage: The most studied and accessible form of creatine is creatine monohydrate, with a general consensus on a dosage of around 5 grams per day. This dosage is believed to support both muscle and brain wellness effectively.     This content is not intended to diagnose, treat, cure, or prevent disease. The information provided by this video should not be used as individual medical advice. You should always consult your healthcare provider for individual recommendations and treatment.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apply to Aether - Independent LLM Agent Safety Research Group, published by RohanS on August 24, 2024 on The Effective Altruism Forum. The basic idea Aether will be a small group of talented early-career AI safety researchers with a shared research vision who work full-time with mentorship on their best effort at making AI go well. That research vision will broadly revolve around the alignment, control, and evaluation of LLM agents. There is a lot of latent talent in the AI safety space, and this group will hopefully serve as a way to convert some of that talent into directly impactful work and great career capital. Get involved! 1. Submit a short expression of interest here by Fri, Aug 23rd at 11:59pm PT if you would like to contribute to the group as a full-time in-person researcher, part-time / remote collaborator, or advisor. (Note: Short turnaround time!) 2. Apply to join the group here by Sat, Aug 31st at 11:59pm PT. 3. Get in touch with Rohan at rs4126@columbia.edu with any questions. Who are we? Team members so far Rohan Subramani I recently completed my undergrad in CS and Math at Columbia, where I helped run an Effective Altruism group and an AI alignment group. I'm now interning at CHAI. I've done several technical AI safety research projects in the past couple years. I've worked on comparing the expressivities of objective-specification formalisms in RL (at AI Safety Hub Labs, now called LASR Labs), generalizing causal games to better capture safety-relevant properties of agents (in an independent group), corrigibility in partially observable assistance games (my current project at CHAI), and LLM instruction-following generalization (part of an independent research group). I've been thinking about LLM agent safety quite a bit for the past couple of months, and I am now also starting to work on this area as part of my CHAI internship. I think my (moderate) strengths include general intelligence, theoretical research, AI safety takes, and being fairly agentic. A relevant (moderate) weakness of mine is programming. I like indie rock music :). Max Heitmann I hold an undergraduate master's degree (MPhysPhil) in Physics and Philosophy and a postgraduate master's degree (BPhil) in Philosophy from Oxford University. I collaborated with Rohan on the ASH Labs project ( comparing the expressivities of objective-specification formalisms in RL), and have also worked for a short while at the Center for AI Safety (CAIS) under contract as a ghostwriter for the AI Safety, Ethics, and Society textbook. During my two years on the BPhil, I worked on a number of AI safety-relevant projects with Patrick Butlin from FHI. These were focussed on deep learning interpretability, the measurement of beliefs in LLMs, and the emergence of agency in AI systems. In my thesis, I tried to offer a theory of causation grounded in statistical mechanics, and then applied this theory to vindicate the presuppositions of Judea Pearl-style causal modeling and inference. Advisors Erik Jenner and Francis Rhys Ward have said they're happy to at least occasionally provide feedback for this research group. We will continue working to ensure this group receives regular mentorship from experienced researchers with relevant background. We are highly prioritizing working out of an AI safety office because of the informal mentorship benefits this brings. Research agenda We are interested in conducting research on the risks and opportunities for safety posed by LLM agents. LLM agents are goal-directed cognitive architectures powered by one or more large language models (LLMs). The following diagram (taken from On AutoGPT) depicts many of the basic components of LLM agents, such as task decomposition and memory. We think future generations of LLM agents might significantly alter the safety landscape, for two ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Bitter Lesson for AI Safety Research, published by Adam Khoja on August 2, 2024 on The AI Alignment Forum. Read the associated paper "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?": https://arxiv.org/abs/2407.21792 Focus on safety problems that aren't solved with scale. Benchmarks are crucial in ML to operationalize the properties we want models to have (knowledge, reasoning, ethics, calibration, truthfulness, etc.). They act as a criterion to judge the quality of models and drive implicit competition between researchers. "For better or worse, benchmarks shape a field." We performed the largest empirical meta-analysis to date of AI safety benchmarks on dozens of open language models. Around half of the benchmarks we examined had high correlation with upstream general capabilities. Some safety properties improve with scale, while others do not. For the models we tested, benchmarks on human preference alignment, scalable oversight (e.g., QuALITY), truthfulness (TruthfulQA MC1 and TruthfulQA Gen), and static adversarial robustness were highly correlated with upstream general capabilities. Bias, dynamic adversarial robustness, and calibration when not measured with Brier scores had relatively low correlations. Sycophancy and weaponization restriction (WMDP) had significant negative correlations with general capabilities. Often, intuitive arguments from alignment theory are used to guide and prioritize deep learning research priorities. We find these arguments to be poorly predictive of these correlations and are ultimately counterproductive. In fact, in areas like adversarial robustness, some benchmarks basically measured upstream capabilities while others did not. We argue instead that empirical measurement is necessary to determine which safety properties will be naturally achieved by more capable systems, and which safety problems will remain persistent.[1] Abstract arguments from genuinely smart people may be highly "thoughtful," but these arguments generally do not track deep learning phenomena, as deep learning is too often counterintuitive. We provide several recommendations to the research community in light of our analysis: Measure capabilities correlations when proposing new safety evaluations. When creating safety benchmarks, aim to measure phenomena which are less correlated with capabilities. For example, if truthfulness entangles Q/A accuracy, honesty, and calibration - then just make a decorrelated benchmark that measures honesty or calibration. In anticipation of capabilities progress, work on safety problems that are disentangled with capabilities and thus will likely persist in future models (e.g., GPT-5). The ideal is to find training techniques that cause as many safety properties as possible to be entangled with capabilities. Ultimately, safety researchers should prioritize differential safety progress, and should attempt to develop a science of benchmarking that can effectively identify the most important research problems to improve safety relative to the default capabilities trajectory. We're not claiming that safety properties and upstream general capabilities are orthogonal. Some are, some aren't. Safety properties are not a monolith. Weaponization risks increase as upstream general capabilities increase. Jailbreaking robustness isn't strongly correlated with upstream general capabilities. However, if we can isolate less-correlated safety properties in AI systems which are distinct from greater intelligence, these are the research problems safety researchers should most aggressively pursue and allocate resources toward. The other model properties can be left to capabilities researchers. This amounts to a "Bitter Lesson" argument for working on safety issues which are relatively uncorrelated (or negatively correlate...

On this episode of The Digital Patient, Dr. Joshua Liu, Co-founder & CEO of SeamlessMD, and marketing colleague, Alan Sardana, chat with Dr. Rachel McEntee, Chief Medical Information Officer, and Dr. Justin Stinnett-Donnelly, Associate VP, Business Owner for Inpatient and Ancillary Applications at The University of Vermont Health Network, about "Assessing AI Scribes via Crossover Comparison, Why AI needs more Patient Safety Research before use on Clinical Care, How to Sunset Old Tech Processes, and more..."

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ML Safety Research Advice - GabeM, published by Gabe M on July 23, 2024 on The AI Alignment Forum. This is my advice for careers in empirical ML research that might help AI safety (ML Safety). Other ways to improve AI safety, such as through AI governance and strategy, might be more impactful than ML safety research (I generally think they are). Skills can be complementary, so this advice might also help AI governance professionals build technical ML skills. 1. Career Advice 1.1 General Career Guides Preventing an AI-related catastrophe - 80,000 Hours A Survival Guide to a PhD (Andrej Karpathy) How to pursue a career in technical AI alignment - EA Forum AI safety technical research - Career review - 80,000 Hours Beneficial AI Research Career Advice 2. Upskilling 2.1 Fundamental AI Safety Knowledge AI Safety Fundamentals - BlueDot Impact AI Safety, Ethics, and Society Textbook Forming solid AI safety threat models helps you select impactful research ideas. 2.2 Speedrunning Technical Knowledge in 12 Hours Requires some basic coding, calculus, and linear algebra knowledge Build Intuition for ML (5h) Essence of linear algebra - 3Blue1Brown (3h) Neural networks - 3Blue1Brown (2h) Backpropagation, the foundation of deep learning (3h) Neural Networks: Backpropagation - CS 231N (0.5h) The spelled-out intro to neural networks and backpropagation: building micrograd (2.5h) Transformers and LLMs (4h) [1hr Talk] Intro to Large Language Models (1h) The Illustrated Transformer - Jay Alammar (1h) Let's build GPT: from scratch, in code, spelled out. (2h) 2.3 How to Build Technical Skills Traditionally, people take a couple of deep learning classes. Stanford CS 224N | Natural Language Processing with Deep Learning (lecture videos) Practical Deep Learning for Coders - Practical Deep Learning (fast.ai) Other curricula that seem good: Syllabus | Intro to ML Safety Levelling Up in AI Safety Research Engineering [Public] ARENA Maybe also check out recent topical classes like this with public lecture recordings: CS 194/294-267 Understanding Large Language Models: Foundations and Safety Beware of studying too much. You should aim to understand the fundamentals of ML through 1 or 2 classes and then practice doing many manageable research projects with talented collaborators or a good mentor who can give you time to meet. It's easy to keep taking classes, but you tend to learn many more practical ML skills through practice doing real research projects. You can also replicate papers to build experience. Be sure to focus on key results rather than wasting time replicating many experiments. "One learns from books and reels only that certain things can be done. Actual learning requires that you do those things." -Frank Herbert Note that ML engineering skills will be less relevant over time as AI systems become better at writing code. A friend didn't study computer science but got into MATS 2023 with good AI risk takes. Then, they had GPT-4 write most of their code for experiments and did very well in their stream. Personally, GitHub Copilot and language model apps with code interpreters/artifacts write a significant fraction of my code. However, fundamental deep learning knowledge is still useful for making sound decisions about what experiments to run. 2.4 Math You don't need much of it to do empirical ML research. Someone once told me, "You need the first chapter of a calculus textbook and the first 5 pages of a linear algebra textbook" to understand deep learning. You need more math for ML theory research, but theoretical research is not as popular right now. Beware mathification: authors often add unnecessary math to appease (or sometimes confuse) conference reviewers. If you don't understand some mathematical notation in an empirical paper, you can often send a screenshot to an LLM chatbot f...

Beef Checkoff Funds Beef Safety Research to Tackle Pathogen Risks

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The case for stopping AI safety research, published by catubc on May 24, 2024 on LessWrong. TLDR: AI systems are failing in obvious and manageable ways for now. Fixing them will push the failure modes beyond our ability to understand and anticipate, let alone fix. The AI safety community is also doing a huge economic service to developers. Our belief that our minds can "fix" a super-intelligence - especially bit by bit - needs to be re-thought. I wanted to write this post forever, but now seems like a good time. The case is simple, I hope it takes you 1min to read. 1. AI safety research is still solving easy problems. We are patching up the most obvious (to us) problems. As time goes we will no longer be able to play this existential risk game of chess with AI systems. I've argued this a lot (preprint; ICML paper accepted (shorter read, will repost), will be out in a few days; www.agencyfoundations.ai). Seems others have this thought. 2. Capability development is getting AI safety research for free. It's likely in the millions to tens of millions of dollars. All the "hackathons", and "mini" prizes to patch something up or propose a new way for society to digest/adjust to some new normal (and increasingly incentivizing existing academic labs). 3. AI safety research is speeding up capabilities. I hope this is somewhat obvious to most. I write this now because in my view we are about 5-7 years before massive human biometric and neural datasets will enter our AI training. These will likely generate amazing breakthroughs in long-term planning and emotional and social understanding of the human world. They will also most likely increase x-risk radically. Stopping AI safety research or taking it in-house with security guarantees etc, will slow down capabilities somewhat - and may expose capabilities developers more directly to public opinion of still manageable harmful outcomes. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The case for stopping AI safety research, published by catubc on May 24, 2024 on LessWrong. TLDR: AI systems are failing in obvious and manageable ways for now. Fixing them will push the failure modes beyond our ability to understand and anticipate, let alone fix. The AI safety community is also doing a huge economic service to developers. Our belief that our minds can "fix" a super-intelligence - especially bit by bit - needs to be re-thought. I wanted to write this post forever, but now seems like a good time. The case is simple, I hope it takes you 1min to read. 1. AI safety research is still solving easy problems. We are patching up the most obvious (to us) problems. As time goes we will no longer be able to play this existential risk game of chess with AI systems. I've argued this a lot (preprint; ICML paper accepted (shorter read, will repost), will be out in a few days; www.agencyfoundations.ai). Seems others have this thought. 2. Capability development is getting AI safety research for free. It's likely in the millions to tens of millions of dollars. All the "hackathons", and "mini" prizes to patch something up or propose a new way for society to digest/adjust to some new normal (and increasingly incentivizing existing academic labs). 3. AI safety research is speeding up capabilities. I hope this is somewhat obvious to most. I write this now because in my view we are about 5-7 years before massive human biometric and neural datasets will enter our AI training. These will likely generate amazing breakthroughs in long-term planning and emotional and social understanding of the human world. They will also most likely increase x-risk radically. Stopping AI safety research or taking it in-house with security guarantees etc, will slow down capabilities somewhat - and may expose capabilities developers more directly to public opinion of still manageable harmful outcomes. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Why was government research on cell phone radiation dangers halted after it produced evidence it causes cancer and DNA damage in animals?Subscribe to my two podcasts: “The Sharyl Attkisson Podcast” and “Full Measure After Hours.” Leave a review, subscribe and share with your friends! Support independent journalism by visiting the new Sharyl Attkisson store. Preorder Sharyl's new book: “Follow the $Science.” Visit SharylAttkisson.com and www.FullMeasure.news for original reporting. Do your own research. Make up your own mind. Think for yourself.See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Today's episode covers the challenges of navigating the copyright minefield and the impact of generative AI on content creation. Legislation targeting AI copyright compliance and fines for lack of creator disclosures is discussed. Kaseya introduces a generative AI assistant to empower sales teams, reducing documentation time.  Additionally, AI safety research is examined, revealing that only 2% of AI research is dedicated to the topic despite an increase in global AI safety research papers. Three things to know today 00:00 Navigating the Copyright Minefield: Generative AI's Challenge to Content Creation04:05 Legislation Targets AI Copyright Compliance, Introduces Fines for Lack of Creator Disclosures08:07 Kaseya Empowers Sales Teams with Generative AI Assistant, Slashing Documentation Time  Supported by:  https://atakama.com/mspradio/https://huntress.com/mspradio/  Looking for a link from the stories? The entire script of the show, with links to articles, are posted in each story on https://www.businessof.tech/ Do you want the show on your podcast app or the written versions of the stories? Subscribe to the Business of Tech: https://www.businessof.tech/subscribe/ Support the show on Patreon: https://patreon.com/mspradio/ Want our stuff? Cool Merch? Wear “Why Do We Care?” - Visit https://mspradio.myspreadshop.com Follow us on:LinkedIn: https://www.linkedin.com/company/28908079/YouTube: https://youtube.com/mspradio/Facebook: https://www.facebook.com/mspradionews/Instagram: https://www.instagram.com/mspradio/TikTok: https://www.tiktok.com/@businessoftechBluesky: https://bsky.app/profile/businessoftech.bsky.social

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why I think it's net harmful to do technical safety research at AGI labs, published by Remmelt on February 7, 2024 on LessWrong. IMO it is harmful on expectation for a technical safety researcher to work at DeepMind, OpenAI or Anthropic. Four reasons: Interactive complexity. The intractability of catching up - by trying to invent general methods for AI corporations to somehow safely contain model interactions, as other engineers scale models' combinatorial complexity and outside connectivity. Safety-capability entanglements Commercialisation. Model inspection and alignment techniques can support engineering and productisation of more generally useful automated systems. Infohazards. Researching capability risks within an AI lab can inspire researchers hearing about your findings to build new capabilities. Shifts under competitive pressure DeepMind merged with Google Brain to do commercialisable research, OpenAI set up a company and partnered with Microsoft to release ChatGPT, Anthropic pitched to investors they'd build a model 10 times more capable. If you are an employee at one of these corporations, higher-ups can instruct you to do R&D you never signed up to do.[1] You can abide, or get fired. Working long hours surrounded by others paid like you are, by a for-profit corp, is bad for maintaining bearings and your epistemics on safety.[2] Safety-washing. Looking serious about 'safety' helps labs to recruit idealistic capability researchers, lobby politicians, and market to consumers. 'let's build AI to superalign AI' 'look, pretty visualisations of what's going on inside AI' This is my view. I would want people to engage with the different arguments, and think for themselves what ensures that future AI systems are actually safe. ^ I heard via via that Google managers are forcing DeepMind safety researchers to shift some of their hours to developing Gemini for product-ready launch. I cannot confirm whether that's correct. ^ For example, I was in contact with a safety researcher at an AGI lab who kindly offered to read my comprehensive outline on the AGI control problem, to consider whether to share with colleagues. They also said they're low energy. They suggested I'd remind them later, and I did, but they never got back to me. They're simply too busy it seems. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why I think it's net harmful to do technical safety research at AGI labs, published by Remmelt on February 7, 2024 on LessWrong. IMO it is harmful on expectation for a technical safety researcher to work at DeepMind, OpenAI or Anthropic. Four reasons: Interactive complexity. The intractability of catching up - by trying to invent general methods for AI corporations to somehow safely contain model interactions, as other engineers scale models' combinatorial complexity and outside connectivity. Safety-capability entanglements Commercialisation. Model inspection and alignment techniques can support engineering and productisation of more generally useful automated systems. Infohazards. Researching capability risks within an AI lab can inspire researchers hearing about your findings to build new capabilities. Shifts under competitive pressure DeepMind merged with Google Brain to do commercialisable research, OpenAI set up a company and partnered with Microsoft to release ChatGPT, Anthropic pitched to investors they'd build a model 10 times more capable. If you are an employee at one of these corporations, higher-ups can instruct you to do R&D you never signed up to do.[1] You can abide, or get fired. Working long hours surrounded by others paid like you are, by a for-profit corp, is bad for maintaining bearings and your epistemics on safety.[2] Safety-washing. Looking serious about 'safety' helps labs to recruit idealistic capability researchers, lobby politicians, and market to consumers. 'let's build AI to superalign AI' 'look, pretty visualisations of what's going on inside AI' This is my view. I would want people to engage with the different arguments, and think for themselves what ensures that future AI systems are actually safe. ^ I heard via via that Google managers are forcing DeepMind safety researchers to shift some of their hours to developing Gemini for product-ready launch. I cannot confirm whether that's correct. ^ For example, I was in contact with a safety researcher at an AGI lab who kindly offered to read my comprehensive outline on the AGI control problem, to consider whether to share with colleagues. They also said they're low energy. They suggested I'd remind them later, and I did, but they never got back to me. They're simply too busy it seems. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reflections on my first year of AI safety research, published by Jay Bailey on January 9, 2024 on The Effective Altruism Forum. Last year, I wrote a post about my upskilling in AI alignment. To this day, I still get people occasionally reaching out to me because of this article, to ask questions about getting into the field themselves. I've also had several occasions to link people to the article who asked me about getting into the field from other means, like my local AI Safety group. Essentially, what this means is that people clearly found this useful (credit to the EA Forum for managing to let the article be findable to those who need it, a year after its publication!) and therefore people would likely find a sequel useful too! This post is that sequel, but reading the first post is not necessary to read this one. The major lesson of this post is this: I made a ton of mistakes, but those mistakes taught me things. By being open to that feedback and keeping my eye on the ball, I managed to find work that suited me in the field eventually. Just like the previous post, I'm happy to answer more questions via PM or in the comments. It's worth noting, this isn't a bold story of me getting a ton of stuff done. Most of the story, by word count, is me flailing around unsure of what to do and making a lot of mistakes along the way. I don't think you'll learn a lot about how to be a good researcher from this post, but I hope you might learn some tips to avoid being a bad one. Summary I was a software engineer for 3-4 years with little to no ML experience before I was accepted for my initial upskilling grant. (More details are in my initial post) I attended SERI MATS, working on aligning language models under Owain Evans. Due to a combination of factors, some my fault and some not, I don't feel like I got a great deal of stuff done. I decided to pivot away from evals towards mechanistic interpretability since I didn't see a good theory of change for evals - this was two weeks before GPT-4 came out and the whole world sat up and took notice. Doh! After upskilling in mechanistic interpretability, I struggled quite a bit with the research. I eventually concluded that it wasn't for me, but was already funded to work on it. Fortunately I had a collaborator, and eventually I wound up using my engineering skills to accelerate his research instead of trying to contribute to the analysis directly. After noticing my theory of change for evals had changed now that governments and labs were committing to red-teaming, I applied for some jobs in the space. I received an offer to work in the UK's task force, which I accepted. List of Lessons It's important to keep in mind two things - your theory of change for how your work helps reduce existential risk, and your comparative advantage in the field. These two things determined what I should work on, and keeping them updated was crucial for me finding a good path in the end. Poor productivity is more likely to be situational than you might think, especially if you're finding yourself having unusual difficulty compared to past projects or jobs. It's worth considering how your situation might be tweaked before blaming yourself. Trying out different subfields is useful, but don't be afraid to admit when one isn't working out as well as you'd like. See the first lesson. If you're going to go to a program like SERI MATS, do so because you have a good idea of what you want, not just because it's the thing to do or it seems generically helpful. I'm not saying you can't do such a program for that reason, but it is worth thinking twice about it. It is entirely possible to make mistakes, even several of them, and still wind up finding work in the field. There is no proper roadmap, everyone needs to figure things out as they go. While it's worth having...

Instagram's new generative AI-powered background editing tool allows users to change the background of their images with fun prompts. FunSearch, a new method for searching for solutions in mathematics and computer science, discovered new solutions for a longstanding open problem in mathematics. The concept of weak-to-strong generalization in AI is explored in a new research direction for superalignment. Finally, the paper "CLIP as RNN" proposes a recurrent framework that enhances mask quality without the need for additional training efforts, setting new state-of-the-art records for image segmentation tasks. Contact:  sergi@earkind.com Timestamps: 00:34 Introduction 01:50 Instagram introduces GenAI powered background editing tool 03:18 FunSearch: Making new discoveries in mathematical sciences using Large Language Models 05:23 The AI trust crisis 06:45 Weak-to-strong generalization 08:29 Fake sponsor 10:15 Invariant Graph Transformer 11:47 CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor 13:39 Outro

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety Research Organization Incubation Program - Expression of Interest, published by kaykozaronek on November 21, 2023 on The Effective Altruism Forum. Tl;dr: If you might want to participate in our incubation program and found an AI safety research organization, express your interest here. If you want to help out in other ways please fill out that same form. We Catalyze Impact - believe it is a bottleneck in AI safety that there are too few AI safety organizations. To address this bottleneck we are piloting an incubation program, similar to Charity Entrepreneurship's program. The incubation program is designed to help you find a complementary co-founder acquire additional knowledge and skills for founding an AI safety research organization get access to a network of mentors, advisors and potential funders Program overview We aim to deliver this program end of Q1 2024. Here's a broad outline of the 3 phases we are planning: Phase 1: Online preparation focused on skill building, workshops from experts, and relationship building (1 month) Phase 2: An immersive in-person experience in London, focused on testing cofounder fit, continuous mentorship, and networking (2 months) Phase 3: Continued individualized coaching and fundraising support Who is this program for? We are looking for motivated and ambitious engineers, generalists, technical researchers, or entrepreneurs who would like to contribute significantly to reducing the risks from AI. Express your Interest! If you are interested in joining the program, funding Catalyze, or helping out in other ways, please fill in this form! For more information, feel free to reach out at alexandra@catalyze-impact.org crossposted to LessWrong Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

We speak with Adam Gleave, CEO of FAR AI (https://far.ai). FAR AI's mission is to ensure AI systems are trustworthy & beneficial. They incubate & accelerate research that's too resource-intensive for academia but not ready for commercialisation. They work on everything from adversarial robustness, interpretability, preference learning, & more.We talk to Adam about:* The founding story of FAR as an AI safety org, and how it's different from the big commercial labs (e.g. OpenAI) and academia.* Their current research directions & how they're going* Promising agendas & notable gaps in the AI safety researchHosted by Soroush Pour. Follow me for more AGI content:Twitter: https://twitter.com/soroushjpLinkedIn: https://www.linkedin.com/in/soroushjp/== Show links ==-- About Adam --Adam Gleave is the CEO of FAR, one of the most prominent not-for-profits focused on research towards AI safety & alignment. He completed his PhD in artificial intelligence (AI) at UC Berkeley, advised by Stuart Russell, a giant in the field of AI. Adam did his PhD on trustworthy machine learning and has dedicated his career to ensuring advanced AI systems act according to human preferences. Adam is incredibly knowledgeable about the world of AI, having worked directly as a researcher and now as leader of a sizable and growing research org.-- Further resources --* Adam  * Website: https://www.gleave.me/  * Twitter: https://twitter.com/ARGleave  * LinkedIn: https://www.linkedin.com/in/adamgleave/  * Google Scholar: https://scholar.google.com/citations?user=lBunDH0AAAAJ&hl=en&oi=ao* FAR AI  * Website: https://far.ai  * Twitter: https://twitter.com/farairesearch  * LinkedIn: https://www.linkedin.com/company/far-ai/  * Job board: https://far.ai/category/jobs/* AI safety training bootcamps:  * ARENA: https://www.arena.education/  * See also: MLAB, WMLB, https://aisafety.training/* Research  * FAR's adversarial attack on Katago https://goattack.far.ai/* Ideas for impact mentioned by Adam  * Consumer report for AI model safety  * Agency model to support AI safety researchers   * Compute cluster for AI safety researchers* Donate to AI safety  * FAR AI: https://www.every.org/far-ai-inc#/donate/card  * ARC Evals: https://evals.alignment.org/  * Berkeley CHAI: https://humancompatible.ai/Recorded Oct 9, 2023

Are you interested in the privacy and security aspects of smart city solutions? Summary of the article titled Perception of the quality of smart city solutions as a sense of residents' safety from 2021 by Justína Zywiolek and Francesco Schiavone, published in the Energies journal. Since we are investigating the future of cities, I thought it would be interesting to see how to examine the safety and level of satisfaction for the smart city. This article presents a methodology for examining residents' satisfaction and potential threats in order to investigate undefined desires and identified and confirmed needs. As the most important things, I would like to highlight 3 aspects: Smart cities and smart city communities can be defined as systems of people that interact and use the flows of energy, materials, services and financing to catalyse a sustainable economy, development, resilience and high quality of life. Perceiving risk and fear of privacy loss may induce people to avoid smart city solutions, question their quality and intention, and past studies emphasized that residents' satisfaction rose when they viewed their city's intelligent solutions positively. The urban transformation to smart cities and better urban asset and resource management needs to incorporate citizens and their perceived risks to their privacy and security which cannot be disturbed. You can find the article through this link. Abstract: Personalization, mobility, artificial intelligence, corporate life transferred to the online world—all these elements will shape all intelligent solutions, including those for cities in the future also in the field of energy management. A necessary condition is to determine which specific repetitive behaviors and features smart cities will have to meet in order to build an image among residents and adapt to their preferences and requirements and energy requirements. Smart cities were created to support residents in using various services, to give them the possibility of easy communication without time and local barriers. Therefore, high-quality smart solutions in cities significantly affect trust in the city and can affect its reputation. Given that the purpose of the article is to examine the perception of intelligent solutions also in the field of energy and their impact on the sense of privacy and security, different exchanges of perceptions of quality, the risks they pose to residents and their perception of what gives a picture, have been studied. The results of empirical research clearly showed that the safety and level of satisfaction with the activities carried out by the city have a significant impact on the perceived quality, which in turn has a positive impact on reputation. The authors proposed a methodology based on the Kano model for examining residents' satisfaction in order to investigate undefined desires and identified and confirmed needs and to study the analysis of risk and potential threats. The study was in the form of a proprietary questionnaire that can be used in similar surveys on the satisfaction of residents; 2685 correctly completed questionnaires were analyzed and the results obtained after submission were included in management action plans. The city government has expressed an interest that the measures taken will be reviewed after one to two years. Connecting episodes you might be interested in: No.004R - Will the real smart city please stand up? No.162 - Interview with Warren Hill about securing things for citizens' use; You can find the transcript through ⁠⁠this link⁠⁠. What wast the most interesting part for you? What questions did arise for you? Let me know on Twitter ⁠⁠@WTF4Cities⁠⁠ or on the ⁠⁠wtf4cities.com⁠⁠ website where the ⁠⁠shownotes⁠⁠⁠⁠ ⁠⁠are also available. I hope this was an interesting episode for you and thanks for tuning in. Music by ⁠⁠Lesfm ⁠⁠from ⁠⁠Pixabay

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Technical AI Safety Research Landscape [Slides], published by Magdalena Wache on September 18, 2023 on LessWrong. I recently gave a technical AI safety research overview talk at EAGx Berlin. Many people told me they found the talk insightful, so I'm sharing the slides here as well. I edited them for clarity and conciseness, and added explanations. Outline This presentation contains An overview of different research directions Concrete examples for research in each category Disagreements in the field Intro Overview I'll start with an overview of different categories of technical AI safety research. The first category of research is what I would just call alignment, which is about making AIs robustly do what we want. Then there are various "meta" research directions such as automating alignment, governance, evaluations, threat modeling and deconfusion. And there is interpretability. Interpretability is probably not enough to build safe AI on its own, but it's really helpful/probably necessary for various alignment proposals. Interpretability also helps with deconfusion. I'm using clouds because the distinction between the categories often isn't very clear. Let's take a closer look at the first cloud. What exactly do I mean by alignment? What do we align with what? In general, we want to make AIs do what we want, so we want to align "what we want" with "what the AI does". That's why it's called alignment. We can split this up into intent alignment (make the AI want what we want) and capability robustness (make it able to robustly do what it wants). And we can split intent alignment up into outer alignment (find a function that captures what we want) and inner alignment (ensure that what the AI ends up wanting is the same as what's specified in the function that we trained it on). There are a few ways in which this slide is simplified: The outer/inner alignment split is not necessarily the right frame to look at things. Maybe "what the AI wants" isn't even a meaningful concept. And many approaches don't really fit into these categories. Also, this frame looks at making one AI do what we want, but we may end up in a multipolar scenario with many AIs. Concrete Technical Research In this section I'll give some examples to give you a flavor of what kinds of research exists in this space. There is of course a lot more research. Let's start with outer alignment. Outer alignment is the problem of finding a mathematical function which robustly captures what we want. The difficulty here is specification gaming. In this experiment the virtual robot learned to turn the red lego block upside down instead of the intended outcome of stacking it on top of the blue block. This might not seem like a big problem - the AI did what we told it to do. We just need to find a better specification and then it does what we want. But this toy example is indicative of a real and important problem. It is extremely hard to capture everything that we want in a specification. And if the specification is missing something, then the AI will do what is specified rather than what we meant to specify. A well-known technique in reward specification is called Reinforcement Learning from Human Feedback (RLHF). In the Deep reinforcement learning from human preferences paper they were able to make a virtual leg perform a backflip, despite "backflip" being very hard to specify mathematically. (Links: blogpost, paper) Let's continue with inner alignment. Inner alignment is about making sure that the AI actually ends up wanting the thing which it is trained on. The failure mode here is goal misgeneralization: (Links: forum post, paper) One way to train in more diverse environments is adversarial training: (Links: paper, takeaways post, deceptive alignment) As I mentioned above, for many approaches it doesn't really...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Technical AI Safety Research Landscape [Slides], published by Magdalena Wache on September 18, 2023 on LessWrong. I recently gave a technical AI safety research overview talk at EAGx Berlin. Many people told me they found the talk insightful, so I'm sharing the slides here as well. I edited them for clarity and conciseness, and added explanations. Outline This presentation contains An overview of different research directions Concrete examples for research in each category Disagreements in the field Intro Overview I'll start with an overview of different categories of technical AI safety research. The first category of research is what I would just call alignment, which is about making AIs robustly do what we want. Then there are various "meta" research directions such as automating alignment, governance, evaluations, threat modeling and deconfusion. And there is interpretability. Interpretability is probably not enough to build safe AI on its own, but it's really helpful/probably necessary for various alignment proposals. Interpretability also helps with deconfusion. I'm using clouds because the distinction between the categories often isn't very clear. Let's take a closer look at the first cloud. What exactly do I mean by alignment? What do we align with what? In general, we want to make AIs do what we want, so we want to align "what we want" with "what the AI does". That's why it's called alignment. We can split this up into intent alignment (make the AI want what we want) and capability robustness (make it able to robustly do what it wants). And we can split intent alignment up into outer alignment (find a function that captures what we want) and inner alignment (ensure that what the AI ends up wanting is the same as what's specified in the function that we trained it on). There are a few ways in which this slide is simplified: The outer/inner alignment split is not necessarily the right frame to look at things. Maybe "what the AI wants" isn't even a meaningful concept. And many approaches don't really fit into these categories. Also, this frame looks at making one AI do what we want, but we may end up in a multipolar scenario with many AIs. Concrete Technical Research In this section I'll give some examples to give you a flavor of what kinds of research exists in this space. There is of course a lot more research. Let's start with outer alignment. Outer alignment is the problem of finding a mathematical function which robustly captures what we want. The difficulty here is specification gaming. In this experiment the virtual robot learned to turn the red lego block upside down instead of the intended outcome of stacking it on top of the blue block. This might not seem like a big problem - the AI did what we told it to do. We just need to find a better specification and then it does what we want. But this toy example is indicative of a real and important problem. It is extremely hard to capture everything that we want in a specification. And if the specification is missing something, then the AI will do what is specified rather than what we meant to specify. A well-known technique in reward specification is called Reinforcement Learning from Human Feedback (RLHF). In the Deep reinforcement learning from human preferences paper they were able to make a virtual leg perform a backflip, despite "backflip" being very hard to specify mathematically. (Links: blogpost, paper) Let's continue with inner alignment. Inner alignment is about making sure that the AI actually ends up wanting the thing which it is trained on. The failure mode here is goal misgeneralization: (Links: forum post, paper) One way to train in more diverse environments is adversarial training: (Links: paper, takeaways post, deceptive alignment) As I mentioned above, for many approaches it doesn't really...

Darren saw a story about KZN which he had to share with Carmen and Sky. Here's what you missed out on during the Five Things You Needed To Know, Webpage

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cost-effectiveness of professional field-building programs for AI safety research, published by Center for AI Safety on July 10, 2023 on The Effective Altruism Forum. Summary This post explores the cost-effectiveness of AI safety field-building programs aimed at ML professionals, specifically the Trojan Detection Challenge (a prize), NeurIPS ML Safety Social, and NeurIPS ML Safety Workshop. We estimate the benefit of these programs in ‘Quality-Adjusted Research Years', using cost-effectiveness models built for the Center for AI Safety (introduction post here, full code here). We intend for these models to support - not determine - strategic decisions. We do not believe, for instance, that programs which a model rates as lower cost-effectiveness are necessarily not worthwhile as part of a portfolio of programs. The models' tentative results, summarized below, suggest that field-building programs for professionals compare favorably to ‘baseline' programs - directly funding a talented research scientist or PhD student working on trojans research for 1 year or 5 years respectively. Further, the cost-effectiveness of these programs can be significantly improved with straightforward modifications - such as focusing a hypothetical prize on a more ‘relevant' research avenue, or running a hypothetical workshop with a much smaller budget. ProgramCost (USD)Benefit (counterfactual expected QARYs)Cost-effectiveness (QARYs per $1M)Trojan Detection Challenge65,00026390NeurIPS ML Safety Social520015029,000NeurIPS ML Safety Workshop110,0003603300Hypothetical: Power Aversion Prize50,0004909900Hypothetical: Cheaper Workshop35,0002507000Baseline: Scientist Trojans500,000 84170Baseline: PhD Trojans250,0008.735 For readers who are after high-level takeaways, including which factors are driving these results, skip ahead to the cost-effectiveness in context section. For those keen on understanding the model and results in more detail, read on as we: Give important disclaimers. (Read more.) Direct you to background information about this project. (Read more.) Walk through the model. (Read more.) Contrast these programs with one another, and with funding researchers directly. (Read more.) Consider the scalability and robustness properties of the model. (Read more.) Disclaimer This analysis is a starting point for discussion, not a final verdict. The most critical reasons for this are that: These models are reductionist. Even if we have avoided other pitfalls associated with cost-effectiveness analyses, the models might ignore factors that turn out to be crucial in practice, including (but not limited to) interactions between programs, threshold effects, and diffuse effects. The models' assumptions are first-pass guesses, not truths set in stone. Most assumptions are imputed second-hand following a short moment of thought, before being adjusted ad-hoc for internal consistency and differences of beliefs between Center for AI Safety (CAIS) staff and external practitioners. In some cases, parameters have been redefined since initial practitioner input. Instead, the analyses in this post represent an initial effort in explicitly laying out assumptions, in order to take a more systematic approach towards AI safety field-building. Background For an introduction to our approach to modeling - including motivations for using models, the benefits and limitations of our key metric, guidance for adopting or adapting the models for your own work, comparisons between programs for students and professionals, and more - refer to the introduction post. 2. The models' default parameters are based on practitioner surveys and the expertise of CAIS staff. Detailed information on the values and definitions of these parameters, and comments on parameters with delicate definitions or contestable views, can be found i...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cost-effectiveness of student programs for AI safety research, published by Center for AI Safety on July 10, 2023 on The Effective Altruism Forum. Summary This post explores the cost-effectiveness of field-building programs for students, specifically the Atlas Fellowship (a rationality program, with some AI safety programming), MLSS (an ML safety course for undergraduates), a top-tier university student group, and undergraduate research stipends. We estimate the benefit of these programs in ‘Quality-Adjusted Research Years', using cost-effectiveness models built for the Center for AI Safety (introduction post here, full code here). Since our framework focuses on benefits for technical AI safety research exclusively, we will not account for other benefits of programs with broader objectives, such as the Atlas Fellowship. We intend for these models to support - not determine - strategic decisions. We do not believe, for instance, that programs which a model rates as lower cost-effectiveness are necessarily not worthwhile as part of a portfolio of programs. The models' tentative results, summarized below, suggest that student groups and undergraduate research stipends are considerably more cost-effective than Atlas and MLSS. (With many important caveats and uncertainties, discussed in the post.) Additionally, student groups and undergraduate research stipends compare favorably to ‘baseline' programs - directly funding a talented research scientist or PhD student working on trojans research for 1 year or 5 years respectively. ProgramCost (USD) Benefit (counterfactual expected QARYs)Cost-effectiveness (QARYs per $1M)Atlas9,000,000 434.7MLSS330,0006.419Student Group350,00050140Undergraduate Stipends50,00017340Baseline: Scientist Trojans500,00084170Baseline: PhD Trojans250,0008.735 For readers who are after high-level takeaways, including which factors are driving these results, skip ahead to the cost-effectiveness in context section. For those keen on understanding the model and results in more detail, read on as we: Give important disclaimers. (Read more.) Direct you to background information about this project. (Read more.) Walk through the model. (Read more.) Contrast these programs with one another, and with funding researchers directly. (Read more.) Test the robustness of the model. (Read more.) Disclaimers This analysis is a starting point for discussion, not a final verdict. The most critical reasons for this are that: These models are reductionist. Even if we have avoided other pitfalls associated with cost-effectiveness analyses, the models might ignore factors that turn out to be crucial in practice, including (but not limited to) interactions between programs, threshold effects, and diffuse effects. The models' assumptions are first-pass guesses, not truths set in stone. Most assumptions are imputed second-hand following a short moment of thought, before being adjusted ad-hoc for internal consistency and differences of beliefs between Center for AI Safety (CAIS) staff and external practitioners. In some cases, parameters have been redefined since initial practitioner input. This caveat is particularly important for the Atlas Fellowship, where we have not discussed parameter values with key organizers. Instead, the analyses in this post represent an initial effort in explicitly laying out assumptions, in order to take a more systematic approach towards AI safety field-building. Background For an introduction to our approach to modeling - including motivations for using models, the benefits and limitations of our key metric, guidance for adopting or adapting the models for your own work, comparisons between programs for students and professionals, and more - refer to the introduction post. The models' default parameters are based on practitioner surveys and the ex...

This is my conversation with Steve Kerber from the UL's Fire Safety Research Institute from the 2023 NFPA Convention in Las Vegas! We discuss what the Research Institute is doing to reduce the damage and injuries that come from Lithium Ion batteries. What's the future for Lithium, and how are they testing current products available, and more.

Episode 613: NAMIC CEO Neil Alldredge talks with the Insurance Institute for Highway Safety President David Harkey about how the organization continuously updates road safety standards to better protect drivers and deter distractions. 

Rishi Sunak is the Prime Minister of the United Kingdom. He was previously appointed Chancellor of the Exchequer from 13 February 2020 to 5 July 2022. He was Chief Secretary to the Treasury from 24 July 2019 to 13 February 2020, and Parliamentary Under Secretary of State at the Ministry of Housing, Communities and Local Government from 9 January 2018 to 24 July 2019. Before entering the world of politics, Rishi co-founded an investment firm. In Today's Episode with Rishi Sunak We Discuss: 1. The United Kingdom: Open for AI: Open for Business Why does Rishi believe the UK is best placed to lead the way for innovation in AI? What can the government do to ensure the public and private sectors work together most efficiently? Why has Rishi created an entirely new division just for this? How does this change how decisions for AI and technology are made? 2. $100M Funding: The Largest Government Funding in the World: Why did Rishi decide to allocate the largest pool of capital of any nation toward AI safety? What is the strategy for the $100M? How will it be invested? Who will manage it? What are the challenges and opportunities in setting up this $100M funding program? 3. Education: Attracting the Best in the World: What has Rishi done to ensure the best talent in the world, wants to and can work in the UK? What new initiative has Rishi put in place to ensure the world's brightest students can freely move to and work in the UK? What can be done to ensure the UK continues to foster the same level of homegrown talent that we always have done? What can we do to improve our current education system for AI even further? Why does Rishi believe one of the greatest opportunities for AI lies in education and teaching? 4. Making Regulation Work Effectively: How does Rishi think about creating regulation which is both effective and not prohibitive? What can we do to create a government that moves at the speed of business? What does Rishi believe are the biggest mistakes made in regulatory provisions? What are we doing to avoid them with AI in the UK?

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is:

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is:

America Out Loud PULSE with Dr. Harvey Risch – Why is vaccine safety research so filled with manipulation and corruption? Not just during Covid, but going back decades. Dr. Harvey Risch is joined by Mary Holland and Zoey O'Toole. In today's conversation, we talk about the massive scale of corruption of vaccine safety studies going back decades...

America Out Loud PULSE with Dr. Harvey Risch – Why is vaccine safety research so filled with manipulation and corruption? Not just during Covid, but going back decades. Dr. Harvey Risch is joined by Mary Holland and Zoey O'Toole. In today's conversation, we talk about the massive scale of corruption of vaccine safety studies going back decades...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: $20 Million in NSF Grants for Safety Research, published by Dan H on February 28, 2023 on The AI Alignment Forum. After a year of negotiation, the NSF has announced a $20 million request for proposals for empirical AI safety research. Here is the detailed program description. The request for proposals is broad, as is common for NSF RfPs. Many safety avenues, such as transparency and anomaly detection, are in scope: "reverse-engineering, inspecting, and interpreting the internal logic of learned models to identify unexpected behavior that could not be found by black-box testing alone" "Safety also requires... methods for monitoring for unexpected environmental hazards or anomalous system behaviors, including during deployment." Note that research that has high capabilities externalities is explicitly out of scope: "Proposals that increase safety primarily as a downstream effect of improving standard system performance metrics unrelated to safety (e.g., accuracy on standard tasks) are not in scope." Thanks to OpenPhil for funding a portion the RfP---their support was essential to creating this opportunity! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: $20 Million in NSF Grants for Safety Research, published by Dan H on February 28, 2023 on LessWrong. After a year of negotiation, the NSF has announced a $20 million request for proposals for empirical AI safety research. Here is the detailed program description. The request for proposals is broad, as is common for NSF RfPs. Many safety avenues, such as transparency and anomaly detection, are in scope: "reverse-engineering, inspecting, and interpreting the internal logic of learned models to identify unexpected behavior that could not be found by black-box testing alone" "Safety also requires... methods for monitoring for unexpected environmental hazards or anomalous system behaviors, including during deployment." Note that research that has high capabilities externalities is explicitly out of scope: "Proposals that increase safety primarily as a downstream effect of improving standard system performance metrics unrelated to safety (e.g., accuracy on standard tasks) are not in scope." Thanks to OpenPhil for funding a portion the RfP---their support was essential to creating this opportunity! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Has private AGI research made independent safety research ineffective already? What should we do about this?, published by Roman Leventov on January 23, 2023 on LessWrong. This post is a variation on "Private alignment research sharing and coordination" by porby. You can consider my question as signal-boosting that post. AGI research is becoming private. Research at MIRI is nondisclosed-by-default for more than four years now. OpenAI stopped publishing details of their work, Hassabis also talked about this here. Does this mean that independent AI safety research begins to suffer from knowledge asymmetry and becomes ineffective? There are two directions possible directions of knowledge asymmetry: State-of-the-art scaling results or even novel architectures are not published, and interpretability researchers use outdated models in their work. Hence, these results may not generalise to bigger model scales and architectures. The counterargument here is that in what comes to scale, relatively low-hanging interpretability fruit is still possible to pick even when analysing toy models. In what comes to architectures, transformer details may not matter that much for interpretability (however, Anthropic's SoLU (Softmax Linear Unit) work seems to be the evidence against this statement: relatively minor architectural change has led to significant changes in the interpretability characteristics of the model); and if one of the AGI labs stumbles upon a major "post-transformer" breakthrough, this is going to be an extremely closely-guarded secret which will not spread with rumours to AI safety labs, and hence joining the "knowledge space" of these AI safety labs won't help independent AI safety researchers. Some safety research has already been done but was not published, either because of its relative infohazardousness or because it references private capability research, as discussed in the previous point. Oblivious to this work, AI safety researchers may reinvent the wheel rather than work on the actual frontier. Private research space If the issue described above is real, maybe the community needs some organisational innovation to address it. Perhaps it could be some NDA + "noncompete" agreement + publishing restriction program led by some AI safety lab (or a consortium of AI safety labs) joining which is not the same as joining the lab itself in the usual sense (no reporting, no salary), but which grants access to the private knowledge space of the lab(s). The effect for the lab will be as if they enlarged their research group, the extra part of which is not controllable and is not guaranteed to contribute to their agenda, but costs them little: there are only costs for supporting the legal and IT infrastructure of the "private research space". There are some concerns that put the usefulness of this system in question, though. First, there may be too few people who will be willing to join this private research space without joining the lab(s) themselves. Academics, including PhD students, want to publish. Only independent AI safety researchers may be eligible, which is a much smaller pool of people. Furthermore, the reasons why people opt for being independent AI safety researchers sometimes correlate with low involvement or low capability (of conducting good research), so these people who join the private research space may not be able to push the frontier by much. Second, the marginal usefulness of the work done by independent people in the private research space may be offset by the marginally higher risk of information leakage. On the other hand, if the private research space is organised not by a single AI safety lab but by a consortium (or organised by a single lab but more labs join this research space later), the first concern above becomes irrelevant and the second c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My AGI safety research—2022 review, '23 plans, published by Steven Byrnes on December 14, 2022 on LessWrong. The short version: In this post I'm briefly summarizing how I spent my work-time in 2022, and what I'm planning for 2023. The first half of 2022 was writing the “Intro to Brain-Like-AGI Safety” blog post series. The second half of 2022 was split maybe 45%-45%-10% between my main research project (on reverse-engineering human social instincts), miscellaneous other research and correspondence, and outreach mostly targeted towards neuroscientists. I expect to carry on with a similar time allocation into 2023. If you think there are other things I should be doing instead or differently, please don't be shy, the comment section is below, or DM me, email, etc. The long version: 1. First half of 2022: Writing “Intro to Brain-Like AGI Safety” So, I was writing some technical post in late 2021, and realized that the thing I was talking about was a detail sitting on top of a giant pile of idiosyncratic beliefs and terminology that nobody else would understand. So I started writing a background section to that post. That background section grew and grew and grew, and eventually turned into a book-length series of 15 blog posts entitled “Intro to Brain-Like AGI Safety”, which reorganized and re-explained almost everything I had written and thought about up to that point, since I started in the field around 2019. (My palimpsest!) Writing that series took up pretty much 100% of my work time until May 2022. Then I spent much of the late spring and summer catching up on lots of miscellaneous to-do-list stuff that I had put off while writing the series, and everyone in my family caught COVID, and we took a family vacation, and I attended two conferences, and I switched jobs when Jed McCaleb generously offered me a home at Astera Institute, and various other things. So I didn't get much research done during the late spring and summer. Moving on to the rest of the year, my substantive work time has been divided, I dunno, something like 45%-45%-10% between “my main research project”, “other research”, and “outreach”. Let's take those one at a time in the next three sections. 2. Second half of 2022 (1/3): My main research project 2.1 What's the project? I'm working on the open neuroscience problem that I described in the post Symbol Grounding and Human Social Instincts, and motivated in the post Two paths forward: “Controlled AGI” and “Social-instinct AGI”. I'll give an abbreviated version here. As discussed in “Intro to Brain-Like-AGI Safety”, I hold the following opinions: We should think of within-lifetime learning in the human brain as a kind of model-based reinforcement learning (RL) system; We should think of that model-based RL system as potentially similar to how future AGIs will work; We should (to a first approximation) think of the “reward function” of that RL system as encoding “innate drives”, like pain being bad and sweet tastes being good; These “innate drives” correspond to specific genetically-hardwired circuitry primarily in the hypothalamus and brainstem; A subset of that circuitry underlies human social and moral instincts; .And the project I'm working on is an attempt to figure out what those circuits are and how they work. 2.2 Why do I think success on this project would be helpful for AGI safety? I have two arguments: The modest argument is: At some point, I hope, we will have a science that can produce predictions of the form: (“Innate drives” a.k.a. “Reward function” X) + (“Life experience” a.k.a. “Training environment” Y) (“Adult” AGI that's trying to do Z) If we knew exactly what innate drives are in humans (particularly related to sociality, morality, etc.), then we would have actual examples of X+YZ to ground this future science. Even with the benefit...

We sit down with Steve Kerber, Vice President and Executive Director of UL's Fire Safety Research Institute (FSRI). He leads a fire safety research team dedicated to addressing the worlds unresolved fire safety risks and emerging dangers to reduce death, injury and loss from fire. Steve has led research in the areas of fire safety engineering, firefighter safety, fire forensics, and fire science. We talk about the risks of the new products used in residential construction and what new materials are making your house less safe due to fire and other risks. We talk about the risks of our firefighters and how residential construction and what you do inside your home can change your chance of survival in a house fire. For more information head to https://closeyourdoor.org/ For more information about FSRI: https://fsri.org/Thanks for listening to Around the house if you want to hear more please subscribe so you get notified of the latest episode as it posts at https://around-the-house-with-e.captivate.fm/listenIf you want to buy Eric G a beer or coffee you can support the show here: https://www.buymeacoffee.com/ATHERICGWe love comments and we would love reviews on how this information has helped you on your house! Thanks for listening! For more information about the show head to https://aroundthehouseonline.com/ We have moved the Pro Insider Special on Thursday to its new feed. It will no longer be on this page. You can find it and subscribe right here: https://around-the-house-pro-insider.captivate.fm/ Information given on the Around the House Show should not be considered construction or design advice for your specific project, nor is it intended to replace consulting at your home or jobsite by a building professional. The views and opinions expressed by those interviewed on the podcast are those of the guests and do not necessarily reflect the views and opinions of the Around the House Show. Mentioned in this episode:Buy Me A Coffee, Beer, or Drink

LISTEN Show Notes: In his From the Heart segment, Dr. Paul talks about how parents who are willing to stand in the gap for their children to protect them from harm are the real heroes in this life. He chooses to stand with these parents and all individuals who are fighting for the truth and fighting for what’s right. Standing together is how we can make progress! This week, Dr. Paul does a Covid deep dive with Steve Kirsch, independent journalist and executive director of t [...]

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety Ideas: An Open AI Safety Research Platform, published by Apart Research on October 17, 2022 on The Effective Altruism Forum. TLDR; We present the AI safety ideas and research platform AI Safety Ideas in open alpha. Add and explore research ideas on the website here: aisafetyideas.com. AI Safety Ideas has been accessible for a while in an alpha state (4 months, on-and-off development) and we now publish it in open alpha to receive feedback and develop it continuously with the community of researchers and students in AI safety. All of the projects are either from public sources (e.g. AlignmentForum posts) or posted on the website itself. The current website represents the first steps towards an accessible crowdsourced research platform for easier research collaboration and hypothesis testing. The gap in AI safety Research prioritization & development Research prioritization is hard and even more so in a pre-paradigmatic field like AI safety. We can grok the highest-karma post on the AlignmentForum but is there another way? With AI Safety Ideas, we introduce a collaborative way to prioritize and work on specific agendas together through social features. We hope this can become a scalable research platform for AI safety. Successful examples of less systematized but similar, collaborative, online, and high quality output projects can be seen in Discord servers such as EleutherAI, CarperAI, Stability AI, and Yannic Kilcher's Discord, in hackathons, and in competitions such as the inverse scaling competition. Additionally, we are also missing an empirically driven impact evaluation of AI safety projects. With the next steps of development described further down, we hope to make this easier and more available while facilitating more iteration in AI safety research. Systemized hypotheses testing with bounties can help funders directly fund specific results and enables open evaluation of agendas and research projects. Mid-career & student newcomers Novice and entrant participation in AI safety research is mostly present in two forms at the moment: 1) Active or passive part-time course participation with a capstone project (AGISF, ML Safety) and 2) flying to London or Berkeley for three months to participate in full-time paid studies and research (MLAB, SERI MATS, PIBBSS, Refine). Both are highly valuable but a third option seems to be missing: 3) An accessible, scalable, low time commitment, open research opportunity. Very few people work in AI safety and allowing decentralized, volunteer or bounty-driven research will allow many more to contribute to this growing field. By allowing this flexible research opportunity, we can attract people who cannot participate in option (2) because of visa, school / life / work commitments, location, rejection, or funding while we can attract a more senior and active audience compared to option (1). Next steps OctReleasing and building up the user base and crowdsourced content. Create an insider build to test beta features. Apply to join the insider build here.NovImplementing hypothesis testing features: Creating hypotheses, linking ideas and hypotheses, adding negative and positive results to hypotheses. Creating an email notification system.DecCollaboration features: Contact others interested in the same idea and mentor ideas. A better commenting system with a results comment that can indicate if the project has been finished or not, what the results are, and by who was it done.JanAdding moderation features: Accepting results, moderating hypotheses, admin users. Add bounty features for the hypotheses and a simple user karma system.FebShare with ML researchers and academics in EleutherAI and CarperAI. Implement the ability to create special pages with specific private and public ideas curated for a specific purpose (title and desc...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Levelling Up in AI Safety Research Engineering, published by Gabriel Mukobi on September 2, 2022 on The Effective Altruism Forum. Summary: A level-based guide for independently up-skilling in AI Safety Research Engineering that aims to give concrete objectives, goals, and resources to help anyone go from zero to hero. Cross-posted to LessWrong. View a pretty Google Docs version here. Introduction I think great career guides are really useful for guiding and structuring the learning journey of people new to a technical field like AI Safety. I also like role-playing games. Here's my attempt to use levelling frameworks and break up one possible path from zero to hero in Research Engineering for AI Safety (e.g. jobs with the “Research Engineer” title) through objectives, concrete goals, and resources. I hope this kind of framework makes it easier to see where one is on this journey, how far they have to go, and some options to get there. I'm mostly making this to sort out my own thoughts about my career development and how I'll support other students through Stanford AI Alignment, but hopefully, this is also useful to others! Note that I assume some interest in AI Safety Research Engineering—this guide is about how to up-skill in Research Engineering, not why (though working through it should be a great way to test your fit). Also note that there isn't much abstract advice in this guide (see the end for links to guides with advice), and the goal is more to lay out concrete steps you can take to improve. For each level, I describe the general capabilities of someone at the end of that level, some object-level goals to measure that capability, and some resources to choose from that would help get there. The categories of resources within a level are listed in the order you should progress, and resources within a category are roughly ordered by quality. There's some redundancy, so I would recommend picking and choosing between the resources rather than doing all of them. Also, if you are a student and your university has a good class on one of the below topics, consider taking that instead of one of the online courses I listed. As a very rough estimate, I think each level should take at least 100-200 hours of focused work, for a total of 700-1400 hours. At 10 hours/week (quarter-time), that comes to around 16-32 months of study but can definitely be shorter (e.g. if you already have some experience) or longer (if you dive more deeply into some topics)! I think each level is about evenly split between time spent reading/watching and time spent building/testing, with more reading earlier on and more building later. Confidence: mid-to-high. I am not yet an AI Safety Research Engineer (but I plan to be)—this is mostly a distillation of what I've read from other career guides (linked at the end) and talked about with people working on AI Safety. I definitely haven't done all these things, just seen them recommended. I don't expect this to be the “perfect” way to prepare for a career in AI Safety Research Engineering, but I do think it's a very solid way. Level 1: AI Safety Fundamentals Objective‏‏‎ ‎ You are familiar with the basic arguments for existential risks due to advanced AI, models for forecasting AI advancements, and some of the past and current research directions within AI alignment/safety. Note: You should be coming back to these readings and keeping up to date with the latest work in AI Safety throughout your learning journey. It's okay if you don't understand everything on your first try—Level 1 kind of happens all the time. Goals‏‏‎ ‎ Complete an AI Safety introductory reading group fellowship. Write a reflection distilling, recontextualizing, or expanding upon some AI Safety topic and share it with someone for feedback. Figure out how convinced you are of the arg...

Libby Znaimer is joined, first, by Dr. Michael Warner, medical director of critical care at Toronto

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Social scientists interested in AI safety should consider doing direct technical AI safety research, (meta-research if they've got a clear, decently-visionary path to impact), or moving into governance, other support roles (e.g. ops), or AI safety community building instead, published by Vael Gates on July 20, 2022 on The Effective Altruism Forum. Hello! This post (thesis in the title) is a bit of a hot take of mine, so I'm happy to hear pushbacks and for it to be wrong. It's also written in a casual style, so that I will ever publish it. What follows are some reflections I had after trying for several years to answer: “What should social scientists who are primarily interested in reducing risks from AI be doing, specifically?” My main takeaways are the headings / on the sidebar, so to skim just read those. Note: This post is specifically aimed at social science-type EAs (and also neuroscientists) who are most interested in contributing to AI safety, which is meant as a statement of interests (people who like social science) and goals (contributing to AI safety). It's not meant to apply outside of the set of people who self-identify into this cluster. I wrote the post because I happen to fall into this cluster (PhD in computational cognitive science, interested in AI safety) and have a lot of latent thoughts about career exploration within it. In particular, EAs with social science BAs will sometimes ask me what options I think they should pursue, and I'd like to pass them this doc about how I'm currently thinking about the space. Some final notes: if you're not interested in dedicating your career to AI safety, I salute you and you are not the target of this post! All opinions are my own and I expect people in the community to disagree with me. Many thanks to comments from Abby Novick Hoskin, Aaron Scher, TJ, Nora Ammann, Noemi Dreksler, Linch Zhang, Lucius Caviola, Joshua Lewis, Peter Slattery, and Michael Keenan for making this post better; thank you for raising disagreements and agreements! (1) “AI Safety Needs Social Scientists” → I think more specifically, this article was describing a new paradigm that means that AI safety has a limited number of roles (0-2 per year?) open for people who are approximately top computational / quantitative PhD-level cognitive(/neuro) scientists. This is great, but people often take this article to mean something broader than that based on the title, and I think that's a misleading interpretation. In 2019, Geoffrey Irving and Amanda Askell (then at OpenAI) published an article called “AI Safety Needs Social Scientists”. This was great, and the purpose to my eye seemed to be introducing a new paradigm in AI safety that would require hiring for a new role. Specifically, it seemed they were looking to hire approximately PhD-level researchers who'd done a lot of experiments with humans in the past, who could collaborate with machine learning (ML) researchers to help run integrated human / AI experiments. Note, however, that that's a pretty specific ask: “social science” includes fields like anthropology, sociology, psychology, political science, linguistics, and economics. If I were advertising this position, I'd be looking for “computational / quantitative PhD-level cognitive(/neuro) scientists”, which are academic labels that imply: a researcher who does empirical human experiments, who knows how to program and regularly does data analysis in e.g. Python, who is likely to be familiar with ML paradigms and used to being an interdisciplinary researcher who occasionally publishes in computer science (CS) journals. I happen to be one of those! I'm a computational cognitive scientist who did very large-scale human experiments by academia's lights– I had thousands of people in one of my experiments. I did my PhD with one of the world's t...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Which AI Safety research agendas are the most promising?, published by Chris Leong on July 13, 2022 on The AI Alignment Forum. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some alternative AI safety research projects, published by Michele Campolo on June 28, 2022 on The AI Alignment Forum. These are some "alternative" (in the sense of non-mainstream) research projects or questions related to AI safety that seem both relevant and underexplored. If instead you think they aren't, let me know in the comments, and feel free to use the ideas as you want if you find them interesting. Access-to-the-internet scenario and related topics A potentially catastrophic scenario that appears somewhat frequently in AI safety discourse involves a smarter-than-human AI which gets unrestricted access to the internet, and then bad things happen. For example, the AI manages to persuade or bribe one or more humans so that they perform actions which have a high impact on the world. What are the worst (i.e. with worst consequences) examples of similar scenarios that already happened in the past? Can we learn anything useful from them? Considering these scenarios, why is it the case that nothing worse has happened yet? Is it simply because human programmers with bad intentions are not smart enough? Or because the programs/AIs themselves are not agentic enough? I would like to read well-thought arguments on the topic. Can we learn something from the history of digital viruses? What's the role played by cybersecurity? If we assume that slowing down progress in AI capabilities is not a viable option, can we make the above scenario less likely to happen by changing or improving cybersecurity? Intuitively, it seems to me that the relation of AI safety with cybersecurity is similar to the relation with interpretability: even though the main objective of the other fields is not the reduction of global catastrophic risk, some of the ideas in those fields are likely to be relevant for AI safety as well. Cognitive and moral enhancement in bioethics A few days ago I came across a bioethics paper that immediately made me think of the relation between AI safety and AI capabilities. From the abstract: "Cognitive enhancement [...] could thus accelerate the advance of science, or its application, and so increase the risk of the development or misuse of weapons of mass destruction. We argue that this is a reason which speaks against the desirability of cognitive enhancement, and the consequent speedier growth of knowledge, if it is not accompanied by an extensive moral enhancement of humankind." As far as I understand, some researchers in the field are pro cognitive enhancement—sometimes even instrumentally as a way to achieve moral enhancement itself. Others, like the authors above, are much more conservative: they see research into cognitive enhancement as potentially very dangerous, unless accompanied by research into moral enhancement. Are we going to solve all our alignment problems by reading the literature on cognitive and moral enhancement in bioethics? Probably not. Would it be useful if at least some individuals in AI safety knew more than the surface-level info given here? Personally, I would like that. Aiming at “acceptably safe” rather than “never catastrophic” Let's say you own a self-driving car and you are deciding whether to drive or give control to the car. If all you care about is safety of you and others, what matters for your decision is the expected damage of you driving the car versus the expected damage of self-driving. This is also what we care about on a societal level. It would be great if self-driving cars were perfectly safe, but what is most important is that they are acceptably safe, in the sense that they are safer than the human counterpart they are supposed to replace. Now, the analogy with AI safety is not straightforward because we don't know to what extent future AIs will replace humans, and also because it will be a matter of “coexistence" (...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to get into AI safety research, published by Stuart Armstrong on May 18, 2022 on The AI Alignment Forum. Recently, I had a conversation with someone from a math background, asking how they could get into AI safety research. Based on my own path from mathematics to AI alignment, I recommended the following sources. It may prove useful to others contemplating a similar change in career: Superintelligence by Nick Bostrom. It condenses all the main arguments for the power and the risk of AI, and gives a framework in which to think of the challenges and possibilities. Sutton and Barto's Book: Reinforcement Learning: An Introduction. This gives the very basics of what ML researchers actually do all day, and is important for understanding more advanced concepts. It gives (most of) the vocabulary to understand what ML and AI papers are talking about. Gödel without too many tears. This is how I managed to really grok logic and the completeness/incompleteness theorems. Important for understanding many of MIRI's and LessWrong's approaches to AI and decision theory. Safely Interruptible agents. It feels bad to recommend one of my own papers, but I think this is an excellent example of bouncing between ML concepts and alignment concepts to make some traditional systems interruptible (so that we can shut them down without them resisting the shutdown). Alignment for Advanced Machine Learning Systems. Helps give an overall perspective on different alignment methods, and some understanding of MIRI's view on the subject (for a deeper understanding, I recommend diving into MIRI's or Eliezer's publications/writings). You mileage may vary, but these are the sources that I would recommend. And I encourage you to post any sources you'd recommend, in the comments. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A grand strategy to recruit AI capabilities researchers into AI safety research, published by Peter S. Park on April 15, 2022 on The Effective Altruism Forum. AI capabilities research seems to be substantially outpacing AI safety research. It is most likely true that successfully solving the AI alignment problem before the successful development of AGI is critical for the continued survival and thriving of humanity. Assuming that AI capabilities research continues to outpace AI safety research, the former will eventually result in the most negative externality in history: a significant risk of human extinction. Despite this, a free-rider problem causes AI capabilities research to myopically push forward, both because of market competition and great power competition (e.g., U.S. and China). AI capabilities research is thus analogous to the societal production and usage of fossil fuels, and AI safety research is analogous to green-energy research. We want to scale up and accelerate green-energy research as soon as possible, so that we can halt the negative externalities of fossil fuel use. My claim: A task that seems extremely effective on expectation (and potentially, maximally effective for a significant number of people) is to recruit AI capabilities researchers into AI safety research. More generally, the goal is to convince people (especially, but not limited to, high-impact decision-makers like politicians, public intellectuals, and leaders in relevant companies) of why it is so important that AI safety research outpace AI capabilities research. Logan has written an excellent post on the upside of pursuing trial-and-error experience in convincing others of the importance of AI alignment. The upside includes, but is not limited to: the development of an optimized curriculum for how to convince people, and the use of this curriculum to help numerous and widely located movement-builders pitch the importance of AI alignment to high-impact individuals. Logan has recently convinced me to collaborate on this plan. Please let us know if you are interested in collaborating as well! You can email me at pspark@math.harvard.edu. I would also like to propose the following additional suggestions: 1) For behavioral-science researchers like myself, prioritize research that is laser-focused on (a) practical persuasion, (b) movement-building, (c) how to solve coordination problems (e.g., market competition, U.S.-China competition) in high-risk domains, (d) how to scale up the AI safety research community, (e) how to recruit AI capabilities researchers into AI safety, and (f) how to help them transition. Examples include RCTs (and data collection in general; see Logan's post!) that investigate effective ways to scale up AI safety research at the expense of AI capabilities research. More generally, scientific knowledge on how to help transition specialists of negative-externality or obsolete occupations would be broadly applicable. 2) For meta-movement-builders, brainstorm and implement practical ways for decentralized efforts around the world to scale up the AI safety research community and to recruit talented people from AI capabilities research. Examples of potential contributions include the development of optimized curricula for training AI safety researchers or for persuading AI capabilities researchers, a well-thought-out grand strategy, and coordination mechanisms. 3) For marketing specialists, brainstorm and implement practical ways to make the field of AI safety prestigious among the AI researcher community. Potential methods to recruit AI capabilities researchers include providing fellowships and other financial opportunities, organizing enjoyable and well-attended AI-safety social events like retreats and conferences, and inviting prestigious experts to said social event...

We will go through each letter of the amusing and memorable acronym and give you our thoughts on ways to make sure each point is addressed, and different methodologies to consider when verifying or assuring that each element has been satisfied before you cite the source.Sarah Blakeslee writes (about her CRAAP guidelines): Sometimes a  person needs an acronym that sticks. Take CRAAP for instance. CRAAP is an acronym that most students don't expect a librarian to be using, let alone using to lead a class. Little do they  know that librarians can be crude and/or rude, and do almost anything in order to penetrate their students'  deep memories and satisfy their instructional objectives.  So what is CRAAP and how does it relate to libraries? Here begins a long story about a short acronym… Discussion Points:The CRAAP guidelines were so named to make them memorableThe five CRAAP areas to consider when using sources for your work are:Currency- timeliness, how old is too old?Relevance- who is the audience, does the info answer your questionsAuthority- have you googled the author? What does that search show you?Accuracy- is it verifiable, supported by evidence, free of emotion?Purpose- is the point of view objective?  Or does it seem colored by political, religious, or cultural biases?Takeaways:You cannot fully evaluate a source without looking AT the sourceBe cautious about second-hand sources– is it the original article, or a press release about the article?Be cautious of broad categories, there are plenty of peer-reviewed, well-known university articles that aren't credibleTo answer our title question, use the CRAAP guidelines as a basic guide to evaluating your sources, it is a useful toolSend us your suggestions for future episodes, we are actively looking! Quotes:“The first thing I found out is there's pretty good evidence that teaching students using the [CRAAP] guidelines doesn't work.” - Dr. Drew“It turns out that even with the [CRAAP] guidelines right in front of them, students make some pretty glaring mistakes when it comes to evaluating sources.” - Dr. Drew“Until I was in my mid-twenties, I never swore at all.” - Dr. Drew“When you're talking about what someone else said [in your paper], go read what that person said, no matter how old it is.” - Dr. Drew“The thing to look out for in qualitative research is, how much are the participants being led by the researchers.” - Dr. Drew“So what I really want to know when I'm reading a qualitative study is not what the participant answered.  I want to know what the question was in the first place.” - Dr. Drew Resources:Link to the CRAAP TestThe Safety of Work PodcastThe Safety of Work on LinkedInFeedback@safetyofwork.com

In this episode of the Side Alpha Podcast, Fire Chief Marc Bashoor speaks with Battalion Chief Keith Stakes, a research engineer with UL's Fire Safety Research Institute (FSRI), about the organization's newly expanded mission, one that focuses on a broader range of emerging fire threats. Stakes shares more about the expanded research efforts at the organization and details some of the research currently underway at FSRI, including the Search & Size-up project. Learn more about UL's FSRI at fsri.org and the organization's expanded focus on FireRescue1.

In this episode of the podcast, series host Jill James interviews Dr. Esteban Tristan. Dr. Tristan holds a PhD in industrial and organizational psychology and is a member of the Society for Industrial and Organizational Psychology and the American Society of Safety Professionals. He's actively involved in applied research and has published articles in professional journals such as the International Journal of selection Assessment, the Journal of Organizational Psychology, the Journal of Safety Research, and EHS today. Come listen to Jill and Dr. Tristan talk about safety DNA and much more!