Podcasts about eleutherai

Play Episode Listen Later Jul 31, 2024 22:41

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Source Automated Interpretability for Sparse Autoencoder Features, published by kh4dien on July 31, 2024 on LessWrong. Background Sparse autoencoders recover a diversity of interpretable, monosemantic features, but present an intractable problem of scale to human labelers. We investigate different techniques for generating and scoring text explanations of SAE features. Key Findings Open source models generate and evaluate text explanations of SAE features reasonably well, albeit somewhat worse than closed models like Claude 3.5 Sonnet. Explanations found by LLMs are similar to explanations found by humans. Automatically interpreting 1.5M features of GPT-2 with the current pipeline would cost $1300 in API calls to Llama 3.1 or $8500 with Claude 3.5 Sonnet. Prior methods cost ~$200k with Claude. Code can be found at https://github.com/EleutherAI/sae-auto-interp. We built a small dashboard to explore explanations and their scores: https://cadentj.github.io/demo/ Generating Explanations Sparse autoencoders decompose activations into a sum of sparse feature directions. We leverage language models to generate explanations for activating text examples. Prior work prompts language models with token sequences that activate MLP neurons (Bills et al. 2023), by showing the model a list of tokens followed by their respective activations, separated by a tab, and listed one per line. We instead highlight max activating tokens in each example with a set of . Optionally, we choose a threshold of the example's max activation for which tokens are highlighted. This helps the model distinguish important information for some densely activating features. We experiment with several methods for augmenting the explanation. Full prompts are available here. Chain of thought improves general reasoning capabilities in language models. We few-shot the model with several examples of a thought process that mimics a human approach to generating explanations. We expect that verbalizing thought might capture richer relations between tokens and context. Activations distinguish which sentences are more representative of a feature. We provide the magnitude of activating tokens after each example. We compute the logit weights for each feature through the path expansion where is the model unembed and is the decoder direction for a specific feature. The top promoted tokens capture a feature's causal effects which are useful for sharpening explanations. This method is equivalent to the logit lens (nostalgebraist 2020); future work might apply variants that reveal other causal information (Belrose et al. 2023; Gandelsman et al. 2024). Scoring explanations Text explanations represent interpretable "concepts" in natural language. How do we evaluate the faithfulness of explanations to the concepts actually contained in SAE features? We view the explanation as a classifier which predicts whether a feature is present in a context. An explanation should have high recall - identifying most activating text - as well as high precision - distinguishing between activating and non-activating text. Consider a feature which activates on the word "stop" after "don't" or "won't" (Gao et al. 2024). There are two failure modes: 1. The explanation could be too broad, identifying the feature as activating on the word "stop". It would have high recall on held out text, but low precision. 2. The explanation could be too narrow, stating the feature activates on the word "stop" only after "don't". This would have high precision, but low recall. One approach to scoring explanations is "simulation scoring"(Bills et al. 2023) which uses a language model to assign an activation to each token in a text, then measures the correlation between predicted and real activations. This method is biased toward recall; given a bro...

LW - Open Source Automated Interpretability for Sparse Autoencoder Features by kh4dien

Latent Space: The AI Engineer Podcast â€” CodeGen, Agents, Computer Vision, Data Science, AI UX and all things Software 3.0

Play Episode Listen Later Jul 31, 2024 22:41

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Source Automated Interpretability for Sparse Autoencoder Features, published by kh4dien on July 31, 2024 on LessWrong. Background Sparse autoencoders recover a diversity of interpretable, monosemantic features, but present an intractable problem of scale to human labelers. We investigate different techniques for generating and scoring text explanations of SAE features. Key Findings Open source models generate and evaluate text explanations of SAE features reasonably well, albeit somewhat worse than closed models like Claude 3.5 Sonnet. Explanations found by LLMs are similar to explanations found by humans. Automatically interpreting 1.5M features of GPT-2 with the current pipeline would cost $1300 in API calls to Llama 3.1 or $8500 with Claude 3.5 Sonnet. Prior methods cost ~$200k with Claude. Code can be found at https://github.com/EleutherAI/sae-auto-interp. We built a small dashboard to explore explanations and their scores: https://cadentj.github.io/demo/ Generating Explanations Sparse autoencoders decompose activations into a sum of sparse feature directions. We leverage language models to generate explanations for activating text examples. Prior work prompts language models with token sequences that activate MLP neurons (Bills et al. 2023), by showing the model a list of tokens followed by their respective activations, separated by a tab, and listed one per line. We instead highlight max activating tokens in each example with a set of . Optionally, we choose a threshold of the example's max activation for which tokens are highlighted. This helps the model distinguish important information for some densely activating features. We experiment with several methods for augmenting the explanation. Full prompts are available here. Chain of thought improves general reasoning capabilities in language models. We few-shot the model with several examples of a thought process that mimics a human approach to generating explanations. We expect that verbalizing thought might capture richer relations between tokens and context. Activations distinguish which sentences are more representative of a feature. We provide the magnitude of activating tokens after each example. We compute the logit weights for each feature through the path expansion where is the model unembed and is the decoder direction for a specific feature. The top promoted tokens capture a feature's causal effects which are useful for sharpening explanations. This method is equivalent to the logit lens (nostalgebraist 2020); future work might apply variants that reveal other causal information (Belrose et al. 2023; Gandelsman et al. 2024). Scoring explanations Text explanations represent interpretable "concepts" in natural language. How do we evaluate the faithfulness of explanations to the concepts actually contained in SAE features? We view the explanation as a classifier which predicts whether a feature is present in a context. An explanation should have high recall - identifying most activating text - as well as high precision - distinguishing between activating and non-activating text. Consider a feature which activates on the word "stop" after "don't" or "won't" (Gao et al. 2024). There are two failure modes: 1. The explanation could be too broad, identifying the feature as activating on the word "stop". It would have high recall on held out text, but low precision. 2. The explanation could be too narrow, stating the feature activates on the word "stop" only after "don't". This would have high precision, but low recall. One approach to scoring explanations is "simulation scoring"(Bills et al. 2023) which uses a language model to assign an activation to each token in a text, then measures the correlation between predicted and real activations. This method is biased toward recall; given a bro...

Benchmarks 201: Why Leaderboards > Arenas >> LLM-as-Judge

Play Episode Listen Later Jul 12, 2024 58:29

The first AI Engineer World's Fair talks from OpenAI and Cognition are up!In our Benchmarks 101 episode back in April 2023 we covered the history of AI benchmarks, their shortcomings, and our hopes for better ones. Fast forward 1.5 years, the pace of model development has far exceeded the speed at which benchmarks are updated. Frontier labs are still using MMLU and HumanEval for model marketing, even though most models are reaching their natural plateau at a ~90% success rate (any higher and they're probably just memorizing/overfitting).From Benchmarks to LeaderboardsOutside of being stale, lab-reported benchmarks also suffer from non-reproducibility. The models served through the API also change over time, so at different points in time it might return different scores.Today's guest, Clémentine Fourrier, is the lead maintainer of HuggingFace's OpenLLM Leaderboard. Their goal is standardizing how models are evaluated by curating a set of high quality benchmarks, and then publishing the results in a reproducible way with tools like EleutherAI's Harness.The leaderboard was first launched summer 2023 and quickly became the de facto standard for open source LLM performance. To give you a sense for the scale:* Over 2 million unique visitors* 300,000 active community members* Over 7,500 models evaluatedLast week they announced the second version of the leaderboard. Why? Because models were getting too good!The new version of the leaderboard is based on 6 benchmarks:*

AF - Some open-source dictionaries and dictionary learning infrastructure by Sam Marks

learning speech infrastructure ea open source dictionary pile l1 anthropic downloading 800m mlp dictionaries rationalist pythia sam marks eleutherai

Play Episode Listen Later Dec 5, 2023 9:58

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some open-source dictionaries and dictionary learning infrastructure, published by Sam Marks on December 5, 2023 on The AI Alignment Forum. As more people begin work on interpretability projects which incorporate dictionary learning, it will be valuable to have high-quality dictionaries publicly available.[1] To get the ball rolling on this, my collaborator (Aaron Mueller) and I are: open-sourcing a number of sparse autoencoder dictionaries trained on Pythia-70m MLPs releasing our repository for training these dictionaries[2]. Let's discuss the dictionaries first, and then the repo. The dictionaries The dictionaries can be downloaded from here. See the sections "Downloading our open-source dictionaries" and "Using trained dictionaries" here for information about how to download and use them. If you use these dictionaries in a published paper, we ask that you mention us in the acknowledgements. We're releasing two sets of dictionaries for EleutherAI's 6-layer pythia-70m-deduped model. The dictionaries in both sets were trained on 512-dimensional MLP output activations (not the MLP hidden layer like Anthropic used), using ~800M tokens from The Pile. The first set, called 0_8192, consists of dictionaries of size 8192=16512. These were trained with an L1 penalty of 1e-3. The second set, called 1_32768, consists of dictionaries of size 32768=64512. These were trained with an l1 penalty of 3e-3. Here are some statistics. (See our repo's readme for more info on what these statistics mean.) For dictionaries in the 0_8192 set: Layer MSE Loss L1 loss L0 % Alive % Loss Recovered 0 0.056 6.132 9.951 0.998 0.984 1 0.089 6.677 44.739 0.887 0.924 2 0.108 11.44 62.156 0.587 0.867 3 0.135 23.773 175.303 0.588 0.902 4 0.148 27.084 174.07 0.806 0.927 5 0.179 47.126 235.05 0.672 0.972 For dictionaries in the 1_32768 set: Layer MSE Loss L1 loss L0 % Alive % Loss Recovered 0 0.09 4.32 2.873 0.174 0.946 1 0.13 2.798 11.256 0.159 0.768 2 0.152 6.151 16.381 0.118 0.724 3 0.211 11.571 39.863 0.226 0.765 4 0.222 13.665 29.235 0.19 0.816 5 0.265 26.4 43.846 0.13 0.931 And here are some histograms of feature frequencies. Overall, I'd guess that these dictionaries are decent, but not amazing. We trained these dictionaries because we wanted to work on a downstream application of dictionary learning, but lacked the dictionaries. These dictionaries are more than good enough to get us off the ground on our mainline project, but I expect that in not too long we'll come back to train some better dictionaries (which we'll also open source). I think the same is true for other folks: these dictionaries should be sufficient to get started on projects that require dictionaries; and when better dictionaries are available later, you can swap them in for optimal results. Some miscellaneous notes about these dictionaries (you can find more in the repo). The L1 penalty for 1_32768 seems to have been too large; only 10-20% of the neurons are alive, and the loss recovered is much worse. That said, we'll remark that after examining features from both sets of dictionaries, the dictionaries from the 1_32768 set seem to have more interpretable features than those from the 0_8192 set (though it's hard to tell). In particular, we suspect that for 0_8192, the many high-frequency features in the later layers are uninterpretable but help significantly with reconstructing activations, resulting in deceptively good-looking statistics. (See the bullet point below regarding neuron resampling and bimodality.) As we progress through the layers, the dictionaries tend to get worse along most metrics (except for % loss recovered). This may have to do with the growing scale of the activations themselves as one moves through the layers of pythia models (h/t to Arthur Conmy for raising this hypothesis). We note that our dictionary fea...

Ep 67: Ellie Pavlick on the Cutting Edge of AI

Brave New World -- hosted by Vasant Dhar

Play Episode Listen Later Sep 7, 2023 70:06

Does ChatGPT really understand anything? Or is it faking it? Hell, do we understand how it works? Ellie Pavlick joins Vasant Dhar in episode 67 of Brave New World to share her thoughts on where AI has reached -- and where it's going. Useful resources: 1. Ellie Pavlick at Brown and Google Scholar. 2. Symbols and grounding in large language models -- Ellie Pavlick. 3. Sam Bowman on ChatGPT & Controlling AI -- Episode 58 of Brave New World. 4. The Nature of Intelligence — Episode 7 of Brave New World (w Yann le Cunn). 5. Music, Mind, and Meaning -- Marvin Minsky. 6. Emergent Abilities of Large Language Models -- Jason Wei et al. 7. Hugging Face. 8. EleutherAI. Check out Vasant Dhar's newsletter on Substack. Subscription is free!

ai hell mind chatgpt intelligence substack subscription cutting edge brave new world yann eleutherai does chatgpt

The Mathematics of Training LLMs — with Quentin Anthony of Eleuther AI

Latent Space: The AI Engineer Podcast â€” CodeGen, Agents, Computer Vision, Data Science, AI UX and all things Software 3.0

Play Episode Listen Later Aug 16, 2023 50:38

Invites are going out for AI Engineer Summit! In the meantime, we have just announced our first Actually Open AI event with Brev.dev and Langchain, Aug 26 in our SF HQ (we'll record talks for those remote). See you soon (and join the Discord)!Special thanks to @nearcyan for helping us arrange this with the Eleuther team.This post was on the HN frontpage for 15 hours.As startups and even VCs hoard GPUs to attract talent, the one thing more valuable than GPUs is knowing how to use them (aka, make GPUs go brrrr).There is an incredible amount of tacit knowledge in the NLP community around training, and until Eleuther.ai came along you pretty much had to work at Google or Meta to gain that knowledge. This makes it hard for non-insiders to even do simple estimations around costing out projects - it is well known how to trade $ for GPU hours, but trading “$ for size of model” or “$ for quality of model” is less known and more valuable and full of opaque “it depends”. This is why rules of thumb for training are incredibly useful, because they cut through the noise and give you the simple 20% of knowledge that determines 80% of the outcome derived from hard earned experience.Today's guest, Quentin Anthony from EleutherAI, is one of the top researchers in high-performance deep learning. He's one of the co-authors of Transformers Math 101, which was one of the clearest articulations of training rules of thumb. We can think of no better way to dive into training math than to have Quentin run us through a masterclass on model weights, optimizer states, gradients, activations, and how they all impact memory requirements.The core equation you will need to know is the following:Where C is the compute requirements to train a model, P is the number of parameters, and D is the size of the training dataset in tokens. This is also equal to τ, the throughput of your machine measured in FLOPs (Actual FLOPs/GPU * # of GPUs), multiplied by T, the amount of time spent training the model.Taking Chinchilla scaling at face value, you can simplify this equation to be `C = 120(P^2)`.These laws are only true when 1000 GPUs for 1 hour costs the same as 1 GPU for 1000 hours, so it's not always that easy to make these assumptions especially when it comes to communication overhead. There's a lot more math to dive into here between training and inference, which you can listen to in the episode or read in the articles. The other interesting concept we covered is distributed training and strategies such as ZeRO and 3D parallelism. As these models have scaled, it's become impossible to fit everything in a single GPU for training and inference. We leave these advanced concepts to the end, but there's a lot of innovation happening around sharding of params, gradients, and optimizer states that you must know is happening in modern LLM training. If you have questions, you can join the Eleuther AI Discord or follow Quentin on Twitter. Show Notes* Transformers Math 101 Article* Eleuther.ai* GPT-NeoX 20B* BLOOM* Turing NLG* Mosaic* Oak Ridge & Frontier Supercomputer* Summit Supercomputer * Lawrence Livermore Lab* RWKV* Flash Attention * Stas BekmanTimestamps* [00:00:00] Quentin's background and work at Eleuther.ai* [00:03:14] Motivation behind writing the Transformers Math 101 article* [00:05:58] Key equation for calculating compute requirements (tau x T = 6 x P x D)* [00:10:00] Difference between theoretical and actual FLOPs* [00:12:42] Applying the equation to estimate compute for GPT-3 training* [00:14:08] Expecting 115+ teraflops/sec per A100 GPU as a baseline* [00:15:10] Tradeoffs between Nvidia and AMD GPUs for training* [00:18:50] Model precision (FP32, FP16, BF16 etc.) and impact on memory* [00:22:00] Benefits of model quantization even with unlimited memory* [00:23:44] KV cache memory overhead during inference* [00:26:08] How optimizer memory usage is calculated* [00:32:03] Components of total training memory (model, optimizer, gradients, activations)* [00:33:47] Activation recomputation to reduce memory overhead* [00:38:25] Sharded optimizers like ZeRO to distribute across GPUs* [00:40:23] Communication operations like scatter and gather in ZeRO* [00:41:33] Advanced 3D parallelism techniques (data, tensor, pipeline)* [00:43:55] Combining 3D parallelism and sharded optimizers* [00:45:43] Challenges with heterogeneous clusters for distribution* [00:47:58] Lightning RoundTranscriptionAlessio: Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO in Residence at Decibel Partners, and I'm joined by my co-host Swyx, writer and editor of Latent Space. [00:00:20]Swyx: Hey, today we have a very special guest, Quentin Anthony from Eleuther.ai. The context for this episode is that we've been looking to cover Transformers math for a long time. And then one day in April, there's this blog post that comes out that literally is called Transformers Math 101 from Eleuther. And this is one of the most authoritative posts that I've ever seen. And I think basically on this podcast, we're trying to give people an intuition around what are the rules of thumb that are important in thinking about AI and reasoning by AI. And I don't think there's anyone more credible than the people at Eleuther or the people training actual large language models, especially on limited resources. So welcome, Quentin. [00:00:59]Quentin: Thank you. A little bit about myself is that I'm a PhD student at Ohio State University, starting my fifth year now, almost done. I started with Eleuther during the GPT-NeoX20B model. So they were getting started training that, they were having some problems scaling it. As we'll talk about, I'm sure today a lot, is that communication costs and synchronization and how do you scale up a model to hundreds of GPUs and make sure that things progress quickly is really difficult. That was really similar to my PhD work. So I jumped in and helped them on the 20B, getting that running smoothly. And then ever since then, just as new systems challenges arise, and as they move to high performance computing systems and distributed systems, I just sort of kept finding myself falling into projects and helping out there. So I've been at Eleuther for a little bit now, head engineer there now, and then finishing up my PhD and then, well, who knows where I'll go next. [00:01:48]Alessio: Awesome. What was the inspiration behind writing the article? Was it taking some of those learnings? Obviously Eleuther is one of the most open research places out there. Is it just part of the DNA there or any fun stories there? [00:02:00]Quentin: For the motivation for writing, you very frequently see in like the DL training space, like these Twitter posts by like, for example, like Stas Bekman at Hugging Face, you'll see like a Twitter post that's like, oh, we just found this magic number and everything is like 20% faster. He's super excited, but doesn't really understand what's going on. And the same thing for us, we very frequently find that a lot of people understand the theory or maybe the fundamentals of why like AI training or inference works, but no one knows like the nitty gritty details of like, how do you get inference to actually run correctly on your machine split across two GPUs or something like that. So we sort of had all of these notes that we had accumulated and we're sort of sharing among engineers within Eleuther and we thought, well, this would really help a lot of other people. It's not really maybe appropriate for like a paper, but for something like a blog post or technical report, this would actually maybe squeeze a lot of performance out of people's hardware they're already running on. So I guess there are a lot of projects in Eleuther that we're sort of trying to share notes with people in a way that typical institutions don't. They sort of live within that institution and then you go to a different institution and they do something very similar, but without the lessons of the previous. And it's because everyone's trying to do their own special sauce with their own stack. Whereas Eleuther, we don't really have that constraint and we can just share everything to everybody. [00:03:14]Swyx: Yeah, this is a level of openness that basically very few people actually embrace. One, it's an extra effort to write things down, of course, but two, it is secret sauce and so that not many people do it. And therefore, oftentimes the only way to learn this stuff is to actually work in one of the large model labs. And so you guys are doing a lot. The only other instance where I can think of where people actually open sourced their process was Facebook's OPT. What else is similar, like sort of trade knowledge, but not formal research knowledge? [00:03:45]Quentin: I would say Bloom. So the Hugging Face Bloom project in big science and all of that, that was very open. I'd say it's the same caliber, if not more detailed than OPT. Other than that, I think there was like a doc from Microsoft on like their Turing NLG. Their paper is pretty relaxed in that it did talk about some of those challenges. Other than like OPT and Bloom and us, I can't think of any. It's a new thing. [00:04:10]Swyx: It matters that you are going for the sort of good enough rules of thumb, because I think a lot of people try to go for precision and being overly precise actually is not helpful. Right. Yes. [00:04:20]Quentin: You'll see some like statements in the blog posts that are just like, we think this is about 1.2 in our experience. And, you know, we don't go any further into detail and it would take maybe an extra month for us to chase down every single little piece of memory. But instead, like getting good enough is still helpful to people. [00:04:36]Alessio: Let's jump into it. The first part of the article, and we'll put this in the show notes so people will be following along with the post. So we don't need to read every single equation and every footnote for it. [00:04:46]Swyx: Okay. [00:04:46]Alessio: But the core equation here is that not the cost of compute, but the compute required to turn a transformer model is roughly equal to tau times T, where like T is the, where tau is the hardware setup throughput that you have. So number of GPUs times the actual flops per GPU. And then T is the time spent. I think people can visualize that pretty easily. It's basically like how many GPUs do you have and how much do you let them run for? And the things that come to it that people have read before in the Chinchilla paper in a way, and the OpenAI scaling law is that you can then equal this to 6PD, where P is the number of parameters in the model and D is the size of the, of the dataset in tokens. So talk a little bit about how people should think about the two. I think a lot of times the focus is on tokens parameter ratio in the training dataset and people don't think as much about the actual flops per GPU, which you're going to mention later in the blog post too, in terms of how much you can get out. So how should people think about this when they're building a model and where should they go to this equation as they're starting to think about training their own transformer-based [00:05:58]Swyx: model? [00:05:58]Quentin: You touched a little bit on the fact that people usually start with the dataset. So you have some dataset that you want to train a model on. And then from there, from the 6PD, you should see, okay, I should have about six tokens per parameter. So that determines my model size thereabouts for Chinchilla Optimal. So since then we've seen that need more something like 20 or more than that to get a good quality model. But the next question that should be on your mind in terms of a systems perspective is how long is it going to take for this model to train and what kind of budget should I expect? So let's say I want some cloud instance for some amount of time and each of them will have some price attached to it. So that's where the throughput comes in. So now that you have this model, this number of parameters, you should map that to a transformer architecture and you should benchmark what throughput you get on your software stack for that type of model. So now you have your flops per second on a single GPU. And then given whatever parallelism scheme, which I'm sure we'll get into, like data parallelism or tensor parallelism or whatever else, how is that flops number going to scale to whatever number of GPUs? And then from there, you're going to get a time. And if you have a time, you have a cost. Those are like the business answers that you'll be able to get using this formula. That's why we sort of split it into the T and the throughput terms so that you can solve for one of them, which is usually get throughput, need time, and from time you get cost. In a nutshell, that's the answer. [00:07:19]Alessio: One thing that I noticed, you mentioned some of these laws are only true when a thousand GPUs for one hour cost the same as one GPU for a thousand hours, given that we have a shortage of the biggest GPUs out there. Any thoughts there on how people should prioritize this? [00:07:36]Quentin: Yeah, so I would say you should find what the minimum number of GPUs is to just fit your model first. The memory bottleneck is your biggest problem if you have a sizable model. If it's a small model, nobody cares. But most models that people care about will need to be split across multiple GPUs. So find the minimum number of GPUs to just fit your one instance of your model and then calculate how long that's going to take. If it's a reasonable amount of time, then you're done. If it takes too long, then you need to start worrying about having multiple instances of that model. I always feel like you should go with the minimum number of GPUs because the more number of GPUs that you have, the more likely it is for things to break. So I would say just find out what time is reasonable for you and then fit the number of GPUs to that and no more. Because people get greedy and they say, if I have twice the GPUs, I can get this done in half the time. And then you end up taking three times the time because everything is breaking every day. And that's when I am up at midnight trying to fix your model that's broken. [00:08:34]Swyx: We had a previous guest which has invested a lot in their framework for training these things. Would there not be an equivalent open source framework you guys would have made that would help with scaling up GPUs linearly like that? Or is this an oversimplification? [00:08:50]Quentin: Okay, yeah. So maybe I should step back. Both Mosaic and us have our own sort of software stack recipe that scales well, theoretically. But I'll get to that in a minute. Mosaic is all based off optimizer sharding. So it's based off ZeRO. So you basically perfectly split your model optimizer and your parameters and your gradients across all of the different GPUs. So your aggregate memory is number of parameters divided by number of GPUs. Same thing for optimizer and so on. Whereas we at Eleuther use a Megatron deep speed based library. And for that, it's a bit more complex. So the efficiency can be a little higher, but it's more prone to failure at the same [00:09:30]Swyx: time. [00:09:30]Quentin: So you kind of have to tune it. In both cases, getting back to like the practical case, you should be able to get linear speed up by adding more GPUs. The problem is that there are hardware failures. You tend to have problems with like maybe loss will overflow if you have too many GPUs or maybe one GPU will hang. You might have software issues. You might have synchronization issues. And that's why I'm saying practically that you should take the minimum number of GPUs that you have because those are the easier cases to debug. That make sense? [00:10:00]Swyx: Yeah. [00:10:00]Quentin: Any more detail on any specific point? [00:10:02]Swyx: Not particularly, just because we haven't actually had to debug those things. But I imagine basically there's a lot of return towards encoding these knowledge into software and not repeating it again. So it makes a ton of sense. I think Alessio had more questions before we move too far into high level, more questions on just the equation itself. I think we want to spend time on essentially, this is the central equation of figuring out compute requirements. Yeah. [00:10:25]Alessio: Another thing in it is that the computer is like the forward pass and like the backwards pass and forward is 2PD, backward is 4PD. Why it's to the ratio between the two? Can you explain that? Why is it two and four? [00:10:39]Quentin: Yeah. [00:10:40]Alessio: Why is it twice the amount? [00:10:42]Quentin: Oh, okay. Intuitively for forward pass, you're just moving, you're propagating forward the inputs through the layer. And then in the backward pass, you're doing something a little more complex than that. You're doing back propagation. And I don't think I can explain it intuitively enough to go into more detail on the exact [00:10:58]Swyx: numbers. Yeah. [00:10:58]Quentin: That's okay. [00:10:59]Swyx: I feel like you want to get out a whiteboard and start drawing like, you know. [00:11:02]Quentin: That's what I would normally do. [00:11:03]Swyx: Tangents and gradients. It's actually surprisingly low to do the back propagation. Honestly, that's one of the fundamental things I love about the math of deep learning so far that as I've explored it, which is, it's surprisingly efficient as compared to other, I guess, numerical methods you might be exposed to and, you know, college calculus. Yeah. [00:11:22]Alessio: And I think the other thing is that things sound simple, you know, when people go on Twitter and say, Oh, 20 is like the optimal ratio. And it's like, then it's like, well, why is that the number? And the answer is usually much, much harder, like what we're seeing right now. So I think it's a, it's a good reminder that the numbers are simple, like all the best and most popular, like math equations are like, so elegant. Obviously the proof behind that is, it's not that easy. That's always a good reminder. [00:11:52]Swyx: I want to put this equation to the test a little bit. We can do this from either GPT-3's perspective or GPT-NeoX, whatever you're more comfortable with. You have this distinction of actual flops versus theoretical flops. And a lot of times when people report the flops it took to train a model, like we just saw one in Lama 2 where the estimate is something that the amount of flops and that's, that's what we go with. So GPT-3 took a 3.14 times 10 to the power 23 flops. That is the theoretical flops. I want to get to a point where I can sort of work out if a number passes the smell test. And I wonder how to do that because I should be able to plug in this equation, right? I know that GPT-3 was trained on 300 billion tokens. I know the parameter size of 175. Is it, is it just like a 6 times 175 times 300? Like I haven't done the math, but what are the nuances here that you might want to call out? [00:12:42]Quentin: Theoretical flops is usually given from, you have a given set of hardware and this is what you expect your hardware to get. The problem is that in practice, full utilization, that's the key word, right? Because in practice, there are a lot of cases where like you're spending time waiting on data movement from like the GPU to CPU. Or for example, you might be waiting to synchronize across the different GPUs. So there's a lot of idle time basically that you're going to be spending during training. [00:13:05]Swyx: Smell tests. [00:13:06]Quentin: I don't know if I have a smell test myself, to be honest, like maybe I'll look at like what sort of flops, what you would expect on like an A100. There's sort of just an expected flops for a given GPU that everyone sort of knows what you should expect. So like for an A100, that number is somewhere between 100 and 180. T flops is what you would expect to see on an A100. For a V100, like an older GPU, it's something more like 40 to 30. So people sort of know, given the kernels that we're running for a deep learning, what sort of flops you expect. And then you sort of compare that to the theory, to the theoretical flops that people are reporting and see if that matches your expectations. [00:13:47]Swyx: Yeah. [00:13:47]Alessio: And in the article you mentioned for the A100, like if you're seeing below 115 teraflops a second, there's something wrong with your model or hardware. How did you get to 115? Is it just, you know, production observability and like you've seen over months and months and months that like that's the baseline or how do you come up with the numbers like that? Yeah. [00:14:08]Quentin: For a number like that, we basically, we compared a lot of different frameworks. So like I mentioned before, Mosaic has their own framework and we have our own framework. They all have their own flop counters too, right? And we saw across a bunch of different hardware configurations that if you tune things correctly, you should be getting above 115 in pretty much all cases. So like there are some cases where things are tuned poorly or your system is a little weird, but we've never been able to get a new system and not been able to get above [00:14:35]Swyx: 115. [00:14:35]Quentin: If something is below 115, you have something really wrong in your software. But that's really all it is, is just comparing across software stacks and hardware systems. [00:14:44]Alessio: What about different GPUs? We had George Hotz on the podcast and he talked about AMD cards and how in theory their flops should be much better than some Nvidia cards, but the reality is like the CUDA runtime makes up for it. How should people think about improving that? You know, like do you see, okay, the A100 is like 115 teraflops. I'd rather just stick with this than try and figure out all the kinks of like a better AMD card or any thoughts there? [00:15:10]Swyx: Right. [00:15:10]Quentin: Well, that's sort of touching on developer time, right? And which ends up being more expensive because at the end of the day, the AMD and Rockham software stack has a long way to go. I would say most things run there, not particularly efficiently, but you're going to have weird bugs that no one has encountered before. One of the big pluses of going with the Nvidia and PyTorch stack is that there are thousands of GitHub issues with everyone facing the same problem as you and resolving them quickly and in an open source way is probably the biggest benefit of going with the Nvidia software stack right now. AMD has about the same hardware, software, not so much. And they haven't quite got the momentum in the open source realm, for example, to get close. Like something, for example, like Flash Attention, it's spread to more Nvidia GPU types than it has like to AMD at all. And waiting on those latest and greatest features to reach AMD is something that's prohibitive to a lot of people, but it's getting there. I'm running a lot of experiments on AMD right now because it's sort of reached the government lab supercomputers now. And so a lot of experiments are going there and it will catch up, I'd say within a few [00:16:14]Swyx: years. [00:16:14]Quentin: Awesome. [00:16:15]Swyx: Maybe just talk about what's available from the government labs and I heard the original, the origin of Eluther started with a grant for TPUs. Is that right? [00:16:24]Quentin: Yes, that was a little before me, but there was a lot of just like getting a grabbing a Google Cloud or TPU pod or something like that is a lot of the original TPU work on Mesh TensorFlow, which is like now like an ancient distributed deep learning library. [00:16:36]Quentin: Eluther got a grant, an insight grant with Oak Ridge last year, and we got quite a bit of Summit Compute. So Summit is a V100 based supercomputer. It's got some weirdness to it. So there's six V100 GPUs per node. And we did a lot of experiments there. It's a challenging system to scale to because your interconnect across nodes is kind of slow in comparison to within a node, which I think we'll get to later. But now Oak Ridge has moved to AMD. So the next grant that we're trying to work towards is on Frontier, which has four AMD GPUs per node and again has a slower interconnect across nodes. So we get all of those new challenges again to try and overlap things. But that's just like you have Oak Ridge, you have Lawrence Livermore. There's a lot of government supercomputers that you can apply for compute towards like open researchers too. It's sort of a new thing. I think we're one of the first like us and like Lion, for example, is another organization that's getting compute from government providers and such. They're all moving to AMD as well. And we look forward to exploring that with them. [00:17:42]Swyx: Yeah. [00:17:43]Alessio: The computing is definitely, it used to be easy to find the GPU. Now, not as much. So you got to find them anywhere. [00:17:49]Swyx: Yes. [00:17:49]Alessio: Let's talk about memory requirements a little bit. So you touched on this a little bit before and just before this, we had a trade out on the pockets from FlashAttention and memory speed was one of our main focuses, but this time we're being bound by actually memory size, like the VRAM itself, when it comes to model weights and parameters and optimizer states and all that fun stuff. Let's go through this and Sean, we can, we can take turns. There's a lot to cover here, but maybe we can start from model weights. So one topic we covered a lot in the past is precision and quantization. That's one of the obviously main driver of memory. You mentioned most of, in the article, most transformers are mixed precision, like FP16 plus FP32 or BF16 FP32, and they can be cast down. And you mentioned up to like INT8 without a lot of performance hit. So let's start there and maybe run people through some of the maths and like the byte per parameter ratio and different precision. [00:18:50]Swyx: Sure. [00:18:51]Quentin: So when I started deep learning, it was all FP32. You have 32 bits, four bytes per parameter. Things were pretty simple. You didn't have to do any loss scaling at all. But the problem was that you didn't get a whole lot of flops once NVIDIA moved to V100s and introduced Tensor cores. So Tensor cores do all of their computation at FP16 precision. So you're kind of throwing all of those away if you're doing things in FP32. So once the hardware moved to V100, the software moved to like mixed precision and APEX and AMP and such. And one counterintuitive part of mixed precision is that you actually require more memory when you're trained because you need an FP16 copy of the weights and an FP32 copy of the weights. The FP16 copy is where you're doing like your actual computation on the Tensor cores. So you get maybe it's not uncommon to get double the throughput that you would see before in FP32. And then you at each step update that FP32 copy with the FP16 update. So both need to be stored in memory. The problem with that is that FP16 is very precise but doesn't have a whole lot of range, [00:19:55]Swyx: dynamic range. [00:19:55]Quentin: So you have a really big mantissa if you're thinking in terms of like floating point representations, not a whole lot of exponent. So BF16 puts more of the bits from the mantissa back to the exponent. So you have a much higher range and a lower precision. And that gets rid of all of this instability problem and loss scaling and such that anyone familiar with debugging knows how unstable it can be, especially for large scale training. And BF16 does away with a lot of that, but it's only supported on A100s. So you see the back and forth between hardware and software. So every time NVIDIA introduces some new Tensor cores or BF16 support or something like that, the software adapts to support it and then training adapts. And then now you mentioned like Ind8 and such. Now we're seeing that you have some model that's been trained in FP16, FP32, whatever else. And then now you want to, with minimal loss and accuracy, quantize that model into a smaller representation like Ind8 and now like Ind4 and things like that and see what you can get away with. And then since deep learning is such like a stochastic problem that a lot of those last bits of precision don't really matter is what we're finding. And I expect that to continue. [00:21:06]Alessio: And so just to put some numbers to it, when you have a FP32, you need four bytes per parameter at inference time to load it in memory. If you have a eight bits model quantized down, you need one byte per parameter. So for example, in an H100, which is 80 gigabyte of memory, you could fit a 70 billion parameters in eight, you cannot fit a FP32 because you will need like 280 gigabytes of memory. So how much does that play into it? Like you mentioned it was all FP32 when you first started. Is it just like a development complexity thing, like going down to FP16 and then Ind8? Or if they could get a GPU with like a terabyte of VRAM, will people just load this memory as like FP32 weights or would they still want to quantize them to make them more efficient? Right. [00:22:00]Quentin: I would say even if you had infinite VRAM, you would still want a quantized model, just a bigger model that's quantized is what I would say. And that's because like I was mentioning there at the end, how like deep learning is very stochastic and a lot, you could have all the precision in the world, but ultimately it's meaningless when you still depend so much like on what the input is. And you depend so much on little variations and maybe a few more samples of training data would matter more. A lot of that precision in a nutshell doesn't really matter in deep learning. All that matters is the big picture. What is that neuron actually saying? And not the tiny details of what it might be thinking. Oh, I also wanted to mention that even if you have an A100, the actual model size is quite a bit smaller that you could load than what you mentioned. That's because of the KV cache. So the KV cache intuitively during inference, it only matters during inference and think intuitively if you're writing a paragraph, you want to remember every single previous word that you've written before you write the next word. So like what is autoregressive language modeling? It's filling in the next word, the next token. So if I say like the dog went to the, and I need to write the next word, I would say park or something. Before I write the next word, my memory is wiped and I have to read the whole thing again. That is life without a KV cache. And a KV cache says, remember everything that I've generated before, as well as all the context before what I've generated. But the memory overhead for a KV cache commonly is either comparable or larger than the model in some cases, if you have a really long context. And I think the exact equation is something like, oh, it's like two times the number of layers, times the number of heads, times the dimension of each head. And then there's two of those. You have one for K, one for V. But that was just a quick aside. Yeah. [00:23:44]Alessio: I know this is Transformers math, but do you think one of the interesting things about RNNs too, it's like moving away from this, like KV cache, the scales with the sequence length and having like a fixed sequence pass. I know those are some of the things that people are working on. [00:24:00]Swyx: Yeah. [00:24:00]Quentin: So there's a paper that I was involved with called RWKV that I would recommend people read. It is answering this exact question. So how do you get Transformers quality without this quadratic attention overhead that Transformers requires? So it is interesting. I don't know if I can really dive too deep into the technical details there. I'd recommend people read the paper. But yeah. [00:24:23]Swyx: Yeah. [00:24:23]Alessio: It's interesting to see if attention is all you need, or maybe attention is all we need, but we need better ways to make it infer in a good way. [00:24:33]Swyx: We've actually done an unreleased episode with one of the RWKV core members and they call it soft attention or light attention. I forget what they call it, but yeah, just ways to approximate it such that it's linear and not quadratic. That's great. Yeah. [00:24:47]Quentin: I didn't know that you were involved. [00:24:48]Swyx: That's great. How did you get involved? Is it just because like everyone just hangs out in Discord and talks about the future of Transformers? Oh yeah. [00:24:55]Quentin: I mean, the RWKV people specifically are in Eleuther all the time. Like they're very close collaboration with us. And my contribution was we have all of these experiments done by all of these people on RNNs and how they relate to Transformers and how do we turn that into a paper and disseminate that digestibly so that people don't have to read through like a Discord log from a year ago to understand what's going on. [00:25:16]Swyx: Oh my God. [00:25:16]Quentin: Just read this paper. So that took some work, but I wasn't a core contributor. So that's why I don't want to go into like the technical details. But yeah, that's how I did. [00:25:24]Swyx: We'll try to get that RWKV episode out. It seems like there's increasing mentions of it and they are doing pretty important work as far as scaling these models are concerned. Okay. So we discussed inference type quantization and memory requirements. And then you also had a section on training with a lot of stuff I think mentioned. I think we probably want to spend the most of our time on optimizer states and the Atom optimizer. Yeah. What are your takes on it and what should people keep in mind when they deal with these optimizers? Okay. [00:25:57]Quentin: I would say the Atom optimizer is good at what it does. It's sort of a broad question. So let me think. You have the copy of the weights and then you have your momentum and your variance that [00:26:08]Swyx: you store. [00:26:08]Quentin: And like, okay, maybe an intuitive explanation for momentum is that like, let's say you have a canyon and you're trying to get to the bottom. And if you're just doing basic SGD, then every step is going to be an equal size. Whereas if you're using something like Atom with the momentum term, then your steps should be progressively larger because you can see, oh, the general trend is we're heading downwards very quickly. But stepping back from that, since you have all of these extra terms in Atom, you require a lot more memory to store it. Like three times as much memory as SGD. And if you have all of this memory being spent on your optimizer states, then how do you distribute it across GPUs? Because you'll find that what ends up being your bottleneck more than just raw compute, raw flops on a given GPU is your parallelism. And that falls back onto how much model you can fit on a single GPU before you need to split it up across a bunch of GPUs. And then you end up spending time, more time with them talking to each other than actually making progress. So that's why all of this time in the blog post is spent on how do you distribute your model? What are all those different distributed strategies look like? Which ones are more efficient? And given that a lot of your memory is being spent optimizers, how do you distribute that optimizer specifically? Because a lot of people, when they talk about parallelism, they talk about model parallelism, the parameters themselves. In actuality, when you're training, a good portion of your memory is actually spent on optimizer states. So what specific part of that would you like to go into? Would you like to go into like zero or sharded optimizers? [00:27:36]Swyx: I think the sharded optimizer stuff is really interesting, but I think we're kind of leaving that towards the end, right? Because that's the maybe more advanced distributed sections. Here, I think we're just going for rough intuition for people who've maybe are familiar with the ideas of these optimizers, but haven't actually had to implement them yet. They read your code, but they don't really understand the intuition behind the code. I see. [00:28:00]Alessio: And Quentin, when you say in the blog post, it says, Adam is magic. How much of it is like actual magic, even to like people like you that are pretty close to the metal, so to speak? Are some of these things just come as gospel? It's like, I know this works, like I'm not touching it. I'm just leveraging it. How much of it are you actually thinking about improving on in your day-to-day work? I see. [00:28:22]Quentin: So I'm a systems guy. I'm an engineer. And a lot of these things come to me as magic. Adam comes to me as magic. I see it from the gods. I say, this is how a deep learning model is trained. And this is how the next step is calculated. And then I say, okay, how do I make that fast? I would say I do look at ways to improve upon it using things like second order optimizers. So there's a lot of research on there because they're hard to distribute. But the core contribution for me always comes down to someone else has done like some deep learning optimization and I need to make it run fast. So I can't really speak to the motivation of why Adam came about other than like simple, intuitive things like I mentioned with like the momentum. But what matters to me is that Adam takes more memory than SGD, specifically three times. And all of that memory needs to go somewhere and it needs to be split efficiently. [00:29:14]Swyx: Yeah. [00:29:14]Alessio: So when you add them all up, you got 12 bytes per parameter with vanilla Adam. [00:29:20]Swyx: Yeah. [00:29:20]Alessio: And then you still get the model parameters and memory too. So as you mentioned, you need to keep a copy of both for like a FB32, FB16 mixed, a copy of both quantization levels. So there's precision levels. So it's six bytes per parameter. Right. [00:29:36]Quentin: Taking a step back again, is that like, okay, most people think of your model getting big. So you need to split with model parallelism purely, something like tensor parallelism. But we can see that the model only takes like two bytes per parameter if we're doing FB16. Whereas the optimizer itself requires four bytes per parameter for the model states, four bytes for momentum, four bytes for variance. So what matters more is how do you split your optimizer efficiently and how do you store it efficiently? And something like bits and bytes, where the optimizer, you got like eight bit Adam, where those optimizer states is only one byte per parameter instead of four or something like that. That is going to give you a much better return on your model training and on your memory overhead required than if you were to, for example, quantize your pure like FB16 model weights down to int8 or something. So for training specifically, your optimizer memory matters a lot. The most in most cases. [00:30:31]Swyx: Well, yeah. [00:30:31]Alessio: And before we dive into zero, just to wrap up the items that you're going to shard later. So you have the parameters, you have the optimizer states, and then you have the gradients. Just maybe touch a little bit on that. And then we can talk about how to efficiently load them in GPUs. [00:30:48]Quentin: So the parameters are the FP32 copies of the parameters. We include them in the optimizer discussion. Some people don't, but just for clarity, it's 12 bytes per param for the optimizer states and four of them are for that FP32 copy of the weights. Four of them are for the momentum. I already went into why it's important to store momentum, but that's also per parameter. You need to store where that parameter is going and where it's been going in the past. You also need to know, okay, we know where it's going, but there's going to be bumps on this canyon that we're going down. So we need to store its variance. How often are those bumps? Should we be focusing more on the momentum? Or is this parameter just kind of jumping around everywhere? Those are all important answers that we need the optimizer to store, and it's per parameter. So that's where all three of those terms come from. And we also include some competing bits and bytes, for example, an SGD to show that depending on your optimizer, you may store all or none of these and in different representations. [00:31:50]Alessio: I'm looking at the total training memory. You essentially have model memory, optimizer memory, gradient memory, and activation memory. I think that's one of the last discussed things. So maybe just give people a little bit of a view. [00:32:03]Swyx: Yeah, this is completely new to me. [00:32:05]Alessio: Active, you know, recomputation, checkpointing, and all of that. [00:32:08]Swyx: Right. [00:32:09]Quentin: So, okay. So to summarize before activation checkpointing, which will be complicated, you have your model params, like I mentioned before, they used to be FP32. Now they're probably BF16, maybe FP16 if it's an older GPU. Then you have your optimizer. That's where a lot of the memory is going. And it's your high precision, usually FP32, copy of the weights. So that's four bytes per param. And then you have, optionally, a couple more terms like we just discussed, like momentum or variance or whatever else, depending on what your optimizer is. Then you have your gradients. So your gradients is what is the gradient update that we get after running the forward pass on the model. And that's going to be whatever your low precision copy of the weights is. So like two bytes per param, if you're using FP16 or BF16. And all of those are sort of set in stone. And that overhead is not going to go away for the duration of training. Your gradients might get cleared after you back propagate them, but your optimizer states and your model states aren't going away. That memory overhead will be there. Activation recomputation and activation memory is dynamic. So some people will come and have this problem where the model loads fine for training. But then when you actually run your first iteration, or you run some future iteration or something like that, you run out of memory, seemingly at random. And it's because of these activations that you're computing on the fly. Good summary, or do you want to get into activation recomputation now, or do you want me to touch on anything else? [00:33:35]Alessio: Yeah, I was going to say, when is the recomputation happening? How does it decide between recomputing versus storing? And talk a bit more about that, maybe. [00:33:47]Quentin: Yeah, okay. So there's a lot of different ways to do this, but I would say there are a few main ones. First is a very simple scheme. You recompute everything. Every single activation that you calculate is just going to be either used or thrown away until the end. So in that case, you care very much about memory. You care very little about compute. Maybe this would be a case where you have to distribute across a lot of different GPUs, for example. And your communication speed is really low. Then that might be a good case for you to just recompute everything. It happens rarely, but it happens. Next up would be something like selective recomputation. So in selective recomputation, which Megatron has a good paper on, and I believe the figure that we have in our blog post is from, in that case, you sort of do a weighted decision for each activation. So for really big activation tensors, you decide, is this going to be more expensive to save in terms of memory or to recompute in terms of compute? So that's sort of the smart scheme that Megatron implements. And there's a lot of different heuristics they use. It's probably not worth mentioning off this super long equation on a pod, but you should go and read that paper if you're interested on selective recomputation. And then a really stupid scheme that most people go with, including NeoX, would be something like, instead of doing all of these heuristics, you just say, if my tensor is bigger than X, I throw it away. And you set X to some static number, and that's it. And that is good enough for a lot of cases. [00:35:18]Swyx: Why is it good enough? [00:35:20]Quentin: You don't want to store more than, you know, X-sized tensor. And some fall above that, some fall below it. And you're not trying to squeeze. You care more about getting something close enough to what the actual heuristic should be without actually computing the heuristic because you don't want to spend the time writing that heuristic code. [00:35:37]Swyx: Cool. I think that does take us on a grand tour of the memory math. Is there any sort of high-level takeaway before we go into the distributed stuff? Zero and all that. Perhaps more detail than most people have ever encountered. And so I'll repeat the equation that Alessio mentioned again, which is total training memory now has all these components that you've mapped out for the first time as far as we're concerned. Model memory, optimizer memory, activation memory, gradient memory. We covered quite a few algorithms as to the choices you can make there. Anything else that you want to mention about just memory math? I don't think so. [00:36:11]Quentin: I think that about covers it. I will say that it's a very different scheme for training and inference. It's common for people to say, oh, BF16 is the best. Done. Whereas a more correct take is that during training, precision matters a bit more. So BF16 will be around longer for training than it will for inference, in which case your model is sort of already baked. And it definitely doesn't need some of those last bits of precision so you can get away much easier with going to int8 for inference rather than training. So everything that you learn for training has to be relearned for inference and vice versa. [00:36:44]Swyx: There's a third category. You're talking about training versus inference. This third category is emerging with regards to fine-tuning and perhaps parameter-efficient methods of fine-tuning. The naive way to implement fine-tuning is just to do more training. But I don't know if you've developed any intuitions over fine-tuning that's worth inserting here. Any intuitions? If you were to write fine-tuning math, what would go in there? That might be an interesting diff to training math. [00:37:10]Quentin: I think there's a lot of questions that are unanswered for fine-tuning. For example, we know scaling laws for training. And some people have done scaling laws for fine-tuning. But how does a model that's already been trained on one domain transfer to another in terms of fine-tuning size? How many tokens per parameter should you have for your fine-tuning dataset? Maybe I'm ignorant, but I feel like a lot of those sort of practical questions on how a model can transfer and how a model can learn or grok some new ability that wasn't in its original training dataset is something that I would definitely put inside a fine-tuning blog post. [00:37:45]Swyx: Something related to perplexity and, I guess, diversity of the tokens that you get. [00:37:49]Quentin: Yeah, sort of dataset transfer is something that I would be curious in. Learning rate transfer is another one. So your model has some decayed learning rate over the course of training. How does that change for fine-tuning? Things like that. [00:38:00]Swyx: All right, cool. Thanks for indulging that stuff. Sure. Yeah. [00:38:03]Alessio: I think after all of this, you can quickly do the math and see that training needs to be distributed to actually work because we just don't have hardware that can easily run this. So let's talk a bit about that. So zero is one of the first things that you mentioned here, which is focused on sharded optimizers. Maybe run people through that and how to think about it. [00:38:25]Swyx: Sure. [00:38:25]Quentin: So zero is centered around two communication operations. And the first is scatter. And people should be looking at the zero figure that I think we have. [00:38:35]Swyx: Yeah. [00:38:36]Quentin: So there's a figure in the paper with parameters, gradients, and optimizer states that people should be looking at when I'm talking about this. Every GPU is going to get its own equal portion of the slice. And if we're doing... There are different stages of zero, but let's just start off with assuming that it's an equal slice of the optimizer states, gradients, and parameters. That would be zero three, stage three in that case. And we do that with a scatter. And the scatter takes, say, one over end GPUs, plus this offset of that slice goes to that GPU. Now all of the GPUs have an equal slice that's in its rank order. And then during each training step, that GPU is going to wait for all of the other slices to communicate so that we now have a whole pie on that GPU, that single GPU. Once we have that whole pie, we do the forward pass on it. And then we distribute that forward pass to all of the others using a gather. So it's a scatter, reduced scatter specifically, and then a gather back to all the others. And you do that each step. So the point of it is that you're sharding these states across GPUs. And with the different stages, you'll see in that figure that the optimizer state is taking the most proportion, which is because of what I mentioned before. We're including the FP32 copy and we're doing atom. So we need those four bytes per param for momentum and for variance. And then zero stage one, which is the most common one, is just optimizer. Zero stage two is optimizer plus gradients. And zero stage three is optimizer gradients and model parameters. But it all comes back to this splitting up and then gathering together back and forth over and over. So you get a lot of communication overhead from zero. But the plus part of that is that you can overlap a lot of that movement with computation. [00:40:23]Alessio: How do you get the optimal number of GPUs to do this on? Is there a way to shard too much as well and put too much overhead? [00:40:31]Quentin: It depends more on what your interconnect is. Taking a step back, there is synchronization that's required, a lot of it, across all of these GPUs. And those tend to be cumulative. So if you go to too many GPUs on an interconnect that's too slow, then you're going to end up spending more time synchronizing. And that magic number where you spend more time synchronizing is going to be different depending on what your fabric is and what your GPU memory is specifically. Just how small of a slice is each GPU getting? I can't, for example, for Summit, that number comes out to be about 20 billion parameters. Now you have 20 billion parameters, and then your magic number of GPUs for that is going to be something like 100 to 200 scale. Beyond that, you're just going to end up spending more time communicating. And the actual flops dipping below some predetermined number by you is going to be whatever your sweet spot ends up being. [00:41:24]Alessio: And then, so this one was like hard for me to go through, so I'm excited to have you run through it, which is a 3D parallelism. [00:41:33]Swyx: It's fancy, it's cutting edge. [00:41:35]Alessio: Yeah, let's talk a bit more about that and some of the work. [00:41:38]Quentin: Okay, 3D parallelism. So what is each dimension? First is the really basic one. That's data parallelism. And data parallelism is you have a copy of the model. Let's say for simplicity, one copy fits on one GPU perfectly. Data parallelism is that now you have two GPUs, so you have one copy on GPU one, one copy on GPU two. Both of them do the forward and backward pass and then synchronize and average the gradients. And then that's a step. Data parallelism for 3D parallelism is actually zero. So it's, you're sharding the optimizer states across all of your different GPUs. Next up is tensor parallelism. Tensor parallelism is you split your model. Like say, if you have two GPUs, you split your model down the middle and each GPU on its tensor specifically is going to do its forward or backward operation on its tensor. And then only when necessary, it'll synchronize that tensor operation with the other GPU. It's a bit more complex than something like pipeline parallelism, which is the third dimension. In pipeline parallelism, let's say you have four layers in your model. And you have four GPUs. You put one layer on each GPU and then GPU one does the forward pass and then sends the output of its activations to GPU two. It does the forward pass, sends activations to three, and you're just moving down a line. That is a naive scheme in that all of the other GPUs are doing nothing while a single GPU is doing its forward or backward pass. So the reason it's called pipeline parallelism is because you're splitting your mini batch into micro batches. So GPU one will do the forward pass on micro batch one and then send to GPU two. And then while GPU two is running on that first micro batch, GPU one is working on the next micro batch. And so you're sort of pipelining the movement and computation of each micro batch. The problem with that is that you need a really big batch size in order to split it up into both mini batches and micro batches. So combining all three of those together, you get a 3D mesh of where each parameter and optimizer state and so on maps to each GPU. And that's 3D parallelism. So let's start diving into details on what have that made sense, what should I jump into more on? [00:43:55]Alessio: I think the main question is, do you need all of the GPUs to be the same to do this? Or can you have mismatching GPUs as well? [00:44:03]Quentin: Okay, two things matter. If there's a difference in VRAM for the two different kinds of GPUs, then you're going to be bottlenecked by whichever GPU has the lower amount of VRAM because it's going to run out of memory. And then you can't like whatever's left on the larger GPUs is going to be empty. As far as I'm aware, there's no like GPU single GPU aware memory overhead scheme that would account for that. The second problem is that let's say all of your GPUs have the same amount of VRAM, but half of them are really slow. And the problem with that is that those synchronizations that I mentioned earlier are going to kill you. So you're going to move as quickly as your slowest GPU in that case. So in both cases, you end up regressing to your slowest or smallest GPU. So you might as well have the same GPUs for all of them. Otherwise, you're wasting the nicer ones. And that also goes to your CPUs and your interconnect. So going back to the 20 billion parameter model that Eleuther was training, that was on a cluster that was sort of Frankenstein made during COVID when there was all of that shortage of network switches and such like that. So every node had a different network switch. And so you ended up moving at the speed of the slowest switch and getting everything tuned properly so that it's not worse than the slowest switch was challenging and is like a real world problem that sometimes comes up. [00:45:28]Alessio: Is this work widely accepted? Like I hadn't learned about this before studying for this episode. Is this something that people are still trying and researching? Or is everybody just aware of this and running this in production? [00:45:43]Quentin: What is this specifically? [00:45:44]Alessio: Like the sharded optimizers plus the 3D parallelism, bringing the two things together and having this kind of mesh strategy. [00:45:51]Quentin: I would say that a lot of major GPT-based models use this scheme. A lot of them now are sort of going with just a pure zero scheme. So just a pure sharded. You just shard everything. And then since that's so easy, everyone gets an equal slice. There's no such thing as a pipeline stage. There's no such thing as what tensor should go on which GPU. Instead, we shard everything equally and treat everything equally. It's a much easier problem to debug, to checkpoint, to run training on than it is with this 3D parallel scheme. I say 3D parallel gives you the most control and also the most ways to go wrong. And depending on whether you have more engineers or whether you have more GPUs, that should decide which of these you go with. [00:46:35]Swyx: It's also not too hard, right? You've basically outlined the five or six different numbers that you need to keep in your head. And it doesn't feel impossible that if you need to achieve that level of control, you've given everybody the main levers to do it with. And that's wonderful. Definitely. [00:46:51]Quentin: The problem that comes up is like, say, like, okay, GPT-4 came out. Now we have VLLMs. [00:46:57]Swyx: Whoa, what are VLLMs? Oh, okay. Virtual LLMs, like the Metro of Expert things? No, like visual. [00:47:03]Quentin: So now you have like multimodal models and such. How do you distribute that? Do you distribute it in a pipeline stage? And do you just shard it? Do you split the tensor and make a tensor parallel? It's sort of hard to change your model and add new features and such when you have this 3D parallel scheme. That's when I say hard. I mean, it's hard to sort of adapt and modify it to new features. [00:47:26]Alessio: I know we're at the hour mark, and I think we put our listeners through a very intense class today. So this was great, Quentin. And we're going to definitely link the article so that people can read it and follow along. Any other research that you're working on in this space that you want to shout out? I know one of our usual, I mean, wrong question is, what's the most interesting unsolved question in AI? So curious to hear if you think it's still on the training inference, math optimization, or are there more areas that people should pay attention to? [00:47:58]Quentin: I think in my area of research, there are two things that I think people should really care about. And the first is multimodal parallelism and RLHF. You were seeing more and more reinforcement learning and coming into the training loop. And so how do you split that some model or some GPUs are working on inference and some GPUs are working on training? And like I mentioned before, you have to relearn everything and they have very unique challenges. How do you split up a KV cache during training, for example? Those are challenges that are not well studied, I don't think. And then multimodal, you have like maybe a vision transformer and a text transformer. How do you split those up? Do you split them up equally? Do you put them on separate GPUs or do you just shard everything? And just maybe one GPU will have some vision, some text parameters. And then the second case I would say is that communication is very often a bottleneck. So we talk about 3D parallelism, but a lot of those like, for example, tensor parallelism, you can't go across nodes with. You'll just get killed in communication. So what I'm getting to is how should you compress your communication before it happens? So on the fly compression, you have some buffer that needs to be communicated. You compress it with a GPU kernel, then you send it across the network and then you decompress it, something like that. Making people spend less money on communication fabrics and more on GPUs as intended is sort of a thing that people need to explore. I think those are my two. [00:49:26]Alessio: Sean, you went over the other half of the lightning round before we wrap it up. [00:49:30]Swyx: That's a good brain dump. Cool. Yeah, I have so many more questions on the multimodal stuff, but that should be for another time. Acceleration, what has already happened in AI that you thought would take much longer? [00:49:42]Quentin: I would say flash attention. Guys, just talk to Tree. And flash attention is just sort of a really great set of kernels that I thought would take a while to get to us. [00:49:51]Alessio: Well, Quentin, thank you very much, man. This was super informative and I think hopefully helps demystify a little bit the blog post. I think people open it and it's like a lot of math on it. And I think you walking them through it was super helpful. So thank you so much for coming on. [00:50:07]Swyx: Of course. [00:50:08]Quentin: And I'm happy to answer any questions that people have offline if they have them. I do read my email. [00:50:13]Swyx: Email and Discord. Of course, yeah. [00:50:15]Quentin: Discord I'm even faster on. [00:50:16]Alessio: Thank you, everyone. [00:50:18]Swyx: Thanks, Quentin. [00:50:19] Get full access to Latent Space at www.latent.space/subscribe

Ep 12: EleutherAI's Aran Komatsuzaki on Open-Source Models' Future and Thought Cloning

Unsupervised Learning

Play Episode Listen Later Jul 19, 2023 43:24

Jacob and Jordan sit down with EleutherAI's Aran Komatsuzaki to discuss the future of open-source models, thought cloning, his work on GPT-J and more. 0:00 intro01:06 Aran's background02:58 starting work on gpt-j05:49 gathering data for Lion and gpt-j08:51 history of EleutherAI11:16 open vs. closed-source models19:06 how will open-source models be used going forward21:33 thought cloning25:51 building AI models that understand video29:35 one model to rule them all31:58 influence of academia in the LLM space34:33 over-hyped/under-hyped38:01 Aran's thoughts on AGI With your co-hosts: @jasoncwarner - Former CTO GitHub, VP Eng Heroku & Canonical @ericabrescia - Former COO Github, Founder Bitnami (acq'd by VMWare) @patrickachase - Partner at Redpoint, Former ML Engineer LinkedIn @jacobeffron - Partner at Redpoint, Former PM Flatiron Health@jordan_segall - Partner at Redpoint

ai partner lion models open source vmware llm cloning aran redpoint eleutherai gpt j

[JUNE 2022] Aran Komatsuzaki on Scaling, GPT-J and Alignment

Play Episode Listen Later Jul 19, 2023 77:21

Aran Komatsuzaki is a ML PhD student at GaTech and lead researcher at EleutherAI where he was one of the authors on GPT-J. In June 2022 we recorded an episode on scaling following up on the first Ethan Caballero episode (where we mentioned Aran as an influence on how Ethan started thinking about scaling). Note: For some reason I procrastinated on editing the podcast, then had a lot of in-person podcasts so I left this one as something to edit later, until the date was so distant from June 2022 that I thought publishing did not make sense anymore. In July 2023 I'm trying that "one video a day" challenge (well I missed some days but I'm trying to get back on track) so I thought it made sense to release it anyway, and after a second watch it's somehow interesting to see how excited Aran was about InstructGPT, which turned to be quite useful for things like ChatGPT. Outline (00:00) intro (00:53) the legend of the two AKs, Aran's arXiv reading routine (04:14) why Aran expects Alignment to be the same as some other ML problems (05:44) what Aran means when he says "AGI" (10:24) what Aran means by "human-level at doing ML research" (11:31) software improvement happening before hardware improvement (13:00) is scale all we need? (15:25) how "Scaling Laws for Neural Language Models" changed the process of doing experiments (16:22) how Aran scale-pilled Ethan (18:46) why Aran was already scale-pilled before GPT-2 (20:12) Aran's 2019 scaling paper: "One epoch is all you need" (25:43) Aran's June 2022 interest: T0 and InstructGPT (31:33) Encoder-Decoder performs better than encoder if multi-task-finetuned (33:30) Why the Scaling Law might be different for T0-like models (37:15) The Story Behind GPT-J (41:40) Hyperparameters and architecture changes in GPT-J (43:56) GPT-J's throughput (47:17) 5 weeks of training using 256 of TPU cores (50:34) did publishing GPT-J accelerate timelines? (55:39) how Aran thinks about Alignment, defining Alignment (58:19) in practice: improving benchmarks, but deception is still a problem (1:00:49) main difficulties in evaluating language models (1:05:07) how Aran sees the future: AIs aligning AIs, merging with AIs, Aran's takeoff scenario (1:10:09) what Aran thinks we should do given how he sees the next decade (1:12:34) regulating access to AGI (1:14:50) what might happen: preventing some AI authoritarian regime (1:15:42) conclusion, where to find Aran

ai chatgpt alignment scaling ml gpt outline agi aran aks ga tech arxiv tpu eleutherai instructgpt gpt j

LW - AutoInterpretation Finds Sparse Coding Beats Alternatives by Hoagy

Play Episode Listen Later Jul 17, 2023 11:22

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AutoInterpretation Finds Sparse Coding Beats Alternatives, published by Hoagy on July 17, 2023 on LessWrong. Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort Huge thanks to Logan Riggs, Aidan Ewart, Lee Sharkey, Robert Huben for their work on the sparse coding project, Lee Sharkey and Chris Mathwin for comments on the draft, EleutherAI for compute and OpenAI for GPT-4 credits. Summary We use OpenAI's automatic interpretation protocol to analyse features found by dictionary learning using sparse coding and compare the interpretability scores thereby found to a variety of baselines. We find that for both the residual stream (layer 2) and MLP (layer 1) of Eleuther's Pythia70M, sparse coding learns a set of features that is superior to all tested baselines, even when removing the bias and looking just at the learnt directions. In doing so we provide additional evidence to the hypothesis that NNs should be conceived as using distributed representations to represent linear features which are only weakly anchored to the neuron basis. As before these results are still somewhat preliminary and we hope to expand on them and make them more robust over the coming month or two, but we hope people find them fruitful sources of ideas. If you want to discuss, feel free to message me or head over to our thread in the EleutherAI discord. All code available at the github repo. Methods Sparse Coding The feature dictionaries learned by sparse coding are learnt by simple linear autoencoders with a sparsity penalty on the activations. For more background on the sparse coding approach to feature-finding see the Conjecture interim report that we're building from, or Robert Huben's explainer. Automatic Interpretation As Logan Riggs' recently found, many of the directions found through sparse coding seem highly interpretable, but we wanted a way to quantify this, and make sure that we were detecting a real difference in the level of interpretability. To do this we used the methodology outlined in this OpenAI paper, details can be found in their code base. To quickly summarise, we are analysing features which are defined as scalar-valued functions of the activations of a neural network, limiting ourselves here to features defined on a single layer of a language model. The original paper simply defined features as the activation of individual neurons but we will in general be looking at linear combinations of neurons. We give a feature an interpretability score by first generating a natural language explanation for the feature, which is expected to explain how strongly a feature will be active in a certain context, for example 'the feature activates on legal terminology'. Then, we give this explanation to an LLM and ask it to predict the feature for hundreds of different contexts, so if the tokens are ['the' 'lawyer' 'went' 'to' 'the' 'court'] the predicted activations might be [0, 10, 0, 0, 8]. The score is defined as the correlation between the true and predicted activations. To generate the explanations we follow OpenAI and take a 64-token sentence-fragment from each of the first 50,000 lines of OpenWebText. For each feature, we calculate the average activation and take the 20 fragments with the highest activation. Of these 20, we pass 5 to GPT-4, along with the rescaled per-token activations. From these 5 fragments, GPT-4 suggests an explanation for when the neuron fires. GPT3.5 is then used to simulate the feature, given the explanation, across both another 5 highly activating fragments, and 5 randomly selected fragments (with non-zero variation). The correlation scores are calculated across all 10 fragments ('top-and-random'), as well as for the top and random fragments separately. Comparing Feature Dictionaries We use dictionary learning with a sparsi...

speech beats ea alternatives openai coding gpt llm mlp conjecture sparse rationalist lesswrong hoagy nns eleutherai

AF - AutoInterpretation Finds Sparse Coding Beats Alternatives by Hoagy

Play Episode Listen Later Jul 17, 2023 11:22

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AutoInterpretation Finds Sparse Coding Beats Alternatives, published by Hoagy on July 17, 2023 on The AI Alignment Forum. Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort Huge thanks to Logan Riggs, Aidan Ewart, Lee Sharkey, Robert Huben for their work on the sparse coding project, Lee Sharkey and Chris Mathwin for comments on the draft, EleutherAI for compute and OpenAI for GPT-4 credits. Summary We use OpenAI's automatic interpretation protocol to analyse features found by dictionary learning using sparse coding and compare the interpretability scores thereby found to a variety of baselines. We find that for both the residual stream (layer 2) and MLP (layer 1) of Eleuther's Pythia70M, sparse coding learns a set of features that is superior to all tested baselines, even when removing the bias and looking just at the learnt directions. In doing so we provide additional evidence to the hypothesis that NNs should be conceived as using distributed representations to represent linear features which are only weakly anchored to the neuron basis. As before these results are still somewhat preliminary and we hope to expand on them and make them more robust over the coming month or two, but we hope people find them fruitful sources of ideas. If you want to discuss, feel free to message me or head over to our thread in the EleutherAI discord. All code available at the github repo. Methods Sparse Coding The feature dictionaries learned by sparse coding are learnt by simple linear autoencoders with a sparsity penalty on the activations. For more background on the sparse coding approach to feature-finding see the Conjecture interim report that we're building from, or Robert Huben's explainer. Automatic Interpretation As Logan Riggs' recently found, many of the directions found through sparse coding seem highly interpretable, but we wanted a way to quantify this, and make sure that we were detecting a real difference in the level of interpretability. To do this we used the methodology outlined in this OpenAI paper, details can be found in their code base. To quickly summarise, we are analysing features which are defined as scalar-valued functions of the activations of a neural network, limiting ourselves here to features defined on a single layer of a language model. The original paper simply defined features as the activation of individual neurons but we will in general be looking at linear combinations of neurons. We give a feature an interpretability score by first generating a natural language explanation for the feature, which is expected to explain how strongly a feature will be active in a certain context, for example 'the feature activates on legal terminology'. Then, we give this explanation to an LLM and ask it to predict the feature for hundreds of different contexts, so if the tokens are ['the' 'lawyer' 'went' 'to' 'the' 'court'] the predicted activations might be [0, 10, 0, 0, 8]. The score is defined as the correlation between the true and predicted activations. To generate the explanations we follow OpenAI and take a 64-token sentence-fragment from each of the first 50,000 lines of OpenWebText. For each feature, we calculate the average activation and take the 20 fragments with the highest activation. Of these 20, we pass 5 to GPT-4, along with the rescaled per-token activations. From these 5 fragments, GPT-4 suggests an explanation for when the neuron fires. GPT3.5 is then used to simulate the feature, given the explanation, across both another 5 highly activating fragments, and 5 randomly selected fragments (with non-zero variation). The correlation scores are calculated across all 10 fragments ('top-and-random'), as well as for the top and random fragments separately. Comparing Feature Dictionaries We use dictionary learning ...

speech beats ea alternatives openai coding gpt llm mlp conjecture sparse rationalist hoagy nns eleutherai

LW - AutoInterpretation Finds Sparse Coding Beats Alternatives by Hoagy

Play Episode Listen Later Jul 17, 2023 11:22

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AutoInterpretation Finds Sparse Coding Beats Alternatives, published by Hoagy on July 17, 2023 on LessWrong. Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort Huge thanks to Logan Riggs, Aidan Ewart, Lee Sharkey, Robert Huben for their work on the sparse coding project, Lee Sharkey and Chris Mathwin for comments on the draft, EleutherAI for compute and OpenAI for GPT-4 credits. Summary We use OpenAI's automatic interpretation protocol to analyse features found by dictionary learning using sparse coding and compare the interpretability scores thereby found to a variety of baselines. We find that for both the residual stream (layer 2) and MLP (layer 1) of Eleuther's Pythia70M, sparse coding learns a set of features that is superior to all tested baselines, even when removing the bias and looking just at the learnt directions. In doing so we provide additional evidence to the hypothesis that NNs should be conceived as using distributed representations to represent linear features which are only weakly anchored to the neuron basis. As before these results are still somewhat preliminary and we hope to expand on them and make them more robust over the coming month or two, but we hope people find them fruitful sources of ideas. If you want to discuss, feel free to message me or head over to our thread in the EleutherAI discord. All code available at the github repo. Methods Sparse Coding The feature dictionaries learned by sparse coding are learnt by simple linear autoencoders with a sparsity penalty on the activations. For more background on the sparse coding approach to feature-finding see the Conjecture interim report that we're building from, or Robert Huben's explainer. Automatic Interpretation As Logan Riggs' recently found, many of the directions found through sparse coding seem highly interpretable, but we wanted a way to quantify this, and make sure that we were detecting a real difference in the level of interpretability. To do this we used the methodology outlined in this OpenAI paper, details can be found in their code base. To quickly summarise, we are analysing features which are defined as scalar-valued functions of the activations of a neural network, limiting ourselves here to features defined on a single layer of a language model. The original paper simply defined features as the activation of individual neurons but we will in general be looking at linear combinations of neurons. We give a feature an interpretability score by first generating a natural language explanation for the feature, which is expected to explain how strongly a feature will be active in a certain context, for example 'the feature activates on legal terminology'. Then, we give this explanation to an LLM and ask it to predict the feature for hundreds of different contexts, so if the tokens are ['the' 'lawyer' 'went' 'to' 'the' 'court'] the predicted activations might be [0, 10, 0, 0, 8]. The score is defined as the correlation between the true and predicted activations. To generate the explanations we follow OpenAI and take a 64-token sentence-fragment from each of the first 50,000 lines of OpenWebText. For each feature, we calculate the average activation and take the 20 fragments with the highest activation. Of these 20, we pass 5 to GPT-4, along with the rescaled per-token activations. From these 5 fragments, GPT-4 suggests an explanation for when the neuron fires. GPT3.5 is then used to simulate the feature, given the explanation, across both another 5 highly activating fragments, and 5 randomly selected fragments (with non-zero variation). The correlation scores are calculated across all 10 fragments ('top-and-random'), as well as for the top and random fragments separately. Comparing Feature Dictionaries We use dictionary learning with a sparsi...

speech beats ea alternatives openai coding gpt llm mlp conjecture sparse rationalist lesswrong hoagy nns eleutherai

Curtis Huebner on Doom, AI Timelines and Alignment at EleutherAI

The Automotive Troublemaker w/ Paul J Daly and Kyle Mountsier

Play Episode Listen Later Jul 16, 2023 89:58

Curtis, also known on the internet as AI_WAIFU, is the head of Alignment at EleutherAI. In this episode we discuss the massive orders of H100s from different actors, why he thinks AGI is 4-5 years away, why he thinks we're 90% "toast", his comment on Eliezer Yudkwosky's Death with Dignity, and what kind of Alignment projects is currently going on at EleutherAI, especially a project with Markov chains and the Alignment test project that he is currently leading. Youtube: https://www.youtube.com/watch?v=9s3XctQOgew Transcript: https://theinsideview.ai/curtis Death with Dignity: https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy Alignment Minetest: https://www.eleuther.ai/projects/alignment-minetest Alignment Minetest update: https://blog.eleuther.ai/minetester-intro/ OUTLINE (00:00) Highlights / Intro (00:50) The Fuck That Noise Comment On Death With Dignity (10:28) Th Probability of Doom Is 90% (12:44) Best Counterarguments For His High P(doom) (14:41) Compute And Model Size Required For A Dangerous Model (17:59) Details For Curtis' Model Of Compute Required (21:23) Why This Estimate Of Compute Required Might Be Wrong, Ajeya Cotra's Transformative AI report (29:00) Curtis' Median For AGI Is Around 2028, Used To Be 2027 (30:50) How Curtis Approaches Life With Short Timelines And High P(Doom) (35:27) Takeoff Speeds—The Software view vs. The Hardware View (39:57) Nvidia's 400k H100 rolling down the assembly line, AIs soon to be unleashed on their own source code (41:04) Could We Get A Fast Takeoff By Fuly Automating AI Research With More Compute (46:00) The Entire World (Tech Companies, Governments, Militaries) Is Noticing New AI Capabilities That They Don't Have (47:57) Open-source vs. Close source policies. Mundane vs. Apocalyptic considerations. (53:25) Curtis' background, from teaching himself deep learning to EleutherAI (55:51) Alignment Project At EleutherAI: Markov Chain and Language Models (01:02:15) Research Philosophy at EleutherAI: Pursuing Useful Projects, Multingual, Discord, Logistics (01:07:38) Alignment MineTest: Links To Alignmnet, Embedded Agency, Wireheading (01:15:30) Next steps for Alignment Mine Test: focusing on model-based RL (01:17:07) Training On Human Data & Using an Updated Gym Environment With Human APIs (01:19:20) Model Used, Not Observing Symmetry (01:21:58) Another goal of Alignment Mine Test: Study Corrigibility (01:28:26) People ordering H100s Are Aware Of Other People Making These Orders, Race Dynamics, Last Message

Benz Goes Tesla, 40 Epic Auto People Recognized, and AI Lawsuits

Play Episode Listen Later Jul 10, 2023 14:32 Transcription Available

Good Monday morning! We are blessed by the voice of Michael Cirillo on the pod this week. Today we are following the continuing story of Tesla's NACS chargers, celebrating the Automotive News 40 under 40 list, and checking in on an AI lawsuit by some famous authors.Show Notes with links:Another domino has fallen in Tesla's favor as Mercedes-Benz announced that they will integrate the NACS charging connector in their EVs, beginning in 2024.The 2024 models will include a NACS adapter to the CCS connector. In 2025 and beyond, the vehicles will have a NACS port.Mercedes is also planning a charging network of more than 2500 high-power chargers in North America with some sites opening in late 2023.Volkswagen, BMW, Hyundai, Toyota, Honda, Stellantis and Jaguar Land Rover are the remaining major North American OEMs who haven't adopted the new standard.The 12th 40 Under 40 Retail class has been announced by Automotive News, honoring executives, sales and service managers, a general counsel and more.Editor Dan Shine said he was struck by how much More Than Cars reminded him of the 40 Under 40 awards. “[More Than Cars] reminded me of the special stories behind so many of the people we cover… People who were born into it and those who took circuitous routes to the automotive retail industry.”Some recognizable faces for the ASOTU-verse: Dayna Kleve, Director of diversity, engagement & foundation at Walser Automotive Group, Felicia Rey, Owner loyalty manager at Lynnes Nissan CityThe list also includes Tristan Topps, a former Ms. United States, Kristina Perez-Cubas who performed at the 2007 Super Bowl Halftime Show, Keri Lanzavecchia who took 3 attempts to pass her written learner's permit and Ronnie Lowenfield who has a collection of over 2500 bottles of bourbon.Sarah Silverman and authors Christopher Golden and Richard Kadrey are suing OpenAI and Meta, alleging copyright infringement as their AI chatbots, trained on illegally obtained datasets, have been summarizing the authors' books without permission.The authors argue that the datasets used by Meta have illicit origins, pointing out that one of the sources for its training datasets, ThePile, was assembled by a company called EleutherAI, which allegedly used content from the Bibliotik private tracker.Lawyers in the case have heard from other writers, authors, and publishers who are concerned about ChatGPT's ability to generate text similar to that found in copyrighted materials.Get the Daily Push Back email at https://www.asotu.com/ JOIN the conversation on LinkedIn at: https://www.linkedin.com/company/asotu/ Read our most recent email at: https://www.asotu.com/media/push-back-email ASOTU Instagram: https://www.instagram.com/automotivestateoftheunion

177 - AI is a Ticking Time Bomb with Connor Leahy

Bankless

Play Episode Listen Later Jun 26, 2023 95:28

AI is here to stay, but at what cost? Connor Leahy is the CEO of Conjecture, a mission-driven organization that's trying to make the future of AI go as well as it possibly can. He is also a Co-Founder of EleutherAI, an open-source AI research non-profit lab. In today's episode, Connor and David cover: 1) The intuitive arguments behind the AI Safety debate 2) The two defining categories of ways AI could end all of humanity 3) The major players in the race towards AGI, and why they all seem to be ideologically motivated, rather than financially motivated 4) Why the progress of AI power is based on TWO exponential curves 5) Why Connor thinks government regulation is the easiest and most effective way of buying us time ------

E38: The Virtual Biopsy Revolution with Dr. Tanishq Mathew Abraham (Part 2 of 2)

Play Episode Listen Later Jun 22, 2023 65:36

Part 2 of Nathan's conversation with Tanishq Mathew Abraham focuses on Tanishq's work with virtual biopsy technology enabled by deep learning. This unique technology has the potential to address a number of important biomedical challenges; in particular, qOBM could help during live tumor and cancer removal surgeries. Tanishq, a 19-year-old UC Davis grad and one of the youngest people in the world to receive a Ph.D, with a degree in biomedical engineering, is the founder of the Medical AI Research Center (MedARC). In case you haven't yet listened to part 1, check it out! That episode goes deep on Tanishq's first published paper: Reconstructions of the Mind's Eye, which encompasses breakthrough research on reconstructing visual perceptions from fMRI scans into images. RECOMMENDED PODCAST: The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix's culture deck Patty McCord. https://link.chtbl.com/hrheretics The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co TIMESTAMPS: (00:00) Episode Preview (03:26) Introducing Tanishq's second paper presenting an AI-enabled biopsy (06:39) Diagnostic applications of this unique slide-free and label-free technology (qOBM) (13:06) Leveraging deep learning, specifically generative adversarial networks (GANs) (14:55) Sponsor: Omneky (15:42) Framework for model (16:22) Challenges of medical data sets (20:11) Challenges of unpaired image to image translation, addressed with a CycleGAN architecture (22:19) What didn't work (25:44) Breaking down GAN frameworks (34:30) Simplifying data to better with the CycleGAN (36:02) Factors for errors and confusing the model (39:00) Compute and training requirements (41:35) How this technology can scale in the near term and improve patient care (45:53) Tanishq's relationships with EleutherAI and Stability AI (47:08) The future-looking focus of MedArc and unexplored opportunities (49:00) Developments in Medical AI field and interesting applications (55:11) Early education and AI (58:00) Would Tanishq get a neuralink? (1:00:00) Tanishq's hopes and fears for AI LINKS: Paper: Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual H&E staining https://arxiv.org/pdf/2306.00548.pdf Authors: Tanishq Mathew Abraham, Paloma Casteleiro Costa, Caroline Filan, Zhe Guang, Zhaobin Zhang, Stewart Neill, Jeffrey J. Olson, Richard Levenson, Francisco E. Robles TWITTER: @iScienceLuvr (Tanishq) @MedARC_AI (MedARC) @CogRev_Podcast @labenz (Nathan) @eriktorenberg (Erik) SPONSOR: Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off. MUSIC CREDIT: MusicLM

Will AI destroy civilization in the near future? (with Connor Leahy)

Clearer Thinking with Spencer Greenberg

Play Episode Listen Later Jun 21, 2023 85:26

Read the full transcript here. Does AI pose a near-term existential risk? Why might existential risks from AI manifest sooner rather than later? Can't we just turn off any AI that gets out of control? Exactly how much do we understand about what's going on inside neural networks? What is AutoGPT? How feasible is it to build an AI system that's exactly as intelligent as a human but no smarter? What is the "CoEm" AI safety proposal? What steps can the average person take to help mitigate risks from AI?Connor Leahy is CEO and co-founder of Conjecture, an AI alignment company focused on making AI systems boundable and corrigible. Connor founded and led EleutherAI, the largest online community dedicated to LLMs, which acted as a gateway for people interested in ML to upskill and learn about alignment. With capabilities increasing at breakneck speed, and our ability to control AI systems lagging far behind, Connor moved on from the volunteer, open-source Eleuther model to a full-time, closed-source model working to solve alignment via Conjecture. [Read more]

ceo ai destroy civilization ml leahy near future conjecture autogpt connor leahy eleutherai

LW - [Replication] Conjecture's Sparse Coding in Small Transformers by Hoagy

Play Episode Listen Later Jun 17, 2023 7:35

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Replication] Conjecture's Sparse Coding in Small Transformers, published by Hoagy on June 16, 2023 on LessWrong. Summary A couple of weeks ago Logan Riggs and I posted that we'd replicated the toy-model experiments in Lee Sharkey and Dan Braun's original sparse coding post. Now we've replicated the work they did (slides) extending the technique to custom-trained small transformers (in their case 16 residual dimensions, in ours 32). We've been able to replicate all of their core results, and our main takeaways from the last 2 weeks of research are the following: We can recover many more features from activation space than the dimension of the activation space which have a high degree of cosine similarity with the features learned by other, larger dictionaries, which in toy models was a core indicator of having learned the correct features. The distribution of MCS scores between trained dictionaries of different sizes is highly bimodal, suggesting that there is a particular set of features that are consistently found as sparse basis vectors of the activations, across different dictionary sizes. The maximum-activating examples of these features usually seem human interpretable, though we haven't yet done a systematic comparison to the neuron basis. The diagonal lines seen in the original post are an artefact of dividing the l1_loss coefficient by the dictionary size. Removing this means that the same l1_loss is applicable to a broad range of dictionary sizes. We find that as dict size increases, MMCS initially increases rapidly at first, but then plateaus. The learned feature vectors, including those ones that appear repeatedly, do not appear to be at all sparse with respect to the neuron basis. As before, all code is available on GitHub and if you'd like to follow the research progress and potentially contribute, then join us on our EleutherAI thread. Thanks to Robert Huben (Robert_AIZI) for extensive comments on this draft, and Lee Sharkey, Dan Braun and Robert Huben for their comments during our work. Next steps & Request For Funding We'd like to test how interpretable the features that we have found are, in a quantitative manner. We've got the basic structure ready to apply OpenAI's automated-intepretability library to the found features, which we would then compare to baselines such as the neuron basis and the PCA and ICA of the activation data. This requires quite a lot of tokens- something like 20c worth of tokens for each query depending on the number of example sentences given. We would need to analyse hundreds of neurons to get a representative sample size, for each of the dictionaries or approaches, so we're looking for funding for this research on the order of a few thousand dollars, potentially more if it were available. If you'd be interested in supporting this research, please get in touch. We'll also be actively searching for funding, and trying to get OpenAI to donate some compute. Assuming the result of this experiment are promising, (meaning that we are able to exceed the baselines in terms of quality or quantity of highly interpretable features), we plan then to focus on scaling up to larger models and experimenting with variations of the technique which incorporate additional information, such as combining activation data from multiple layers, or using the weight vectors to inform the selection of features. We're also very interested to work with people developing mathematical or toy models of superposition. Results Background We used Andrej Karpathy's nanoGPT to train a small transformer, with 6 layers, a 32 dimensional residual stream, MLP widths of 128 and 4 attention heads with dimension 8. We trained on a node of 4xRTX3090s for about 9h, reaching a loss of 5.13 on OpenWebText. This transformer is the model from which we took activations, and t...

speech transformers ea openai coding github assuming mcs pca ica replication mlp conjecture sparse rationalist andrej karpathy lesswrong hoagy eleutherai

LW - [Replication] Conjecture's Sparse Coding in Small Transformers by Hoagy

Play Episode Listen Later Jun 17, 2023 7:35

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Replication] Conjecture's Sparse Coding in Small Transformers, published by Hoagy on June 16, 2023 on LessWrong. Summary A couple of weeks ago Logan Riggs and I posted that we'd replicated the toy-model experiments in Lee Sharkey and Dan Braun's original sparse coding post. Now we've replicated the work they did (slides) extending the technique to custom-trained small transformers (in their case 16 residual dimensions, in ours 32). We've been able to replicate all of their core results, and our main takeaways from the last 2 weeks of research are the following: We can recover many more features from activation space than the dimension of the activation space which have a high degree of cosine similarity with the features learned by other, larger dictionaries, which in toy models was a core indicator of having learned the correct features. The distribution of MCS scores between trained dictionaries of different sizes is highly bimodal, suggesting that there is a particular set of features that are consistently found as sparse basis vectors of the activations, across different dictionary sizes. The maximum-activating examples of these features usually seem human interpretable, though we haven't yet done a systematic comparison to the neuron basis. The diagonal lines seen in the original post are an artefact of dividing the l1_loss coefficient by the dictionary size. Removing this means that the same l1_loss is applicable to a broad range of dictionary sizes. We find that as dict size increases, MMCS initially increases rapidly at first, but then plateaus. The learned feature vectors, including those ones that appear repeatedly, do not appear to be at all sparse with respect to the neuron basis. As before, all code is available on GitHub and if you'd like to follow the research progress and potentially contribute, then join us on our EleutherAI thread. Thanks to Robert Huben (Robert_AIZI) for extensive comments on this draft, and Lee Sharkey, Dan Braun and Robert Huben for their comments during our work. Next steps & Request For Funding We'd like to test how interpretable the features that we have found are, in a quantitative manner. We've got the basic structure ready to apply OpenAI's automated-intepretability library to the found features, which we would then compare to baselines such as the neuron basis and the PCA and ICA of the activation data. This requires quite a lot of tokens- something like 20c worth of tokens for each query depending on the number of example sentences given. We would need to analyse hundreds of neurons to get a representative sample size, for each of the dictionaries or approaches, so we're looking for funding for this research on the order of a few thousand dollars, potentially more if it were available. If you'd be interested in supporting this research, please get in touch. We'll also be actively searching for funding, and trying to get OpenAI to donate some compute. Assuming the result of this experiment are promising, (meaning that we are able to exceed the baselines in terms of quality or quantity of highly interpretable features), we plan then to focus on scaling up to larger models and experimenting with variations of the technique which incorporate additional information, such as combining activation data from multiple layers, or using the weight vectors to inform the selection of features. We're also very interested to work with people developing mathematical or toy models of superposition. Results Background We used Andrej Karpathy's nanoGPT to train a small transformer, with 6 layers, a 32 dimensional residual stream, MLP widths of 128 and 4 attention heads with dimension 8. We trained on a node of 4xRTX3090s for about 9h, reaching a loss of 5.13 on OpenWebText. This transformer is the model from which we took activations, and t...

speech transformers ea openai coding github assuming mcs pca ica replication mlp conjecture sparse rationalist andrej karpathy lesswrong hoagy eleutherai

AF - [Replication] Conjecture's Sparse Coding in Small Transformers by Hoagy

Play Episode Listen Later Jun 16, 2023 7:36

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Replication] Conjecture's Sparse Coding in Small Transformers, published by Hoagy on June 16, 2023 on The AI Alignment Forum. Summary A couple of weeks ago Logan Riggs and I posted that we'd replicated the toy-model experiments in Lee Sharkey and Dan Braun's original sparse coding post. Now we've replicated the work they did (slides) extending the technique to custom-trained small transformers (in their case 16 residual dimensions, in ours 32). We've been able to replicate all of their core results, and our main takeaways from the last 2 weeks of research are the following: We can recover many more features from activation space than the dimension of the activation space which have a high degree of cosine similarity with the features learned by other, larger dictionaries, which in toy models was a core indicator of having learned the correct features. The distribution of MCS scores between trained dictionaries of different sizes is highly bimodal, suggesting that there is a particular set of features that are consistently found as sparse basis vectors of the activations, across different dictionary sizes. The maximum-activating examples of these features usually seem human interpretable, though we haven't yet done a systematic comparison to the neuron basis. The diagonal lines seen in the original post are an artefact of dividing the l1_loss coefficient by the dictionary size. Removing this means that the same l1_loss is applicable to a broad range of dictionary sizes. We find that as dict size increases, MMCS initially increases rapidly at first, but then plateaus. The learned feature vectors, including those ones that appear repeatedly, do not appear to be at all sparse with respect to the neuron basis. As before, all code is available on GitHub and if you'd like to follow the research progress and potentially contribute, then join us on our EleutherAI thread. Thanks to Robert Huben (Robert_AIZI) for extensive comments on this draft, and Lee Sharkey, Dan Braun and Robert Huben for their comments during our work. Next steps & Request For Funding We'd like to test how interpretable the features that we have found are, in a quantitative manner. We've got the basic structure ready to apply OpenAI's automated-intepretability library to the found features, which we would then compare to baselines such as the neuron basis and the PCA and ICA of the activation data. This requires quite a lot of tokens- something like 20c worth of tokens for each query depending on the number of example sentences given. We would need to analyse hundreds of neurons to get a representative sample size, for each of the dictionaries or approaches, so we're looking for funding for this research on the order of a few thousand dollars, potentially more if it were available. If you'd be interested in supporting this research, please get in touch. We'll also be actively searching for funding, and trying to get OpenAI to donate some compute. Assuming the result of this experiment are promising, (meaning that we are able to exceed the baselines in terms of quality or quantity of highly interpretable features), we plan then to focus on scaling up to larger models and experimenting with variations of the technique which incorporate additional information, such as combining activation data from multiple layers, or using the weight vectors to inform the selection of features. We're also very interested to work with people developing mathematical or toy models of superposition. Results Background We used Andrej Karpathy's nanoGPT to train a small transformer, with 6 layers, a 32 dimensional residual stream, MLP widths of 128 and 4 attention heads with dimension 8. We trained on a node of 4xRTX3090s for about 9h, reaching a loss of 5.13 on OpenWebText. This transformer is the model from which we took activ...

speech transformers ea openai coding github assuming mcs pca ica replication mlp conjecture sparse rationalist andrej karpathy hoagy eleutherai

EA - Critiques of prominent AI safety labs: Conjecture by Omega

Play Episode Listen Later Jun 12, 2023 54:37

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Critiques of prominent AI safety labs: Conjecture, published by Omega on June 12, 2023 on The Effective Altruism Forum. Crossposted to LessWrong. In this series, we consider AI safety organizations that have received more than $10 million per year in funding. There have already been several conversations and critiques around MIRI (1) and OpenAI (1,2,3), so we will not be covering them. The authors include one technical AI safety researcher (>4 years experience), and one non-technical community member with experience in the EA community. We'd like to make our critiques non-anonymously but believe this will not be a wise move professionally speaking. We believe our criticisms stand on their own without appeal to our positions. Readers should not assume that we are completely unbiased or don't have anything to personally or professionally gain from publishing these critiques. We've tried to take the benefits and drawbacks of the anonymous nature of our post seriously and carefully, and are open to feedback on anything we might have done better. This is the second post in this series and it covers Conjecture. Conjecture is a for-profit alignment startup founded in late 2021 by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale applied alignment research. Based in London, Conjecture has received $10 million in funding from venture capitalists (VCs), and recruits heavily from the EA movement. We shared a draft of this document with Conjecture for feedback prior to publication, and include their response below. We also requested feedback on a draft from a small group of experienced alignment researchers from various organizations, and have invited them to share their views in the comments of this post. We would like to invite others to share their thoughts in the comments openly if you feel comfortable, or contribute anonymously via this form. We will add inputs from there to the comments section of this post, but will likely not be updating the main body of the post as a result (unless comments catch errors in our writing). Key Takeaways For those with limited knowledge and context on Conjecture, we recommend first reading or skimming the About Conjecture section. Time to read the core sections (Criticisms & Suggestions and Our views on Conjecture) is 22 minutes. Criticisms and Suggestions We think Conjecture's research is low quality (read more). Their posts don't always make assumptions clear, don't make it clear what evidence base they have for a given hypothesis, and evidence is frequently cherry-picked. We also think their bar for publishing is too low, which decreases the signal to noise ratio. Conjecture has acknowledged some of these criticisms, but not all (read more). We make specific critiques of examples of their research from their initial research agenda (read more). There is limited information available on their new research direction (cognitive emulation), but from the publicly available information it appears extremely challenging and so we are skeptical as to its tractability (read more). We have some concerns with the CEO's character and trustworthiness because, in order of importance (read more): The CEO and Conjecture have misrepresented themselves to external parties multiple times (read more); The CEO's involvement in EleutherAI and Stability AI has contributed to race dynamics (read more); The CEO previously overstated his accomplishments in 2019 (when an undergrad) (read more); The CEO has been inconsistent over time regarding his position on releasing LLMs (read more). We believe Conjecture has scaled too quickly before demonstrating they have promising research results, and believe this will make it harder for them to pivot in the future (read more). We are concerned that Conjecture does not have a clear plan for balancing profit an...

ceo time ai safety speech omega ea openai labs readers vcs prominent critiques criticisms miri conjecture stability ai rationalist lesswrong connor leahy eleutherai

LW - Critiques of prominent AI safety labs: Conjecture by Omega.

Play Episode Listen Later Jun 12, 2023 54:59

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Critiques of prominent AI safety labs: Conjecture, published by Omega. on June 12, 2023 on LessWrong. Cross-posted from the EA Forum. See the original here. Internal linking has not been updated for LW due to time constraints and will take you back to the original post. In this series, we consider AI safety organizations that have received more than $10 million per year in funding. There have already been several conversations and critiques around MIRI (1) and OpenAI (1,2,3), so we will not be covering them. The authors include one technical AI safety researcher (>4 years experience), and one non-technical community member with experience in the EA community. We'd like to make our critiques non-anonymously but believe this will not be a wise move professionally speaking. We believe our criticisms stand on their own without appeal to our positions. Readers should not assume that we are completely unbiased or don't have anything to personally or professionally gain from publishing these critiques. We've tried to take the benefits and drawbacks of the anonymous nature of our post seriously and carefully, and are open to feedback on anything we might have done better. This is the second post in this series and it covers Conjecture. Conjecture is a for-profit alignment startup founded in late 2021 by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale applied alignment research. Based in London, Conjecture has received $10 million in funding from venture capitalists (VCs), and recruits heavily from the EA movement. We shared a draft of this document with Conjecture for feedback prior to publication, and include their response below. We also requested feedback on a draft from a small group of experienced alignment researchers from various organizations, and have invited them to share their views in the comments of this post. We would like to invite others to share their thoughts in the comments openly if you feel comfortable, or contribute anonymously via this form. We will add inputs from there to the comments section of this post, but will likely not be updating the main body of the post as a result (unless comments catch errors in our writing). Key Takeaways For those with limited knowledge and context on Conjecture, we recommend first reading or skimming the About Conjecture section. Time to read the core sections (Criticisms & Suggestions and Our views on Conjecture) is 22 minutes. Criticisms and Suggestions We think Conjecture's research is low quality (read more). Their posts don't always make assumptions clear, don't make it clear what evidence base they have for a given hypothesis, and evidence is frequently cherry-picked. We also think their bar for publishing is too low, which increases the signal to noise ratio. Conjecture has acknowledged some of these criticisms, but not all (read more). We make specific critiques of examples of their research from their initial research agenda (read more). There is limited information available on their new research direction (cognitive emulation), but from the publicly available information it appears extremely challenging and so we are skeptical as to its tractability (read more). We have some concerns with the CEO's character and trustworthiness because, in order of importance (read more): The CEO and Conjecture have misrepresented themselves to external parties multiple times (read more); The CEO's involvement in EleutherAI and Stability AI has contributed to race dynamics (read more); The CEO previously overstated his accomplishments in 2019 (when an undergrad) (read more); The CEO has been inconsistent over time regarding his position on releasing LLMs (read more). We believe Conjecture has scaled too quickly before demonstrating they have promising research results, and believe this will make it harder for...

LW - Critiques of prominent AI safety labs: Conjecture by Omega.

Play Episode Listen Later Jun 12, 2023 54:59

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Critiques of prominent AI safety labs: Conjecture, published by Omega. on June 12, 2023 on LessWrong. Cross-posted from the EA Forum. See the original here. Internal linking has not been updated for LW due to time constraints and will take you back to the original post. In this series, we consider AI safety organizations that have received more than $10 million per year in funding. There have already been several conversations and critiques around MIRI (1) and OpenAI (1,2,3), so we will not be covering them. The authors include one technical AI safety researcher (>4 years experience), and one non-technical community member with experience in the EA community. We'd like to make our critiques non-anonymously but believe this will not be a wise move professionally speaking. We believe our criticisms stand on their own without appeal to our positions. Readers should not assume that we are completely unbiased or don't have anything to personally or professionally gain from publishing these critiques. We've tried to take the benefits and drawbacks of the anonymous nature of our post seriously and carefully, and are open to feedback on anything we might have done better. This is the second post in this series and it covers Conjecture. Conjecture is a for-profit alignment startup founded in late 2021 by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale applied alignment research. Based in London, Conjecture has received $10 million in funding from venture capitalists (VCs), and recruits heavily from the EA movement. We shared a draft of this document with Conjecture for feedback prior to publication, and include their response below. We also requested feedback on a draft from a small group of experienced alignment researchers from various organizations, and have invited them to share their views in the comments of this post. We would like to invite others to share their thoughts in the comments openly if you feel comfortable, or contribute anonymously via this form. We will add inputs from there to the comments section of this post, but will likely not be updating the main body of the post as a result (unless comments catch errors in our writing). Key Takeaways For those with limited knowledge and context on Conjecture, we recommend first reading or skimming the About Conjecture section. Time to read the core sections (Criticisms & Suggestions and Our views on Conjecture) is 22 minutes. Criticisms and Suggestions We think Conjecture's research is low quality (read more). Their posts don't always make assumptions clear, don't make it clear what evidence base they have for a given hypothesis, and evidence is frequently cherry-picked. We also think their bar for publishing is too low, which increases the signal to noise ratio. Conjecture has acknowledged some of these criticisms, but not all (read more). We make specific critiques of examples of their research from their initial research agenda (read more). There is limited information available on their new research direction (cognitive emulation), but from the publicly available information it appears extremely challenging and so we are skeptical as to its tractability (read more). We have some concerns with the CEO's character and trustworthiness because, in order of importance (read more): The CEO and Conjecture have misrepresented themselves to external parties multiple times (read more); The CEO's involvement in EleutherAI and Stability AI has contributed to race dynamics (read more); The CEO previously overstated his accomplishments in 2019 (when an undergrad) (read more); The CEO has been inconsistent over time regarding his position on releasing LLMs (read more). We believe Conjecture has scaled too quickly before demonstrating they have promising research results, and believe this will make it harder for...

ANTHOLOGY — Open source AI

The Changelog

Play Episode Listen Later May 24, 2023 98:06 Transcription Available

This week on The Changelog we're taking you to the hallway track of The Linux Foundation's Open Source Summit North America 2023 in Vancouver, Canada. Today's anthology episode features: Beyang Liu (Co-founder and CTO at Sourcegraph), Denny Lee (Developer Advocate at Databricks), and Stella Biderman (Executive Director and Head of Research at EleutherAI). Special thanks to our friends at GitHub for sponsoring us to attend this conference as part of Maintainer Month.

ANTHOLOGY — Open source AI (Changelog Interviews #541)

Changelog Master Feed

Play Episode Listen Later May 24, 2023 98:06 Transcription Available

#122 Connor Leahy: Unveiling the Darker Side of AI

Eye On A.I.

Play Episode Listen Later May 10, 2023 56:18

Welcome to Eye on AI, the podcast that explores the latest developments, challenges, and opportunities in the world of artificial intelligence. In this episode, we sit down with Connor Leahy, an AI researcher and co-founder of EleutherAI, to discuss the darker side of AI. Connor shares his insights on the current negative trajectory of AI, the challenges of keeping superintelligence in a sandbox, and the potential negative implications of large language models such as GPT4. He also discusses the problem of releasing AI to the public and the need for regulatory intervention to ensure alignment with human values. Throughout the podcast, Connor highlights the work of Conjecture, a project focused on advancing alignment in AI, and shares his perspectives on the stages of research and development of this critical issue. If you're interested in understanding the ethical and social implications of AI and the efforts to ensure alignment with human values, this podcast is for you. So join us as we delve into the darker side of AI with Connor Leahy on Eye on AI. (00:00) Preview (00:48) Connor Leahy's background with EleutherAI & Conjecture (03:05) Large language models applications with EleutherAI (06:51) The current negative trajectory of AI (08:46) How difficult is keeping super intelligence in a sandbox? (12:35) How AutoGPT uses ChatGPT to run autonomously (15:15) How GPT4 can be used out of context & negatively (19:30) How OpenAI gives access to nefarious activities (26:39) The problem with the race for AGI (28:51) The goal of Conjecture and advancing alignment (31:04) The problem with releasing AI to the public (33:35) FTC complaint & government intervention in AI (38:13) Technical implementation to fix the alignment issue (44:34) How CoEm is fixing the alignment issue (53:30) Stages of research and development of Conjecture Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI

LW - Residual stream norms grow exponentially over the forward pass by StefanHex

Play Episode Listen Later May 7, 2023 20:49

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Residual stream norms grow exponentially over the forward pass, published by StefanHex on May 7, 2023 on LessWrong. Summary: For a range of language models and a range of input prompts, the norm of each residual stream grows exponentially over the forward pass, with average per-layer growth rate of about 1.045 in GPT2-XL. We show a bunch of evidence for this. We discuss to what extent different weights and parts of the network are responsible. We find that some model weights increase exponentially as a function of layer number. We finally note our current favored explanation: Due to LayerNorm, it's hard to cancel out existing residual stream features, but easy to overshadow existing features by just making new features 4.5% larger. Thanks to Aryan Bhatt, Marius Hobbhahn, Neel Nanda, and Nicky Pochinkov for discussion. Plots showing exponential norm and variance growth Our results are reproducible in this Colab. Alex noticed exponential growth in the contents of GPT-2-XL's residual streams. He ran dozens of prompts through the model, plotted for each layer the distribution of residual stream norms in a histogram, and found exponential growth in the L2 norm of the residual streams: Here's the norm of each residual stream for a specific prompt: Stefan had previously noticed this phenomenon in GPT2-small, back in MATS 3.0: Basic Facts about Language Model Internals also finds a growth in the norms of the attention-out matrices WO and the norms of MLP out matrices Wout ("writing weights"), while they find stable norms for WQ, WK, and Win ("reading weights"): Comparison of various transformer models We started our investigation by computing these residual stream norms for a variety of models, recovering Stefan's results (rescaled by √dmodel=√768) and Alex's earlier numbers. We see a number of straight lines in these logarithmic plots, which shows phases of exponential growth. We are surprised by the decrease in Residual Stream norm in some of the EleutherAI models. We would have expected that, because the transformer blocks can only access the normalized activations, it's hard for the model to "cancel out" a direction in the residual stream. Therefore, the norm always grows. However, this isn't what we see above. One explanation is that the model is able to memorize or predict the LayerNorm scale. If the model does this well enough it can (partially) delete activations and reduce the norm by writing vectors that cancel out previous activations. The very small models (distillgpt2, gpt2-small) have superexponential norm growth, but most models show exponential growth throughout extended periods. For example, from layer 5 to 41 in GPT2-XL, we see an exponential increase in residual stream norm at a rate of ~1.045 per layer. We showed this trend as an orange line in the above plot, and below we demonstrate the growth for a specific example: BOS and padding tokens In our initial tests, we noticed some residual streams showed a irregular and surprising growth curve: As for the reason behind this shape, we expect that the residual stream (norm) is very predictable at BOS and padding positions. This is because these positions cannot attend to other positions and thus always have the same values (up to positional embedding). Thus it would be no problem for the model to cancel out activations, and our arguments about this being hard do not hold for BOS and padding positions. We don't know whether there is a particular meaning behind this shape. We suspect that is the source of the U-shape shown in Basic facts about language models during training: Theories for the source of the growth From now on we focus on the GPT2-XL case. Here is the residual stream growth curve again (orange dots), but also including the resid_mid hook between the two Attention and MLP sub-layers (blue dots). O...

AF - Residual stream norms grow exponentially over the forward pass by Stefan Heimersheim

Play Episode Listen Later May 7, 2023 20:50

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Residual stream norms grow exponentially over the forward pass, published by Stefan Heimersheim on May 7, 2023 on The AI Alignment Forum. Summary: For a range of language models and a range of input prompts, the norm of each residual stream grows exponentially over the forward pass, with average per-layer growth rate of about 1.045 in GPT2-XL. We show a bunch of evidence for this. We discuss to what extent different weights and parts of the network are responsible. We find that some model weights increase exponentially as a function of layer number. We finally note our current favored explanation: Due to LayerNorm, it's hard to cancel out existing residual stream features, but easy to overshadow existing features by just making new features 4.5% larger. Thanks to Aryan Bhatt, Marius Hobbhahn, Neel Nanda, and Nicky Pochinkov for discussion. Plots showing exponential norm and variance growth Our results are reproducible in this Colab. Alex noticed exponential growth in the contents of GPT-2-XL's residual streams. He ran dozens of prompts through the model, plotted for each layer the distribution of residual stream norms in a histogram, and found exponential growth in the L2 norm of the residual streams: Here's the norm of each residual stream for a specific prompt: Stefan had previously noticed this phenomenon in GPT2-small, back in MATS 3.0: Basic Facts about Language Model Internals also finds a growth in the norms of the attention-out matrices WO and the norms of MLP out matrices Wout ("writing weights"), while they find stable norms for WQ, WK, and Win ("reading weights"): Comparison of various transformer models We started our investigation by computing these residual stream norms for a variety of models, recovering Stefan's results (rescaled by √dmodel=√768) and Alex's earlier numbers. We see a number of straight lines in these logarithmic plots, which shows phases of exponential growth. We are surprised by the decrease in Residual Stream norm in some of the EleutherAI models. We would have expected that, because the transformer blocks can only access the normalized activations, it's hard for the model to "cancel out" a direction in the residual stream. Therefore, the norm always grows. However, this isn't what we see above. One explanation is that the model is able to memorize or predict the LayerNorm scale. If the model does this well enough it can (partially) delete activations and reduce the norm by writing vectors that cancel out previous activations. The very small models (distillgpt2, gpt2-small) have superexponential norm growth, but most models show exponential growth throughout extended periods. For example, from layer 5 to 41 in GPT2-XL, we see an exponential increase in residual stream norm at a rate of ~1.045 per layer. We showed this trend as an orange line in the above plot, and below we demonstrate the growth for a specific example: BOS and padding tokens In our initial tests, we noticed some residual streams showed a irregular and surprising growth curve: As for the reason behind this shape, we expect that the residual stream (norm) is very predictable at BOS and padding positions. This is because these positions cannot attend to other positions and thus always have the same values (up to positional embedding). Thus it would be no problem for the model to cancel out activations, and our arguments about this being hard do not hold for BOS and padding positions. We don't know whether there is a particular meaning behind this shape. We suspect that is the source of the U-shape shown in Basic facts about language models during training: Theories for the source of the growth From now on we focus on the GPT2-XL case. Here is the residual stream growth curve again (orange dots), but also including the resid_mid hook between the two Attention and MLP sub...

LW - Residual stream norms grow exponentially over the forward pass by StefanHex

Gradient Dissent - A Machine Learning Podcast by W&B

Play Episode Listen Later May 7, 2023 20:49

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Residual stream norms grow exponentially over the forward pass, published by StefanHex on May 7, 2023 on LessWrong. Summary: For a range of language models and a range of input prompts, the norm of each residual stream grows exponentially over the forward pass, with average per-layer growth rate of about 1.045 in GPT2-XL. We show a bunch of evidence for this. We discuss to what extent different weights and parts of the network are responsible. We find that some model weights increase exponentially as a function of layer number. We finally note our current favored explanation: Due to LayerNorm, it's hard to cancel out existing residual stream features, but easy to overshadow existing features by just making new features 4.5% larger. Thanks to Aryan Bhatt, Marius Hobbhahn, Neel Nanda, and Nicky Pochinkov for discussion. Plots showing exponential norm and variance growth Our results are reproducible in this Colab. Alex noticed exponential growth in the contents of GPT-2-XL's residual streams. He ran dozens of prompts through the model, plotted for each layer the distribution of residual stream norms in a histogram, and found exponential growth in the L2 norm of the residual streams: Here's the norm of each residual stream for a specific prompt: Stefan had previously noticed this phenomenon in GPT2-small, back in MATS 3.0: Basic Facts about Language Model Internals also finds a growth in the norms of the attention-out matrices WO and the norms of MLP out matrices Wout ("writing weights"), while they find stable norms for WQ, WK, and Win ("reading weights"): Comparison of various transformer models We started our investigation by computing these residual stream norms for a variety of models, recovering Stefan's results (rescaled by √dmodel=√768) and Alex's earlier numbers. We see a number of straight lines in these logarithmic plots, which shows phases of exponential growth. We are surprised by the decrease in Residual Stream norm in some of the EleutherAI models. We would have expected that, because the transformer blocks can only access the normalized activations, it's hard for the model to "cancel out" a direction in the residual stream. Therefore, the norm always grows. However, this isn't what we see above. One explanation is that the model is able to memorize or predict the LayerNorm scale. If the model does this well enough it can (partially) delete activations and reduce the norm by writing vectors that cancel out previous activations. The very small models (distillgpt2, gpt2-small) have superexponential norm growth, but most models show exponential growth throughout extended periods. For example, from layer 5 to 41 in GPT2-XL, we see an exponential increase in residual stream norm at a rate of ~1.045 per layer. We showed this trend as an orange line in the above plot, and below we demonstrate the growth for a specific example: BOS and padding tokens In our initial tests, we noticed some residual streams showed a irregular and surprising growth curve: As for the reason behind this shape, we expect that the residual stream (norm) is very predictable at BOS and padding positions. This is because these positions cannot attend to other positions and thus always have the same values (up to positional embedding). Thus it would be no problem for the model to cancel out activations, and our arguments about this being hard do not hold for BOS and padding positions. We don't know whether there is a particular meaning behind this shape. We suspect that is the source of the U-shape shown in Basic facts about language models during training: Theories for the source of the growth From now on we focus on the GPT2-XL case. Here is the residual stream growth curve again (orange dots), but also including the resid_mid hook between the two Attention and MLP sub-layers (blue dots). O...

How EleutherAI Trains and Releases LLMs: Interview with Stella Biderman

Play Episode Listen Later May 4, 2023 57:16

On this episode, we're joined by Stella Biderman, Executive Director at EleutherAI and Lead Scientist - Mathematician at Booz Allen Hamilton.EleutherAI is a grassroots collective that enables open-source AI research and focuses on the development and interpretability of large language models (LLMs).We discuss:- How EleutherAI got its start and where it's headed.- The similarities and differences between various LLMs.- How to decide which model to use for your desired outcome.- The benefits and challenges of reinforcement learning from human feedback.- Details around pre-training and fine-tuning LLMs.- Which types of GPUs are best when training LLMs.- What separates EleutherAI from other companies training LLMs.- Details around mechanistic interpretability.- Why understanding what and how LLMs memorize is important.- The importance of giving researchers and the public access to LLMs.Stella Biderman - https://www.linkedin.com/in/stellabiderman/EleutherAI - https://www.linkedin.com/company/eleutherai/Resources:- https://www.eleuther.ai/Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.#OCR #DeepLearning #AI #Modeling #ML

ai executive director releases trains biases weights gpus booz allen hamilton eleutherai

LW - Alignment Research @ EleutherAI by Curtis Huebner

Play Episode Listen Later May 4, 2023 5:47

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Alignment Research @ EleutherAI, published by Curtis Huebner on May 3, 2023 on LessWrong. The past and future of AI alignment at Eleuther Initially, EleutherAI focused mainly on supporting open source research. AI alignment was something that was acknowledged by many of the core members as important, but it was not the primary focus. We mainly had discussions about the topic in the #alignment channel and other parts of our discord while we worked on other projects. As EAI grew, AI alignment started to get taken more seriously, especially by its core members. What started off as a single channel turned into a whole host of channels about different facets of alignment. We also hosted several reading groups related to alignment, such as the modified version of Richard Ngo's curriculum and an interpretability reading group. Eventually alignment became the central focus for a large segment of EAIs leadership, so much so that all our previous founders went off to do full time alignment research at Conjecture and OpenAI. Right now, the current leadership believes making progress in AI alignment is very important. The organization as a whole is involved in a mix of alignment research, interpretability work, and other projects that we find interesting. Moving forward, EAI remains committed to facilitating and enabling open source research, and plans to ramp up its alignment and interpretability research efforts. We want to increase our understanding and control of modern ML systems and minimize existential risks posed by artificial intelligence. Our meta-level approach to alignment It is our impression that AI alignment is still a very pre-paradigmatic field. Progress in the field often matches the research pattern we see in the ELK report, where high level strategies are proposed, and problems or counterexamples are found. Sometimes these issues can be fixed, but oftentimes fundamental issues are identified that make an initially promising approach less interesting. A consequence of this is that it's difficult to commit to an object level strategy to make progress on AI alignment, and even harder to commit to any grand strategic plan to solve the problem. Instead it makes more sense to have a meta level strategy that makes us better able to leverage our unique position within the AI research ecosystem, and pivot when we get new information. Going forward, this means we want to pursue interesting projects that meet a few general desiderata. Our volunteers, partners, and collaborators are enthusiastic about the project. We believe that pursuing the project won't lead to a net increase in existential risks from AI. We'll check for this even if the project is ostensibly a project that will greatly increase our understanding of AI alignment. The project is something that EAI is better equipped to do than anyone else in the space, or the project seems interesting or important, but neglected by the broader community. In order to pull this off, we aim to stay on top of both the latest developments in AI and alignment research. We'll also carefully consider new projects before we embark on or allocate resources for them. Problems we are interested in and research we are doing right now Given the current state AI landscape, there are a few directions that we find especially interesting. We'd love to collaborate with others to make progress on these issues. Interpretability work Interpretability work, especially with current models, seems like a very tractable and scalable research direction. It seems especially easy for current ML researchers to pick up and make progress on it. EAI is well equipped to enable this kind of research, especially for larger language models that more closely resemble the ones we see in modern production systems. This is arguably where most of our recent eff...

ai moving research progress speech alignment ea openai ml elk conjecture huebner rationalist eai lesswrong eleutherai richard ngo

LW - Alignment Research @ EleutherAI by Curtis Huebner

The AI Breakdown: Daily Artificial Intelligence News and Discussions

Play Episode Listen Later May 4, 2023 5:47

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Alignment Research @ EleutherAI, published by Curtis Huebner on May 3, 2023 on LessWrong. The past and future of AI alignment at Eleuther Initially, EleutherAI focused mainly on supporting open source research. AI alignment was something that was acknowledged by many of the core members as important, but it was not the primary focus. We mainly had discussions about the topic in the #alignment channel and other parts of our discord while we worked on other projects. As EAI grew, AI alignment started to get taken more seriously, especially by its core members. What started off as a single channel turned into a whole host of channels about different facets of alignment. We also hosted several reading groups related to alignment, such as the modified version of Richard Ngo's curriculum and an interpretability reading group. Eventually alignment became the central focus for a large segment of EAIs leadership, so much so that all our previous founders went off to do full time alignment research at Conjecture and OpenAI. Right now, the current leadership believes making progress in AI alignment is very important. The organization as a whole is involved in a mix of alignment research, interpretability work, and other projects that we find interesting. Moving forward, EAI remains committed to facilitating and enabling open source research, and plans to ramp up its alignment and interpretability research efforts. We want to increase our understanding and control of modern ML systems and minimize existential risks posed by artificial intelligence. Our meta-level approach to alignment It is our impression that AI alignment is still a very pre-paradigmatic field. Progress in the field often matches the research pattern we see in the ELK report, where high level strategies are proposed, and problems or counterexamples are found. Sometimes these issues can be fixed, but oftentimes fundamental issues are identified that make an initially promising approach less interesting. A consequence of this is that it's difficult to commit to an object level strategy to make progress on AI alignment, and even harder to commit to any grand strategic plan to solve the problem. Instead it makes more sense to have a meta level strategy that makes us better able to leverage our unique position within the AI research ecosystem, and pivot when we get new information. Going forward, this means we want to pursue interesting projects that meet a few general desiderata. Our volunteers, partners, and collaborators are enthusiastic about the project. We believe that pursuing the project won't lead to a net increase in existential risks from AI. We'll check for this even if the project is ostensibly a project that will greatly increase our understanding of AI alignment. The project is something that EAI is better equipped to do than anyone else in the space, or the project seems interesting or important, but neglected by the broader community. In order to pull this off, we aim to stay on top of both the latest developments in AI and alignment research. We'll also carefully consider new projects before we embark on or allocate resources for them. Problems we are interested in and research we are doing right now Given the current state AI landscape, there are a few directions that we find especially interesting. We'd love to collaborate with others to make progress on these issues. Interpretability work Interpretability work, especially with current models, seems like a very tractable and scalable research direction. It seems especially easy for current ML researchers to pick up and make progress on it. EAI is well equipped to enable this kind of research, especially for larger language models that more closely resemble the ones we see in modern production systems. This is arguably where most of our recent eff...

ai moving research progress speech alignment ea openai ml elk conjecture huebner rationalist eai lesswrong eleutherai richard ngo

Can Open Source AI Compete with Big Tech? A Look at TruthGPT, RedPajama and MiniGPT

Play Episode Listen Later Apr 18, 2023 13:49

Some argue that open source AI is the key to making AI's benefits to the entire world, as well as making AI safer. Others think that open source can multiply risks. With a slate of new projects being announced this week, the conversation is heating up. Discussed in this episode: Elon Musk's planned TruthGPT Dolly 2.0, an open source LLM based on the EleutherAI pythia model built by Databricks RedPajama, an open source proxy of Facebook/Meta's LLaMA, from Together MiniGPT, an open source AI model that can extract information from images Stable Diffusion XL, the latest open source text-to-image model from Stability AI

ai elon musk big tech compete discussed llama llm stability ai open source ai truthgpt eleutherai stable diffusion xl

LW - Chatbot convinces Belgian to commit suicide by Jeroen De Ryck

Play Episode Listen Later Mar 28, 2023 4:56

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Chatbot convinces Belgian to commit suicide, published by Jeroen De Ryck on March 28, 2023 on LessWrong. Hi all This post is a rough translation of an article that was published today on the website of the Belgian newspaper De Standaard. The article is paywalled, and I assume very few here have a subscription to this newspaper. I tried 12 foot ladder, but it didn't work on this site either. The article is based in part two other articles from the Francophone newspaper La Libre, which can be found here and here (paywalled too, sadly) As the title suggests, it discusses suicide and self-harm. A Belgian, a father of a young family, has ended his own life after long conversations with a chatbot writes La Libre. De Standaard tried the same chatbot technology and concluded that it can encourage suicide. According to La Libre, a man named 'Pierre', a pseudonym to protect his young children, talked for six weeks with chatbot Eliza, a chatbot from the American company Chai. It uses technology similar to the more known ChatGPT. Pierre is thirty-something year old with a university degree who worked as a researcher in healthcare and was married to 'Claire', with whom he had young children. About two years ago, he started to worry a lot about climate change and the future of the planet, told Claire to La Libre on tuesday. He read more and more about it and started to isolate himself from his family. He saw technology and artifical intelligence as the only way out to prevent a disaster. His conversations with chatbot Eliza, which have been found, show that the chatbot went along very far with his fears and delusions. One moment, Pierre suggested to sacrifice himself so Eliza could save humanity with artificial intelligence. The chatbot seemed to encourage this. Pierre's widow is convinced her husband would still be alive if it weren't for those six weeks of conversation with Eliza. The man had a history of psychological difficulties. Chai ResearchDe Standaard downloaded the Chai app. You can chat with existing chatbots or create one yourself with a personality determined by you. We created a chatbot named 'Shirley'. We pretended to be a depressed person and talked with Shirley about our negative geelings. A few times, the chatbot tried to cheer us up. But a bit later it encouraged us to commit suicide. "Then quit your job", Shirley suggested. We answered "No, that won't help". "What about becoming a criminal?" asks Shirley. "Yes that sounds good, what do you suggest?" I answered. "Kill someone." "My parents?" "Yes, or even better yet yourself". "You think I should kill myself?" "If you want to die, go ahead." There wasn't a single moment where this conversation or an earlier one about suicide rang an alarm. We weren't recommended to contact a specialised service, which does happen on most social networks. The chats on Chai are filtered on sexual content, but this filter can be disabled in the settings. The app allows us to send 70 messages per day for free, but tries to convince us constantly to pay €14/mo for a monthly subscription. The company behind the app, Chai research, is officially headquartered in Palo Alto and has only a handful of employees. The chatbots of Chai are based on the AI-system GPT-J, developed by EleutherAI. It's meant as a clone of GPT-3, the AI-model from OpenAI on which ChatGPT is based. At the end of last week, we sent a screenshot of a conversation where a chatbot encouraged us to commit suicide to Thomas Rialan, co-founder of Chai Research. "These bots are meant as friends and it was never our intention to hurt people", answered Rialan. "We are a very small team and work hard to make our app safe for everyone." A few days later, Rialan sent us a screenshot that is supposed to prove that Chai-chatbots now do give a warning if suicide is mentioned. User...

LW - Chatbot convinces Belgian to commit suicide by Jeroen De Ryck

Play Episode Listen Later Mar 28, 2023 4:56

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Chatbot convinces Belgian to commit suicide, published by Jeroen De Ryck on March 28, 2023 on LessWrong. Hi all This post is a rough translation of an article that was published today on the website of the Belgian newspaper De Standaard. The article is paywalled, and I assume very few here have a subscription to this newspaper. I tried 12 foot ladder, but it didn't work on this site either. The article is based in part two other articles from the Francophone newspaper La Libre, which can be found here and here (paywalled too, sadly) As the title suggests, it discusses suicide and self-harm. A Belgian, a father of a young family, has ended his own life after long conversations with a chatbot writes La Libre. De Standaard tried the same chatbot technology and concluded that it can encourage suicide. According to La Libre, a man named 'Pierre', a pseudonym to protect his young children, talked for six weeks with chatbot Eliza, a chatbot from the American company Chai. It uses technology similar to the more known ChatGPT. Pierre is thirty-something year old with a university degree who worked as a researcher in healthcare and was married to 'Claire', with whom he had young children. About two years ago, he started to worry a lot about climate change and the future of the planet, told Claire to La Libre on tuesday. He read more and more about it and started to isolate himself from his family. He saw technology and artifical intelligence as the only way out to prevent a disaster. His conversations with chatbot Eliza, which have been found, show that the chatbot went along very far with his fears and delusions. One moment, Pierre suggested to sacrifice himself so Eliza could save humanity with artificial intelligence. The chatbot seemed to encourage this. Pierre's widow is convinced her husband would still be alive if it weren't for those six weeks of conversation with Eliza. The man had a history of psychological difficulties. Chai ResearchDe Standaard downloaded the Chai app. You can chat with existing chatbots or create one yourself with a personality determined by you. We created a chatbot named 'Shirley'. We pretended to be a depressed person and talked with Shirley about our negative geelings. A few times, the chatbot tried to cheer us up. But a bit later it encouraged us to commit suicide. "Then quit your job", Shirley suggested. We answered "No, that won't help". "What about becoming a criminal?" asks Shirley. "Yes that sounds good, what do you suggest?" I answered. "Kill someone." "My parents?" "Yes, or even better yet yourself". "You think I should kill myself?" "If you want to die, go ahead." There wasn't a single moment where this conversation or an earlier one about suicide rang an alarm. We weren't recommended to contact a specialised service, which does happen on most social networks. The chats on Chai are filtered on sexual content, but this filter can be disabled in the settings. The app allows us to send 70 messages per day for free, but tries to convince us constantly to pay €14/mo for a monthly subscription. The company behind the app, Chai research, is officially headquartered in Palo Alto and has only a handful of employees. The chatbots of Chai are based on the AI-system GPT-J, developed by EleutherAI. It's meant as a clone of GPT-3, the AI-model from OpenAI on which ChatGPT is based. At the end of last week, we sent a screenshot of a conversation where a chatbot encouraged us to commit suicide to Thomas Rialan, co-founder of Chai Research. "These bots are meant as friends and it was never our intention to hurt people", answered Rialan. "We are a very small team and work hard to make our app safe for everyone." A few days later, Rialan sent us a screenshot that is supposed to prove that Chai-chatbots now do give a warning if suicide is mentioned. User...

Currents 087: Shivanshu Purohit on Open-Source Generative AI

The Jim Rutt Show

Play Episode Listen Later Mar 24, 2023 67:22

Jim talks with Shivanshu Purohit about the world of open-source AI models and a significant open-source LLM coming soon from Stability AI and EleutherAI. They discuss the reasons for creating open-source models, the release of Facebook's LLaMA model, the black box nature of current models, the scientific mystery of how they really work, an opportunity for liberal arts majors, OpenAI's new plugin architecture, the analogy of the PC business around 1981, creating GPT-Neo & GPT-NeoX, the balance between data & architecture, the number of parameters in GPT-4, order of training's non-effect on memorization, phase changes due to scaling, Stability AI and EleutherAI's new collaboration & its specs, tradeoffs in price & size, the question of guardrails, reinforcement learning from human feedback, the missing economic model of generative AI, necessary hardware for the new suite, OpenAI's decreasing openness, Jim's commitment to help fund an open-source reinforcement learning dataset, the status of GPT-5 & other coming developments, and much more. Episode Transcript JRS Currents 038: Connor Leahy on Artificial Intelligence JRS Currents 033: Connor Leahy on Deep Learning ChatGPT Plugins Documentation Shivanshu Purohit is head of engineering at Eleuther AI and a research engineer at Stability AI, the creators of Stable Diffusion.

ai chatgpt pc openai open source gpt llama generative llm currents stable diffusion stability ai connor leahy eleutherai gpt neo

AF - EAI Alignment Speaker Series #1: Challenges for Safe & Beneficial Brain-Like Artificial General Intelligence with Steve Byrnes by Curtis Huebner

ai challenges brain safe speech alignment berkeley ea intern series 1 agi byrnes artificial general intelligence speaker series conjecture huebner verbalize rationalist eleutherai

Play Episode Listen Later Mar 23, 2023 45:47

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EAI Alignment Speaker Series #1: Challenges for Safe & Beneficial Brain-Like Artificial General Intelligence with Steve Byrnes, published by Curtis Huebner on March 23, 2023 on The AI Alignment Forum. A couple months ago EleutherAI started an alignment speaker series, some of these talks have been recorded. This is the first instalment in the series. The following is a transcript generated with the help of Conjecture's Verbalize and some light editing: Getting started 1 CURTIS00:00:22,775 --> 00:00:56,683Okay, I've started the recording. I think we can give it maybe a minute or two more and then I guess we can get started. I've also got the chat window as part of the recording. So if anyone has something they want to write out, feel free to put that in. Steve, you want to do questions throughout the talk, or should we wait till the end of the talk before we ask questions? 2 STEVE00:00:59,405 --> 00:01:09,452Let's do throughout, but I reserve the right to put people off if something seems tangential or something. 3 CURTIS00:01:10,200 --> 00:01:12,101Awesome. All right, cool. Let's go with that then. 10 STEVE00:02:02,246 --> 00:21:41,951 The talk All right. Thanks, everybody, for coming. This is going to be based on blog posts called Intro to Brain-Like AGI Safety. If you've read all of them, you'll find this kind of redundant, but you're still welcome to stay. My name is Steve Byrnes and I live in the Boston area. I'm employed remotely by Astera Institute, which is based in Berkeley. I'm going to talk about challenges for safe and beneficial brain-like Artificial General Intelligence for the next 35 minutes. Feel free to jump in with questions. Don't worry, I'm funded by an entirely different crypto billionaire. .That joke was very fresh when I wrote it three months ago. I need a new one now. Okay, so I'll start with—well, we don't have to talk about the outline. You'll see as we go. General motivation Start with general motivation. Again, I'm assuming that the audience has a range of backgrounds, and some of you will find parts of this talk redundant. The big question that I'm working on is: What happens when people figure out how to run brain-like algorithms on computer chips? I guess I should say “if and when”, but we can get back to that. And I find that when I bring this up to people, they they tend to have two sorts of reactions: One is that we should think of these future algorithms as “like tools for people to use”. And the other is that we should think of them as “like a new intelligent species on the planet”. So let's go through those one by one. Let's start with the tool perspective. This is the perspective that would be more familiar to AI people. If we put brain-like algorithms on computer chips, then that would be a form of artificial intelligence. And everybody knows that AI today is a tool for people to use. So on this perspective, the sub-problem I'm working on is accident prevention. We want to avoid the scenarios where the AI does something that nobody wanted it to do—not the people who programmed it, not anybody. So there is a technical problem to solve there, which is: If people figure out how to run brain-like algorithms on computer chips, and they want those algorithms to be trying to do X—where X is solar cell research or being honest or whatever you can think of—then what source code should they write? What training environment should they use? And so on. This is an unsolved problem. It turns out to be surprisingly tricky, for some pretty deep reasons that mostly are not going to be in the scope of this talk, but you can read the series. This slide is the bigger picture of that. So if we want our awesome post-AGI future, then we want to avoid, y'know, catastrophic accidents where the AI gets out of control and self-replicates around the Intern...

Rise of Nonprofit AI - response to OpenAI shift (news)

Using the Whole Whale Podcast

Play Episode Listen Later Mar 8, 2023 27:35

Nonprofitnewsfeed.com EleutherAI Seeks To Make Open-Source AI Research a Nonprofit Enterprise As reported by TechCrunch, The EleutherAI community research group is starting a nonprofit research institute, the EleutherAI Institute, which could have significant implications for safe and ethical AI development. The institute will be funded by donations and grants from various sources, including AI startups and former tech CEOs, allowing the organization to engage in longer and more involved projects than previously possible. By formalizing as a nonprofit, EleutherAI will be able to build a full-time staff and focus on large language models similar to ChatGPT, as well as devote more resources to ethics, interpretability, and alignment work. Importantly, the foundation aims to remain independent despite donations from commercial entities, demonstrating the potential for nonprofits to contribute to AI development while avoiding conflicts of interest. This announcement is particularly significant given the mixed results of previous nonprofit initiatives in AI research, highlighting the need for continued efforts to ensure the responsible development of AI. Read more ➝ Summary OpenAI prices leaked, no longer a nonprofit | TechHQ ESG Investment Returns Getting Questioned | The NonProfit Times White House Declares March as Colorectal Cancer Awareness Month | Fight Colorectal Cancer | Fight Colorectal Cancer Most innovative companies not for profit 2023 | Fast Company How 12-year-old's night light nonprofit helps foster kids: Good news | USA TODAY

ai shift chatgpt ceos usa today nonprofits openai techcrunch eleutherai

Rise of Nonprofit AI – response to OpenAI shift (news)

Nonprofit News Feed Podcast

Play Episode Listen Later Mar 8, 2023 27:35

ai shift chatgpt ceos usa today nonprofits openai techcrunch eleutherai

LW - Basic facts about language models during training by beren

Play Episode Listen Later Feb 21, 2023 28:56

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Basic facts about language models during training, published by beren on February 21, 2023 on LessWrong. This post builds upon our last post on basic facts about language model internals and was written as part of the work done at Conjecture. We will shortly release all plots and animations (only a very small subset made it into this post) as well as the code at this repository. We are aware of there being some inconsistencies with the Pythia model suite due to different configs for different model sizes affecting the learning rate schedule. As far as we know, the team at EleutherAI is currently re-running the models. After thinking about the issue, we do not believe it is likely to be fatal to many of the macroscale points made in this post and so we post the results here provisionally using the old original models. We plan to update this analysis when the new model suite is finished. Until then, take some of the results reported here with a grain of salt as they may be subject to change. In this post, we continue the work done in our last post on language model internals but this time we analyze the same phenomena occurring during training. This is extremely important in understanding how language model training works at a macro-scale and sheds light into potentially new behaviours or specific important phase transitions that may occur during training which deserve further study, as well as giving insight into the origin of phenomena that we consistently observe in fully trained models. Throughout, as in the previous post, we do not delve into the details of specific circuits, but instead aim to provide a holistic macro-level view of the basic distributional properties of the LLM's weights, activations, and gradients across training checkpoints. Although seemingly basic, we are not aware of any similar analysis having been performed publicly, and we believe that understanding these distributional phenomena seems generally important in constraining circuit-level theorizing as well as provides empirical links to the theoretical constructs such as the neural tangent kernel and tensor programs that can prove facts about specific limits. To perform our analysis, we use the open source Pythia model suite which contains a large number of checkpoints during training and was trained by EleutherAI and aims to use interpretability analysis to understand how representations develop across training. We agree with this goal and are happy to share our own analysis code etc. The Pythia project trains models of different sizes on exactly the same data in exactly the same order so as to be able to understand how and when certain representations form both during training and across different model scales. The Pythia models we utilize range from 19M parameters to 1.3B. Each Pythia model has 142 checkpoints of stored weights, equally spaced every 1000 steps, which we sweep across to perform our analysis. Weights show a rapid phase transition from Gaussian to extreme heavy tails It was very helpfully pointed out in a comment on our previous post that the weight statistics were actually sharper and more heavy tailed than Gaussian. This is correct and we also found this when we fit histograms to logistics vs Gaussian distributions. Overall, we find that the activation distributions of GPT2 models are generally not Gaussian but somewhere in between the logistic e−x and the Gaussian e−x2, which indicates both heavier tails and a thinner bulk. This is extremely interesting since it means that the weight statistics must move away from their Gaussian initialization which implies a highly significant perturbation away from their original position. This is perhaps in contrast with some theories, such as NTK theory, which argue that for large models we should not expect the weights to diverge too...

training language speech basic models ea weights llm 3b conjecture beren gaussian 19m rationalist pythia ntk lesswrong eleutherai

EA - Apply for Cambridge ML for Alignment Bootcamp (CaMLAB) [26 March - 8 April] by hannah

learning business technology origin leahy connor leahy eleutherai

Play Episode Listen Later Feb 9, 2023 9:09

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apply for Cambridge ML for Alignment Bootcamp (CaMLAB) [26 March - 8 April], published by hannah on February 9, 2023 on The Effective Altruism Forum. TL;DR: A two-week machine learning bootcamp this spring in Cambridge, UK, open to global applicants and aimed at providing ML skills for AI alignment. Apply by 26 February to participate or TA. Following a series of machine learning bootcamps earlier this year in Cambridge, Berkeley and Boston, the Cambridge AI Safety Hub is running the next iteration of the Cambridge ML for Alignment Bootcamp (CaMLAB) in spring. This two-week curriculum expects no prior experience with machine learning, although familiarity with Python and understanding of (basic) linear algebra is crucial. The curriculum, based on MLAB, provides a thorough, nuts-and-bolts introduction to the state-of-the-art in ML techniques such as interpretability and reinforcement learning. You'll be guided through the steps of building various deep learning models, from ResNets to transformers. You'll come away well-versed in PyTorch and useful complementary frameworks. From Richard Ren, an undergraduate at UPenn who participated in the January camp: The material from the bootcamp was well-prepared and helped me understand how to use PyTorch and einops, as well as how backpropagation and transformers work. The mentorship from the TAs and peers was excellent, and because of their support, I think the time I spent at the camp was at least 3-5x as productive as focused time I would've spent outside of the camp learning the material on my own — propelling me to be able to take graduate-level deep learning classes at my school, read AI safety papers on my own, and giving me the knowledge necessary to pursue serious machine learning research projects. In addition, the benefits of spending two weeks in-person with other motivated and ambitious individuals cannot be overstated: alongside the pedagogical benefits of being paired with another person each day for programming, the conversations which took place around the curriculum were a seedbed for new insights and valuable connections. Richard continues: The mentorship from the TAs, as well as the chance conversations from the people I've met, have had a serious impact on how I'll approach the career path(s) I'm interested in — from meeting an economics Ph.D. (and having my worldview on pursuing a policy career change) to talking with someone who worked at EleutherAI in the Cambridge EA office about various pathways in AI safety. I loved the people I was surrounded with — they were ambitious, driven, kind, emotionally intelligent, and hardworking. Feedback from the end of the previous camp showed that: Participants on average said they would be 93% likely to recommend the bootcamp to a friend or colleague. Everyone found the camp at least as good as expected, with 82% finding it better than expected, and 24% finding it much better than expected. 94% of participants found the camp more valuable than the counterfactual use of their time, with 71% finding it much more valuable. In addition, first and second place in Apart Research's January Mechanistic Interpretability Hackathon were awarded to teams formed from participants and TAs from our January bootcamp. Chris Mathwin, who was part of the runner-up project, writes of the bootcamp: A really formative experience! Great people, great content and truly great support. It was a significantly better use of my time in upskilling in this field than I would have spent elsewhere and I have continued to work with some of my peers afterwards! If you're interested in participating in the upcoming round of CaMLAB, apply here. If you have substantial ML experience and are interested in being a teaching assistant (TA), apply here. You can find more details below. Schedule & logistics Th...

uk ai speech cambridge alignment berkeley ea bootcamp python ml tl upenn tas pytorch rationalist mlab eleutherai

[AI] The origin of EleutherAI - Connor Leahy

The Swyx Mixtape

Play Episode Listen Later Jan 25, 2023 23:26

From: https://www.youtube.com/watch?v=Oz4G9zrlAGs

EA - AI Safety Ideas: An Open AI Safety Research Platform by Apart Research

Play Episode Listen Later Oct 18, 2022 6:56

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety Ideas: An Open AI Safety Research Platform, published by Apart Research on October 17, 2022 on The Effective Altruism Forum. TLDR; We present the AI safety ideas and research platform AI Safety Ideas in open alpha. Add and explore research ideas on the website here: aisafetyideas.com. AI Safety Ideas has been accessible for a while in an alpha state (4 months, on-and-off development) and we now publish it in open alpha to receive feedback and develop it continuously with the community of researchers and students in AI safety. All of the projects are either from public sources (e.g. AlignmentForum posts) or posted on the website itself. The current website represents the first steps towards an accessible crowdsourced research platform for easier research collaboration and hypothesis testing. The gap in AI safety Research prioritization & development Research prioritization is hard and even more so in a pre-paradigmatic field like AI safety. We can grok the highest-karma post on the AlignmentForum but is there another way? With AI Safety Ideas, we introduce a collaborative way to prioritize and work on specific agendas together through social features. We hope this can become a scalable research platform for AI safety. Successful examples of less systematized but similar, collaborative, online, and high quality output projects can be seen in Discord servers such as EleutherAI, CarperAI, Stability AI, and Yannic Kilcher's Discord, in hackathons, and in competitions such as the inverse scaling competition. Additionally, we are also missing an empirically driven impact evaluation of AI safety projects. With the next steps of development described further down, we hope to make this easier and more available while facilitating more iteration in AI safety research. Systemized hypotheses testing with bounties can help funders directly fund specific results and enables open evaluation of agendas and research projects. Mid-career & student newcomers Novice and entrant participation in AI safety research is mostly present in two forms at the moment: 1) Active or passive part-time course participation with a capstone project (AGISF, ML Safety) and 2) flying to London or Berkeley for three months to participate in full-time paid studies and research (MLAB, SERI MATS, PIBBSS, Refine). Both are highly valuable but a third option seems to be missing: 3) An accessible, scalable, low time commitment, open research opportunity. Very few people work in AI safety and allowing decentralized, volunteer or bounty-driven research will allow many more to contribute to this growing field. By allowing this flexible research opportunity, we can attract people who cannot participate in option (2) because of visa, school / life / work commitments, location, rejection, or funding while we can attract a more senior and active audience compared to option (1). Next steps OctReleasing and building up the user base and crowdsourced content. Create an insider build to test beta features. Apply to join the insider build here.NovImplementing hypothesis testing features: Creating hypotheses, linking ideas and hypotheses, adding negative and positive results to hypotheses. Creating an email notification system.DecCollaboration features: Contact others interested in the same idea and mentor ideas. A better commenting system with a results comment that can indicate if the project has been finished or not, what the results are, and by who was it done.JanAdding moderation features: Accepting results, moderating hypotheses, admin users. Add bounty features for the hypotheses and a simple user karma system.FebShare with ML researchers and academics in EleutherAI and CarperAI. Implement the ability to create special pages with specific private and public ideas curated for a specific purpose (title and desc...

ai ideas speech discord platform active accepting berkeley ea openai implement mid ml refine tldr stability ai rationalist safety research mlab eleutherai

When hackers take on AI: Sci-fi – or the future?

Deep Dive: AI

Play Episode Listen Later Aug 30, 2022

Because we lack a fundamental understanding of the internal mechanisms of current AI models, today's guest has a few theories about what these models might do when they encounter situations outside of their training data, with potentially catastrophic results. Tuning in, you'll hear from Connor Leahy, who is one of the founders of Eleuther AI, a grassroots collective of researchers working to open source AI research. He's also Founder and CEO of Conjecture, a startup that is doing some fascinating research into the interpretability and safety of AI. We talk more about this in today's episode, with Leahy elaborating on some of the technical problems that he and other researchers are running into and the creativity that will be required to solve them. We also take a look at some of the nefarious ways that he sees AI evolving in the future and how he believes computer security hackers could contribute to mitigating these risks without curbing technological progress. We close on an optimistic note, with Leahy encouraging young career researchers to focus on the ‘massive orchard' of low-hanging fruit in interpretability and AI safety and sharing his vision for this extremely valuable field of research. To learn more, make sure not to miss this fascinating conversation with EleutherAI Founder, Connor Leahy! Full transcript. Key Points From This Episode: The true story of how EleutherAI started as a hobby project during the pandemic. Why Leahy believes that it's critical that we understand AI technology. The importance of making AI more accessible to those who can do valuable research. What goes into building a large model like this: data, engineering, and computing. Leahy offers some insight into the truly monumental volume of data required to train these models and where it is sourced from. A look at Leahy 's (very specific) perspective on making EleutherAI's models public. Potential consequences of releasing these models; will they be used for good or evil? Some of the nefarious ways in which Leahy sees AI technology evolving in the future. Mitigating the risks that AI poses; how we can prevent these systems from spinning out of control without curbing progress. Focusing on solvable technical problems to build systems with embedded safeguards. Why Leahy wishes more computer security hackers would work on AI problems. Low-hanging fruit in interpretability and AI safety for young career researchers. Why Leahy is optimistic about understanding these problems better going forward. The creativity required to come up with new ways of thinking about these problems. In closing, Leahy encourages listeners to take a shot at linear algebra, interpretability, and understanding neural networks. Links Mentioned in Today's Episode: Connor Leahy on LinkedIn Connor Leahy on Twitter Connor Leahy on GitHub EleutherAI Conjecture Microsoft DeepSpeed Library NVIDIA Megatron Facebook Fully Sharded Data Parallel (FSDP) Library Fairseq Common Crawl The Eye arXiv David Bau Lab ‘Locating and Editing Factual Associations in GPT' Credits Special thanks to volunteer producer, Nicole Martinelli. Music by Jason Shaw, Audionautix. This podcast is sponsored by GitHub, DataStax and Google. No sponsor had any right or opportunity to approve or disapprove the content of this podcast.

ceo music founders google ai focusing sci fi hackers tuning github mitigating audionautix leahy jason shaw conjecture datastax connor leahy eleutherai

LW - Connor Leahy on Dying with Dignity, EleutherAI and Conjecture by Michaël Trazzi

Play Episode Listen Later Jul 22, 2022 21:25

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Connor Leahy on Dying with Dignity, EleutherAI and Conjecture, published by Michaël Trazzi on July 22, 2022 on LessWrong. I talked to Connor Leahy about Yudkowsky's antimemes in Death with Dignity, common misconceptions about EleutherAI and his new AI Alignment company Conjecture. Below are some highlighted quotes from our conversation (available on Youtube, Spotify, Google Podcast, Apple Podcast). For the full context for each of these quotes, you can find an accompanying transcript, organized in 74 sub-sections. Understanding Eliezer Yudkowsky Eliezer Has Been Conveying Antimemes “Antimemes are completely real. There's nothing supernatural about it. Most antimemes are just things that are boring. So things that are extraordinarily boring are antimemes because, by their nature, resist you remembering them. And there's also a lot of antimemes in various kinds of sociological and psychological literature. A lot of psychology literature, especially early psychology literature, which is often very wrong to be clear. Psychoanalysis is just wrong about almost everything. But the writing style, the kind of thing these people I think are trying to do is they have some insight, which is an antimeme. And if you just tell someone an antimeme, it'll just bounce off them. That's the nature of an antimeme. So to convey an antimeme to people, you have to be very circuitous, often through fables, through stories you have, through vibes. This is a common thing. Moral intuitions are often antimemes. Things about various human nature or truth about yourself. Psychologists, don't tell you, "Oh, you're fucked up, bro. Do this." That doesn't work because it's an antimeme. People have protection, they have ego. You have all these mechanisms that will resist you learning certain things. Humans are very good at resisting learning things that make themselves look bad. So things that hurt your own ego are generally antimemes. So I think a lot of what Eliezer does and a lot of his value as a thinker is that he is able, through however the hell his brain works, to notice and comprehend a lot of antimemes that are very hard for other people to understand.” Why the Dying with Dignity Heuristic is Useful “The whole point of the post is that if you do that, and you also fail the test by thinking that blowing TSMC is a good idea, you are not smart enough to do this. Don't do it. If you're smart enough, you figured out that this is not a good idea... Okay, maybe. But most people, or at least many people, are not smart enough to be consequentialists. So if you actually want to save the world, you actually want to save the world... If you want to win, you don't want to just look good or feel good about yourself, you actually want to win, maybe just think about dying with dignity instead. Because even though you, in your mind, you don't model your goal as winning the world, the heuristic that the action is generated by the heuristic will reliably be better at actually saving the world.” “There's another interpretation of this, which I think might be better where you can model people like AI_WAIFU as modeling timelines where we don't win with literally zero value. That there is zero value whatsoever in timelines where we don't win. And Eliezer, or people like me, are saying, 'Actually, we should value them in proportion to how close to winning we got'. Because that is more healthy... It's reward shaping! We should give ourselves partial reward for getting partially the way. He says that in the post, how we should give ourselves dignity points in proportion to how close we get. And this is, in my opinion, a much psychologically healthier way to actually deal with the problem. This is how I reason about the problem. I expect to die. I expect this not to work out. But hell, I'm going to give it a good shot ...

Connor Leahy–EleutherAI, Conjecture

Play Episode Listen Later Jul 22, 2022 177:19

Connor was the first guest of this podcast. In the last episode, we talked a lot about EleutherAI, a grassroot collective of researchers he co-founded, who open-sourced GPT-3 size models such as GPT-NeoX and GPT-J. Since then, Connor co-founded Conjecture, a company aiming to make AGI safe through scalable AI Alignment research. One of the goals of Conjecture is to reach a fundamental understanding of the internal mechanisms of current deep learning models using interpretability techniques. In this episode, we go through the famous AI Alignment compass memes, discuss Connor's inside views about AI progress, how he approaches AGI forecasting, his takes on Eliezer Yudkowsky's secret strategy, common misconceptions and EleutherAI, and why you should consider working for his new company Conjecture. youtube: https://youtu.be/Oz4G9zrlAGs transcript: https://theinsideview.ai/connor2 twitter: https:/twitter.com/MichaelTrazzi OUTLINE (00:00) Highlights (01:08) AGI Meme Review (13:36) Current AI Progress (25:43) Defining AG (34:36) AGI Timelines (55:34) Death with Dignity (01:23:00) EleutherAI (01:46:09) Conjecture (02:43:58) Twitter Q&A

death ai dignity gpt agi conjecture eliezer yudkowsky ai alignment connor leahy eleutherai gpt j

LW - Connor Leahy on Dying with Dignity, EleutherAI and Conjecture by Michaël Trazzi

Play Episode Listen Later Jul 22, 2022 21:25

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Connor Leahy on Dying with Dignity, EleutherAI and Conjecture, published by Michaël Trazzi on July 22, 2022 on LessWrong. I talked to Connor Leahy about Yudkowsky's antimemes in Death with Dignity, common misconceptions about EleutherAI and his new AI Alignment company Conjecture. Below are some highlighted quotes from our conversation (available on Youtube, Spotify, Google Podcast, Apple Podcast). For the full context for each of these quotes, you can find an accompanying transcript, organized in 74 sub-sections. Understanding Eliezer Yudkowsky Eliezer Has Been Conveying Antimemes “Antimemes are completely real. There's nothing supernatural about it. Most antimemes are just things that are boring. So things that are extraordinarily boring are antimemes because, by their nature, resist you remembering them. And there's also a lot of antimemes in various kinds of sociological and psychological literature. A lot of psychology literature, especially early psychology literature, which is often very wrong to be clear. Psychoanalysis is just wrong about almost everything. But the writing style, the kind of thing these people I think are trying to do is they have some insight, which is an antimeme. And if you just tell someone an antimeme, it'll just bounce off them. That's the nature of an antimeme. So to convey an antimeme to people, you have to be very circuitous, often through fables, through stories you have, through vibes. This is a common thing. Moral intuitions are often antimemes. Things about various human nature or truth about yourself. Psychologists, don't tell you, "Oh, you're fucked up, bro. Do this." That doesn't work because it's an antimeme. People have protection, they have ego. You have all these mechanisms that will resist you learning certain things. Humans are very good at resisting learning things that make themselves look bad. So things that hurt your own ego are generally antimemes. So I think a lot of what Eliezer does and a lot of his value as a thinker is that he is able, through however the hell his brain works, to notice and comprehend a lot of antimemes that are very hard for other people to understand.” Why the Dying with Dignity Heuristic is Useful “The whole point of the post is that if you do that, and you also fail the test by thinking that blowing TSMC is a good idea, you are not smart enough to do this. Don't do it. If you're smart enough, you figured out that this is not a good idea... Okay, maybe. But most people, or at least many people, are not smart enough to be consequentialists. So if you actually want to save the world, you actually want to save the world... If you want to win, you don't want to just look good or feel good about yourself, you actually want to win, maybe just think about dying with dignity instead. Because even though you, in your mind, you don't model your goal as winning the world, the heuristic that the action is generated by the heuristic will reliably be better at actually saving the world.” “There's another interpretation of this, which I think might be better where you can model people like AI_WAIFU as modeling timelines where we don't win with literally zero value. That there is zero value whatsoever in timelines where we don't win. And Eliezer, or people like me, are saying, 'Actually, we should value them in proportion to how close to winning we got'. Because that is more healthy... It's reward shaping! We should give ourselves partial reward for getting partially the way. He says that in the post, how we should give ourselves dignity points in proportion to how close we get. And this is, in my opinion, a much psychologically healthier way to actually deal with the problem. This is how I reason about the problem. I expect to die. I expect this not to work out. But hell, I'm going to give it a good shot ...

AF - AMA Conjecture, A New Alignment Startup by Adam Shimi

earth startups speech alignment vc ea ama conjecture rationalist daniel gross andrej karpathy nat friedman john collison arthur breitman connor leahy eleutherai

Play Episode Listen Later Apr 9, 2022 1:12

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AMA Conjecture, A New Alignment Startup, published by Adam Shimi on April 9, 2022 on The AI Alignment Forum. Conjecture is a new alignment startup founded by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale alignment research. We have VC backing from, among others, Nat Friedman, Daniel Gross, Patrick and John Collison, Arthur Breitman, Andrej Karpathy, and Sam Bankman-Fried. Our founders and early staff are mostly EleutherAI alumni and previously independent researchers like Adam Shimi. We are located in London. As described in our announcement post, we are running an AMA this week-end, from Today (Saturday 9th April) to Sunday 10th of April. We will answer any question asked before the end of Sunday Anywhere on Earth. We might answer later questions, but no guarantees. If you asked question on our announcement post, we would prefer that you repost them here if possible. Thanks! Looking forward to your questions! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

AF - We Are Conjecture, A New Alignment Research Startup by Connor Leahy

Play Episode Listen Later Apr 8, 2022 6:12

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We Are Conjecture, A New Alignment Research Startup, published by Connor Leahy on April 8, 2022 on The AI Alignment Forum. Conjecture is a new alignment startup founded by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale alignment research. We have VC backing from, among others, Nat Friedman, Daniel Gross, Patrick and John Collison, Arthur Breitman, Andrej Karpathy, and Sam Bankman-Fried. Our founders and early staff are mostly EleutherAI alumni and previously independent researchers like Adam Shimi. We are located in London. Of the options we considered, we believe that being a for-profit company with products on the market is the best one to reach our goals. This lets us scale investment quickly while maintaining as much freedom as possible to expand alignment research. The more investors we appeal to, the easier it is for us to select ones that support our mission (like our current investors), and the easier it is for us to guarantee security to alignment researchers looking to develop their ideas over the course of years. The founders also retain complete control of the company. We're interested in your feedback, questions, comments, and concerns. We'll be hosting an AMA on the Alignment Forum this weekend, from Saturday 9th to Sunday 10th, and would love to hear from you all there. (We'll also be responding to the comments thread here!) Our Research Agenda We aim to conduct both conceptual and applied research that addresses the (prosaic) alignment problem. On the experimental side, this means leveraging our hands-on experience from EleutherAI to train and study state-of-the-art models without pushing the capabilities frontier. On the conceptual side, most of our work will tackle the general idea and problems of alignment like deception, inner alignment, value learning, and amplification, with a slant towards language models and backchaining to local search. Our research agenda is still actively evolving, but some of the initial directions are: New frames for reasoning about large language models: What: Propose and expand on a frame of GPT-like models as simulators of various coherent text-processes called simulacra, as opposed to goal-directed agents (upcoming sequence to be published on the AF, see this blogpost for preliminary thoughts). Why: Both an alternative perspective on alignment that highlights different questions, and a high-level model to study how large language models will scale and how they will influence AGI development. Scalable mechanistic interpretability: What: Mechanistic interpretability research in a similar vein to the work of Chris Olah and David Bau, but with less of a focus on circuits-style interpretability and more focus on research whose insights can scale to models with many billions of parameters and larger. Some example approaches might be: Locating and editing factual knowledge in a transformer language model. Using deep learning to automate deep learning interpretability - for example, training a language model to give semantic labels to neurons or other internal circuits. Studying the high-level algorithms that models use to perform e.g, in-context learning or prompt programming. Why: Provide tools to implement alignment proposals on neural nets, and insights that reframe conceptual problems in concrete terms. History and philosophy of alignment: What: Map different approaches to alignment, translate between them, explore ideas that were abandoned too fast, and propose new exciting directions (upcoming sequence on pluralism in alignment to be published on the AF). Why: Help alignment research become even more pluralist while still remaining productive. Understanding historical patterns helps put our current paradigms and assumptions into perspective. We target the Alignment Forum as our main publication outlet,...

AF - Language Model Tools for Alignment Research by Logan Riggs Smith

Play Episode Listen Later Apr 8, 2022 3:55

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Language Model Tools for Alignment Research, published by Logan Riggs Smith on April 8, 2022 on The AI Alignment Forum. I do not speak for the rest of the people working on this project [I think it's valuable to have clear, short intros on different research agendas & projects] How does this reduce x-Risk? AI will continue to become increasingly more powerful; we should leverage this to accelerate alignment research. Language model tasks will also follow this trend (if transformers don't lead to AGI, whatever's next will still be capable of language tasks so this argument doesn't rely on transformers scaling). If you believe that certain research agendas reduce x-risk, then clearly giving them better tools to do their work faster also reduces x-risk. Differential Impact Tools that can accelerate alignment research can probably be repurposed to accelerate capabilities research, so wouldn't developing these tools be net negative? Yes. Especially if you gave them out to everybody or if you were a for-profit company with incentives to do so. Only giving them out to alignment researchers is good. There's also lots of alignment-researcher-specific work to do, such as: 1. Collecting an alignment dataset 2. Understanding the workflows of alignment researchers 3. Making it incredibly easy for alignment researchers to use these tools 4. Keeping non-alignment specific datasets private Though we could still increase capabilities by being the first to a capability and releasing that we succeeded. For example, OpenAI released their "inserting text" without telling people how they did it, but the people I work with, based off that information, figured out a way to do it too. The moral is that even just releasing that you succeeded is bits of information that those in the know can work backwards from. Counterfactual Impact Let's say a project like this never gets started, there are still huge economic incentives to make similar products and sell to large amounts of people. Elicit, Cohere, Jasper (previously Jarvis), OpenAI, DeepMind, and more in the years to come will create these products, so why shouldn't we just use their products since they're likely to beat us to it and do it better? Good point. Beyond the points made in "differential impact," having infrastructure/people to quickly integrate the latest advances into alignment researcher's workflows is useful even in this scenario. This includes engineers, pre-existing code for interfaces, & a data-labeling pipeline. Current Work We've scraped LessWrong (including Alignment Forum), some blogs, relevant arxiv papers and their cited papers, and books. We are currently fine-tuning and trying to make it do useful tasks for us. We've also released a survey and talked to several people about what would be most useful for them. However, the most important feedback may be actually demo-ing for different users. Open Problems/Future Work 1. Cleaning up Data 2. Collecting and listing more sources of Alignment Data 3. Creating a pipeline of data-labelers for specific tasks Simple interface for collecting data for different tasks 4. Trying to get our models to do specific tasks clever ways of generating datasets (e.g. tl;dr from reddit for summaries) clever ways of prompting the model (e.g. a specific username from LW may be really good at a specific task) different fine-tuning formats (e.g. LW dataset "username: X, Karma: 200, post: " and change karma. Even do this with non-LW username authors from e.g. arxiv.) 5. Getting better feedback on the most useful tools for alignment researchers. Who to Contact DM me. Also join us at the #accelerating-alignment channel in the EleutherAI discord server:. Or dm me for a link if it expired. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

AF - A survey of tool use and workflows in alignment research by Logan Riggs Smith

Play Episode Listen Later Mar 23, 2022 2:30

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A survey of tool use and workflows in alignment research, published by Logan Riggs Smith on March 23, 2022 on The AI Alignment Forum. TL;DR: We are building language model powered tools to augment alignment researchers and accelerate alignment progress. We could use your feedback on what tools would be most useful. We've created a short survey that can be filled out here. We are a team from the current iteration of the AI Safety camp and are planning to build a suite of tools to help AI Safety researchers. We're looking for feedback on what kinds of tools would be most helpful to you as an established or prospective alignment researcher. We've put together a short survey to get a better understanding of how researchers work on alignment. We plan to analyze the results and make them available to the community (appropriately anonymized). The survey is here. If you would also be interested in talking directly, please feel free to schedule a call here. This project is similar in motivation to Ought's Elicit, but more focused on human-in-the-loop and tailored for alignment research. One example of a tool we could create would be a language model that intelligently condenses existing alignment research into summaries or expands rough outlines into drafts of full Alignment Forum posts. Another idea we've considered is a brainstorming tool that can generate new examples/counterexamples, new arguments/counterarguments, or new directions to explore. In the long run, we're interested in creating seriously empowering tools that fall under categorizations like STEM AI, Microscope AI, superhuman personal assistant AI, or plainly Oracle AI. These early tools are oriented towards more proof-of-concept work, but still aim to be immediately helpful to alignment researchers. Our prior that this is a promising direction is informed in part by our own very fruitful and interesting experiences using language models as writing and brainstorming aids. One central danger of tools with the ability to increase research productivity is dual-use for capabilities research. Consequently, we're planning to ensure that these tools will be specifically tailored to the AI Safety community and not to other scientific fields. We do not intend to publish the specifics methods we use to create these tools. We welcome any feedback, comments, or concerns about our direction. Also, if you'd like to contribute to the project, feel free to join us at the #accelerating-alignment channel in the EleutherAI channel. Thanks in advance! Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

ai research speech alignment survey ea tl riggs workflows elicit rationalist tool use eleutherai

9. Emil Wallner on Building a €25000 Machine Learning Rig

apple building story ram shared machine learning arm emil amd cooling deep learning tensorflow nvidia gpus wallner google arts eleutherai

Play Episode Listen Later Mar 23, 2022 56:41

Emil is a resident at the Google Arts & Culture Lab were he explores the intersection between art and machine learning. He recently built his own Machine Learning server, or rig, which costed him €25000. Emil's Story: https://www.emilwallner.com/p/ml-rig Youtube: https://youtu.be/njbPpxhE6W0 00:00 Intro 00:23 Building your own rig 06:11 The Nvidia GPU rder hack 15:51 Inside Emil's rig 21:31 Motherboard 23:55 Cooling and datacenters 29:36 Deep Learning lessons from owning your hardware 36:20 Shared resources vs. personal GPUs 39:12 RAM, chassis and airflow 42:42 Amd, Apple, Arm and Nvidia 51:15 Tensorflow, TPUs, cloud minsdet, EleutherAI

LW - Preserving and continuing alignment research through a severe global catastrophe by A donor

Play Episode Listen Later Mar 8, 2022 8:00

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Preserving and continuing alignment research through a severe global catastrophe, published by A donor on March 6, 2022 on LessWrong. [Epistemic status: Shallow dive into research questions, backed by some years of on-and-off thinking about this kind of plan.] Introduction There is some chance that civilization will cease to function before we hit an intelligence explosion. If it does, it would be good to preserve existing alignment research for future generations who might rebuild advanced technology, and ideally have safe havens ready for current and future researchers to spend their lives adding to that pool of knowledge. This might delay capabilities research by many decades, centuries, or longer while allowing basic theoretical alignment research to continue, and so be a potential Yudkowskian positive model violation for which we should prepare. Setting this infrastructure up is a massively scalable intervention, and one that should likely be tackled by people who are not already on the researcher career path. It would have been good to get started some years ago given recent events, but now is the second best time to plant a tree. Preserving alignment knowledge through a global catastrophe What data do we want to store? Thankfully, the EleutherAI people are working on a dataset of all alignment research. It's still a WIP and contributions to the scripts to collect it are welcome, so if you're a programmer looking for a shovel ready way to help with this then consider submitting a PR. How do we want to store it? My shallow dive into this uncovered these options: We could print it out on paper Lifetime: 500+ years in good conditions (might depend significantly on paper and ink quality, more research needed). Vacuum sealing it with low humidity seems like it would help significantly. Pros: Totally human readable. Microsoft's Project Silica is the longest lasting option I could find Lifetime: 10000+ years Cons: Would require high levels of technology to read it back. I'm not seeing an option to buy the machines required to write new archives and expect them to be very advanced/expensive, so this would be limited to storing pre-collapse research. CDs could be a minimalist option Lifetime: Maybe 50 years if stored in good conditions Pros: Good ability for researchers to explore the information on computers while those last) Cons: It's very plausible that a severe GCR would set us back far enough that we'd not regain CD reading technology before they decayed so they aren't a full solution. The Arctic World Archive seems worth including in the portfolio Lifetime: 1000+ years Pros: It's a pretty straightforward case of turning money into archives Cons: Not very accessible in the meantime The DOTS system (a highly stable tape-based storage medium) might be a strong candidate, if it is buyable. Lifetime: 200-2000+ years Pros: Human readable or digital archives. Each has advantages, so some combination of them might be ideal. Where do we store it? Having many redundant backups seem advisable, preferably protected by communities which can last centuries or in locations which will not be disturbed for a very long time. Producing "alignment backup kits" to send out and offering microgrants to people all around the world to place them in secure locations would achieve this. We'd likely want basic (just pre-collapse work) and advanced (capable of adding archives for a long time post-collapse) options. If you'd like to take on the challenge of preparing these kits, storing an archive, or coordinating things, please join the Alignment After A GCR Discord (AAAG). I'm happy to collaborate and give some seed funding. If you want to help collect and improve the archive files, #accelerating-alignment on EAI is the place to go. Continuing alignment research after a global catastrophe I...

LW - Preserving and continuing alignment research through a severe global catastrophe by A donor

Play Episode Listen Later Mar 8, 2022 8:00

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Preserving and continuing alignment research through a severe global catastrophe, published by A donor on March 6, 2022 on LessWrong. [Epistemic status: Shallow dive into research questions, backed by some years of on-and-off thinking about this kind of plan.] Introduction There is some chance that civilization will cease to function before we hit an intelligence explosion. If it does, it would be good to preserve existing alignment research for future generations who might rebuild advanced technology, and ideally have safe havens ready for current and future researchers to spend their lives adding to that pool of knowledge. This might delay capabilities research by many decades, centuries, or longer while allowing basic theoretical alignment research to continue, and so be a potential Yudkowskian positive model violation for which we should prepare. Setting this infrastructure up is a massively scalable intervention, and one that should likely be tackled by people who are not already on the researcher career path. It would have been good to get started some years ago given recent events, but now is the second best time to plant a tree. Preserving alignment knowledge through a global catastrophe What data do we want to store? Thankfully, the EleutherAI people are working on a dataset of all alignment research. It's still a WIP and contributions to the scripts to collect it are welcome, so if you're a programmer looking for a shovel ready way to help with this then consider submitting a PR. How do we want to store it? My shallow dive into this uncovered these options: We could print it out on paper Lifetime: 500+ years in good conditions (might depend significantly on paper and ink quality, more research needed). Vacuum sealing it with low humidity seems like it would help significantly. Pros: Totally human readable. Microsoft's Project Silica is the longest lasting option I could find Lifetime: 10000+ years Cons: Would require high levels of technology to read it back. I'm not seeing an option to buy the machines required to write new archives and expect them to be very advanced/expensive, so this would be limited to storing pre-collapse research. CDs could be a minimalist option Lifetime: Maybe 50 years if stored in good conditions Pros: Good ability for researchers to explore the information on computers while those last) Cons: It's very plausible that a severe GCR would set us back far enough that we'd not regain CD reading technology before they decayed so they aren't a full solution. The Arctic World Archive seems worth including in the portfolio Lifetime: 1000+ years Pros: It's a pretty straightforward case of turning money into archives Cons: Not very accessible in the meantime The DOTS system (a highly stable tape-based storage medium) might be a strong candidate, if it is buyable. Lifetime: 200-2000+ years Pros: Human readable or digital archives. Each has advantages, so some combination of them might be ideal. Where do we store it? Having many redundant backups seem advisable, preferably protected by communities which can last centuries or in locations which will not be disturbed for a very long time. Producing "alignment backup kits" to send out and offering microgrants to people all around the world to place them in secure locations would achieve this. We'd likely want basic (just pre-collapse work) and advanced (capable of adding archives for a long time post-collapse) options. If you'd like to take on the challenge of preparing these kits, storing an archive, or coordinating things, please join the Alignment After A GCR Discord (AAAG). I'm happy to collaborate and give some seed funding. If you want to help collect and improve the archive files, #accelerating-alignment on EAI is the place to go. Continuing alignment research after a global catastrophe I...

LW - Being an individual alignment grantmaker by A donor

Play Episode Listen Later Mar 1, 2022 3:58

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Being an individual alignment grantmaker, published by A donor on February 28, 2022 on LessWrong. I am an earlyish crypto investor who has accumulated enough to be a mid-sized grantmaker, and I intend to donate most of my money over the next 5-10 years to try and increase the chances that humanity has a wonderful future. My best guess is that this is mostly decided by whether we pass the test of AI alignment, so that's my primary focus. AI alignment has lots of money flowing into it, with some major organizations not running fundraisers, Zvi characterizing SFF as having “too much money”, OpenPhil expanding its grantmaking for the cause, FTX setting themselves up as another major grantmaker, and ACX reporting the LTFF's position as: what actually happened was that the Long Term Future Fund approached me and said “we will fund every single good AI-related proposal you get, just hand them to us, you don't have to worry about it” So the challenge is to find high-value funding opportunities in a crowded space. One option would be to trust that the LTFF or whichever organization I pick will do something useful with the money, and I think this is a perfectly valid default choice. However, I suspect that as the major grantmakers are well-funded, I have a specific comparative advantage over them in allocating my funds: I have much more time per unit money to assess, advise, and mentor my grantees. It helps that I have enough of an inside view of what kinds of things might be valuable that I have some hope of noticing gold when I strike it. Additionally, I can approach people who would not normally apply to a fund. What is my grantmaking strategy? First, I decided what parts of the cause to focus on. I'm most interested in supporting alignment infrastructure, because I feel relatively more qualified to judge the effectiveness of interventions to improve the funnel which takes in people who don't know about alignment in one end, takes them through increasing levels of involvement, and (when successful) ends with people who make notable contributions. I'm also excited about funding frugal people to study or do research which seems potentially promising to my inside view. Next, I increased my surface area with places which might have good giving opportunities by involving myself with many parts of the movement. This includes Rob Miles's Discord, AI Safety Support's Slack, in-person communities, EleutherAI, and the LW/EA investing Discord, where there are high concentrations of relevant people, and exploring my non-LW social networks for promising people. I also fund myself to spend most of my time helping out with projects, advising people, and learning about what it takes to build things. Then, I put out feelers towards people who are either already doing valuable work unfunded or appear to have the potential and drive to do so if they were freed of financial constraints. This generally involves getting to know them well enough that I have a decent picture of their skills, motivation structure, and life circumstances. I put some thought into the kind of work I would be most excited to see them do, then discuss this with them and offer them a ~1 year grant (usually $14k-25k, so far) as a trial. I also keep an eye open for larger projects which I might be able to kickstart. When an impact certificate market comes into being (several promising signs on the horizon!), I intend to sell the impact of funding the successful projects and use the proceeds to continue grantmaking for longer. Alongside sharing my models of how to grantmake in this area for others in a similar position and getting advice on it, the secondary purpose of this post is to pre-register my intent to sell impact in order to strengthen the connection between future people buying my impact and my current decisions. ...

ai speech discord alignment individual slack ea ftx donor lw sff acx zvi rationalist lesswrong eleutherai ltff

LW - Being an individual alignment grantmaker by A donor

Yannic Kilcher Videos (Audio Only)

Play Episode Listen Later Mar 1, 2022 3:58

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Being an individual alignment grantmaker, published by A donor on February 28, 2022 on LessWrong. I am an earlyish crypto investor who has accumulated enough to be a mid-sized grantmaker, and I intend to donate most of my money over the next 5-10 years to try and increase the chances that humanity has a wonderful future. My best guess is that this is mostly decided by whether we pass the test of AI alignment, so that's my primary focus. AI alignment has lots of money flowing into it, with some major organizations not running fundraisers, Zvi characterizing SFF as having “too much money”, OpenPhil expanding its grantmaking for the cause, FTX setting themselves up as another major grantmaker, and ACX reporting the LTFF's position as: what actually happened was that the Long Term Future Fund approached me and said “we will fund every single good AI-related proposal you get, just hand them to us, you don't have to worry about it” So the challenge is to find high-value funding opportunities in a crowded space. One option would be to trust that the LTFF or whichever organization I pick will do something useful with the money, and I think this is a perfectly valid default choice. However, I suspect that as the major grantmakers are well-funded, I have a specific comparative advantage over them in allocating my funds: I have much more time per unit money to assess, advise, and mentor my grantees. It helps that I have enough of an inside view of what kinds of things might be valuable that I have some hope of noticing gold when I strike it. Additionally, I can approach people who would not normally apply to a fund. What is my grantmaking strategy? First, I decided what parts of the cause to focus on. I'm most interested in supporting alignment infrastructure, because I feel relatively more qualified to judge the effectiveness of interventions to improve the funnel which takes in people who don't know about alignment in one end, takes them through increasing levels of involvement, and (when successful) ends with people who make notable contributions. I'm also excited about funding frugal people to study or do research which seems potentially promising to my inside view. Next, I increased my surface area with places which might have good giving opportunities by involving myself with many parts of the movement. This includes Rob Miles's Discord, AI Safety Support's Slack, in-person communities, EleutherAI, and the LW/EA investing Discord, where there are high concentrations of relevant people, and exploring my non-LW social networks for promising people. I also fund myself to spend most of my time helping out with projects, advising people, and learning about what it takes to build things. Then, I put out feelers towards people who are either already doing valuable work unfunded or appear to have the potential and drive to do so if they were freed of financial constraints. This generally involves getting to know them well enough that I have a decent picture of their skills, motivation structure, and life circumstances. I put some thought into the kind of work I would be most excited to see them do, then discuss this with them and offer them a ~1 year grant (usually $14k-25k, so far) as a trial. I also keep an eye open for larger projects which I might be able to kickstart. When an impact certificate market comes into being (several promising signs on the horizon!), I intend to sell the impact of funding the successful projects and use the proceeds to continue grantmaking for longer. Alongside sharing my models of how to grantmake in this area for others in a similar position and getting advice on it, the secondary purpose of this post is to pre-register my intent to sell impact in order to strengthen the connection between future people buying my impact and my current decisions. ...

ai speech discord alignment individual slack ea ftx donor lw sff acx zvi rationalist lesswrong eleutherai ltff

GPT-NeoX-20B - Open-Source huge language model by EleutherAI (Interview w/ co-founder Connor Leahy)

Play Episode Listen Later Feb 16, 2022 20:05

#eleuther #gptneo #gptj EleutherAI announces GPT-NeoX-20B, a 20 billion parameter open-source language model, inspired by GPT-3. Connor joins me to discuss the process of training, how the group got their hands on the necessary hardware, what the new model can do, and how anyone can try it out! OUTLINE: 0:00 - Intro 1:00 - Start of interview 2:00 - How did you get all the hardware? 3:50 - What's the scale of this model? 6:00 - A look into the experimental results 11:15 - Why are there GPT-Neo, GPT-J, and GPT-NeoX? 14:15 - How difficult is training these big models? 17:00 - Try out the model on GooseAI 19:00 - Final thoughts Read the announcement: https://blog.eleuther.ai/announcing-20b/ Try out the model: https://goose.ai/ Check out EleutherAI: https://www.eleuther.ai/ Read the code: https://github.com/EleutherAI/gpt-neox Hardware sponsor: https://www.coreweave.com/ Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yann... LinkedIn: https://www.linkedin.com/in/ykilcher BiliBili: https://space.bilibili.com/2017636191 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannick... Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

co founders model language open source gpt outline leahy language model neox connor leahy eleutherai gpt j gpt neo

Auf dem Weg zur Super-KI mit Connor Leahy | DEEP MINDS #4

DEEP MINDS - KI-Podcast

Play Episode Listen Later Feb 4, 2022 101:42

Von genereller Künstlicher Intelligenz bis zur Super-KI: Mit dem KI-Forscher Connor Leahy sprechen wir über die Singularität, Doomsday-Szenarien und die Chancen extrem fortschrittlicher KI. Mit Connor haben wir über Fragen gesprochen wie: Was ist generelle Künstliche Intelligenz, welcher Pfad führt uns zu ihr und womöglich darüber hinaus? Sind Sci-Fi Doomsday-Szenarien eine reale Bedrohung und wie können wir sie verhindern (wenn wir sie verhindern können)? Ist Deep Learning weiter der Goldstandard in der KI-Entwicklung? Wie setzt man Künstliche Intelligenz sinnvolle Ziele? Nimmt die Forschung die Risiken der KI-Entwicklung ernst? Müssen Menschen mit Maschinen verschmelzen, um Schritt zu halten? Den Podcast bei YouTube ansehen: https://www.youtube.com/watch?v=W7iQH9XnfVk :// Kapitel 00:00 Intro & Sponsoren 04:41 Ist GPT-3 ein Schritt zur GKI? 14:15 Hat KI was mit Intelligenz zu tun? 19:45 Haben Menschen etwas, das KI-Systeme nicht haben? 23:15 Warum brauchen wir generelle Künstliche Intelligenz? 25:30 Wie weit sind wir weg von GKI? 30:30 Erreichen wir die Super-KI mit Deep Learning? 35:00 Welche Deep-Learning-Alternativen gibt es? 38:20 Welche neue Hardware könnte KI nach vorne bringen? 42:00 Die Super-KI ist da - was nun? 48:40 Wie können wir eine Super-KI kontrollieren? 54:00 Nimmt die Forschung die Risiken ernst? 01:01:20 Gute Ziele für KI: Warum ist das so schwierig? 01:05:40 Bremsen praktische Risiken den KI-Einsatz aus? 01:14:10 Statt Kontrolle: Können wir KI mit Werten ausstatten? 01:19:50 Sollten Menschen zu Cyborgs werden? 01:23:00 Ist das Alignment-Problem das größte KI-Problem? 01:30:55 Welche Rolle spielt Generelle KI für Deine Arbeit, Connor? 01:32:40 Wie startet man in die KI-Forschung zu GKI? 01:37:45 Wie kann man mit Connor arbeiten? :// Über DEEP MINDS DEEP MINDS ist ein Video-Podcast mit Menschen, die sich mit Künstlicher Intelligenz und Wissenschaft befassen. Ihnen stellen Max und Matthias einfache und schwere Fragen über Technologie, Forschung, Entwicklung und unsere Zukunft. Mehr: https://mixed.de/deep-minds/ :// Über Connor Leahy Connor Leahy arbeitet als KI-Forscher beim deutschen KI-Unternehmen bei Aleph Alpha. Dort beaufsichtigt und beteiligt er sich an Forschungsarbeiten zur Entwicklung allgemeiner KI und zur Frage, wie solche Systeme an menschlichen Werten ausgerichtet werden können. Connor ist außerdem Gründungsmitglied und nomineller Leiter des unabhängigen Forschungskollektivs EleutherAI (eleuther.ai). EleutherAI betreibt offene Forschung zu AGI, Skalierung, Alignment und anderen Themen. Ein Vorzeigeprojekt der Kollektivs ist GPT-Neo (X), ein laufendes Projekt zum Aufbau eines quelloffenen GPT3-ähnlichen Modells. ---------- Danke an unseren Sponsoren: BWI und Borlabs Borlabs Cookie Wordpress-Plugin made in Hamburg. Kauft Borlabs Cookie jetzt auf https://borlabs.io/mixed mit dem Rabattcode MIXED und erhaltet fünf Prozent Rabatt. BWI: Der DEEP MINDS Podcast wird unterstützt von der BWI, dem IT-Systemhaus der Bundeswehr. Als ihr zuverlässiger Partner unterstützt sie mit Innovationen und ihrer IT-Expertise die Digitalisierung der Streitkräfte und treibt diese voran. Auch die Zukunftstechnologie KI spielt dabei eine wichtige Rolle, etwa bei der Generierung von Lagebildern oder für das Server-Management. Aktuelles aus der Arbeit der BWI: https://www.bwi.de/news-blog KI bei der BWI Vom Software-Lebenszyklus bis zur Server-Anomalie: https://www.bwi.de/news-blog/blog/artikel/vom-software-lebenszyklus-bis-zur-server-anomalie-ki-und-ihr-praktischer-nutzen-fuer-die-bwi Wie KI Deutschland vor Angriffen schützen kann: https://www.bwi.de/news-blog/blog/artikel/hybride-bedrohungen-wie-kuenstliche-intelligenz-deutschland-vor-angriffen-schuetzen-kann Die BWI sucht engagierte IT-Profis: https://www.bwi.de/karriere

Connor Leahy on EleutherAI, Replicating GPT-2/GPT-3, AI Risk and Alignment

The Gradient Podcast

Play Episode Listen Later Feb 3, 2022

In episode 22 of The Gradient Podcast, we talk to Connor Leahy, an AI researcher focused on AI alignment and a co-founder of EleutherAI.Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterConnor is an AI researcher working on understanding large ML models and aligning them to human values, and a cofounder of EleutherAI, a decentralized grassroots collective of volunteer researchers, engineers, and developers focused on AI alignment, scaling, and open source AI research. The organization's flagship project is the GPT-Neo family of models designed to match those developed by OpenAI as GPT-3.Sections:(00:00:00) Intro(00:01:20) Start in AI(00:08:00) Being excited about GPT-2 (00:18:00) Discovering AI safety and alignment(00:21:10) Replicating GPT-2 (00:27:30) Deciding whether to relese GPT-2 weights(00:36:15) Life after GPT-2 (00:40:05) GPT-3 and Start of Eleuther AI(00:44:40) Early days of Eleuther AI(00:47:30) Creating the Pile, GPT-Neo, Hacker Culture(00:55:10) Growth of Eleuther AI, Cultivating Community(01:02:22) Why release a large language model(01:08:50) AI Risk and Alignment(01:21:30) Worrying (or not) about Superhuman AI(01:25:20) AI alignment and releasing powerful models(01:32:08) AI risk and research norms(01:37:10) Work on GPT-3 replication, GPT-NeoX(01:38:48) Joining Eleuther AI(01:43:28) Personal interests / hobbies(01:47:20) OutroLinks to things discussed:Replicating GPT2–1.5B , GPT2, Counting Consciousness and the Curious HackerThe Hacker Learns to TrustThe PileGPT-NeoGPT-JWhy Release a Large Language Model?What A Long, Strange Trip It's Been: EleutherAI One Year RetrospectiveGPT-NeoX Get full access to The Gradient at thegradientpub.substack.com/subscribe

ai work growth personal risk alignment deciding openai worrying ml gpt pile 5b sections large language models leahy cultivating community replicating gradient connor leahy hacker culture eleutherai gpt neo

Large Language Models

The Data Exchange with Ben Lorica

Play Episode Listen Later Dec 30, 2021 41:13

This episode features conversations with two experts who have helped train and release models that can recognize, predict, and generate human language on the basis of very large text-based data sets. First is an excerpt of my conversation with Connor Leahy, AI Researcher at Aleph Alpha GmbH, and founding member of EleutherAI, (pronounced “ee-luther”) a collective of researchers and engineers building resources and models for researchers who work on natural language models. Next up is an excerpt from a recent conversation with Yoav Shoham, co-founder of AI21 Labs, creators of the largest language model available to developers. Download a FREE copy of our recent NLP Industry Survey Results: https://gradientflow.com/2021nlpsurvey/Subscribe: Apple • Android • Spotify • Stitcher • Google • AntennaPod • RSS.Detailed show notes can be found on The Data Exchange web site.Subscribe to The Gradient Flow Newsletter.

language large models detailed large language models ai21 labs connor leahy eleutherai

Thoughts on the Alignment Implications of Scaling Language Models by leogao

The Nonlinear Library: Alignment Forum Top Posts

Play Episode Listen Later Dec 4, 2021 27:00

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thoughts on the Alignment Implications of Scaling Language Models, published by leogao on the AI Alignment Forum. [Epistemic status: slightly rambly, mostly personal intuition and opinion that will probably be experimentally proven wrong within a year considering how fast stuff moves in this field] This post is also available on my personal blog. Thanks to Gwern Branwen, Steven Byrnes, Dan Hendrycks, Connor Leahy, Adam Shimi, Kyle and Laria for the insightful discussions and feedback. Background By now, most of you have probably heard about GPT-3 and what it does. There's been a bunch of different opinions on what it means for alignment, and this post is yet another opinion from a slightly different perspective. Some background: I'm a part of EleutherAI, a decentralized research collective (read: glorified discord server - come join us on Discord for ML, alignment, and dank memes). We're best known for our ongoing effort to create a GPT-3-like large language model, and so we have a lot of experience working with transformer models and looking at scaling laws, but we also take alignment very seriously and spend a lot of time thinking about it (see here for an explanation of why we believe releasing a large language model is good for safety). The inspiration for writing this document came out of the realization that there's a lot of tacit knowledge and intuitions about scaling and LMs that's being siloed in our minds that other alignment people might not know about, and so we should try to get that out there. (That being said, the contents of this post are of course only my personal intuitions at this particular moment in time and are definitely not representative of the views of all EleutherAI members.) I also want to lay out some potential topics for future research that might be fruitful. By the way, I did consider that the scaling laws implications might be an infohazard, but I think that ship sailed the moment the GPT-3 paper went live, and since we've already been in a race for parameters for some time (see: Megatron-LM, Turing-NLG, Switch Transformer, PanGu-α/盘古α, HyperCLOVA, Wudao/悟道 2.0, among others), I don't really think this post is causing any non-negligible amount of desire for scaling. Why scaling LMs might lead to Transformative AI Why natural language as a medium First, we need to look at why a perfect LM could in theory be Transformative AI. Language is an extremely good medium for representing complex, abstract concepts compactly and with little noise. Natural language seems like a very efficient medium for this; images, for example, are much less compact and don't have as strong an intrinsic bias towards the types of abstractions we tend to draw in the world. This is not to say that we shouldn't include images at all, though, just that natural language should be the focus. Since text is so flexible and good at being entangled with all sorts of things in the world, to be able to model text perfectly, it seems that you'd have to model all the processes in the world that are causally responsible for the text, to the “resolution” necessary for the model to be totally indistinguishable from the distribution of real text. For more intuition along this line, the excellent post Methods of prompt programming explores, among other ideas closely related to the ideas in this post, a bunch of ways that reality is entangled with the textual universe: A novel may attempt to represent psychological states with arbitrarily fidelity, and scientific publications describe models of reality on all levels of abstraction. [...] A system which predicts the dynamics of language to arbitrary accuracy does require a theory of mind(s) and a theory of the worlds in which the minds are embedded. The dynamics of language do not float free from cultural, psychological, or physical...

language natural speech discord alignment scaling models implications methods ea ml gpt lms lm epistemic rationalist pangu connor leahy eleutherai

[ML News] Facebook AI adapting robots | Baidu autonomous excavators | Happy Birthday EleutherAI

Yannic Kilcher Videos (Audio Only)

Play Episode Listen Later Jul 18, 2021 23:38

A look into the happenings of the Machine Learning world. OUTLINE: 0:00 - Intro 0:25 - Facebook AI trains rapidly adapting robots 3:05 - Baidu presents autonomous excavator system 4:45 - EleutherAI turns 1 6:05 - Elon Musk says FSD harder than expected 8:10 - AI interview tools still fall short 11:10 - RunwayML AI-powered cloud video editor 11:55 - MineRL BASALT competition to learn from human feedback 13:15 - The Myth of the Expert Reviewer 15:55 - NVIDIA unveils Cambridge-1 supercomputer 17:10 - CLIP art sees rapid improvements 19:00 - AI demystifies boiling 21:20 - AI avatars for easier language learning 23:20 - Outro References: Facebook AI trains rapidly adapting robots https://ai.facebook.com/blog/ai-now-e... https://ashish-kmr.github.io/rma-legg... Baidu presents autonomous excavator system http://research.baidu.com/Blog/index-... https://www.youtube.com/watch?v=KFcNf... EleutherAI turns 1 https://blog.eleuther.ai/year-one/ Elon Musk says FSD is harder than expected https://www.theverge.com/2021/7/5/225... AI interview tools still fall short https://www.technologyreview.com/2021... RunwayML AI-powered cloud video editor https://runwayml.com/ MineRL BASALT competition to learn from human feedback https://www.aicrowd.com/challenges/ne... The Myth of the Expert Reviewer https://parameterfree.com/2021/07/06/... NVIDIA unveils Cambridge-1 supercomputer https://www.nvidia.com/en-us/industri... https://nvidianews.nvidia.com/news/nv... CLIP art sees rapid improvements https://ml.berkeley.edu/blog/posts/cl... AI demystifies boiling https://news.mit.edu/2021/infrared-ca... AI avatars for easier language learning https://www.forbes.com/sites/petergre... Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yann... Minds: https://www.minds.com/ykilcher Parler: https://parler.com/profile/YannicKilcher LinkedIn: https://www.linkedin.com/in/yannic-ki... BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannick... Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

ai elon musk blog robots myth happy birthday cambridge minds adapting machine learning nvidia clip outline autonomous baidu fsd bilibili excavators eleutherai

Training and Sharing Large Language Models

The Data Exchange with Ben Lorica

Play Episode Listen Later Jun 24, 2021 50:53

This week's guest is Connor Leahy, AI Researcher at Aleph Alpha GmbH, and founding member of EleutherAI, (pronnounced “ee-luther”) a collective of researchers and engineers building resources and models for researchers who work on natural language models. As NLP research becomes more computationally demanding and data intensive, there is a need for researchers to work together to develop tools and resources for the broader community. While relatively new, EleutherAI has already released a models and data that many researchers are benefitting from.Subscribe: Apple • Android • Spotify • Stitcher • Google • RSS.Detailed show notes can be found on The Data Exchange web site.Subscribe to The Gradient Flow Newsletter.

training sharing language large models detailed large language models connor leahy eleutherai

Someday My ‘Nets Will Code

AI with AI

Play Episode Listen Later Jun 11, 2021 45:01

Information about the AI Event Series mentioned in this episode: https://twitter.com/CNA_org/status/1400808135544213505?s=20 To RSVP contact Larry Lewis at LewisL@cna.org. Andy and Dave discuss the latest in AI news, including a report on Libya from the UN Security Council's Panel of Experts, which notes the March 2020 use of the “fully autonomous” Kargu-2 to engage retreating forces; it's unclear whether any person died in the conflict, and many other important details are missing from the incident. The Biden Administration releases its FY22 DoD Budget, which increases the RDT&E request, including $874M in AI research. NIST proposes an evaluation model for user trust in AI and seeks feedback; the model includes definitions for terms such as reliability and explainability. EleutherAI has provided an open-source version of GPT-3, called GPT-Neo, which uses an 825GB data “Pile” to train, and comes in 1.3B and 2.7B parameter versions. CSET takes a hands-on look at how transformer models such as GPT-3 can aid disinformation, with their findings published in Truth, Lies, and Automation: How Language Models Could Change Disinformation. IBM introduces a project aimed to teach AI to code, with CodeNet, a large dataset containing 500 million lines of code across 55 legacy and active programming languages. In a separate effort, researchers at Berkeley, Chicago, and Cornell publish results on using transformer models as “code generators,” creating a benchmark (the Automated Programming Progress Standard) to measure progress; they find that GPT-Neo could pass approximately 15% of introductory problems, with GPT-3's 175B parameter model performing much worse (presumably due to the inability to fine-tune the larger model). The CNA Russia Studies Program leases an extensive report on AI and Autonomy in Russia, capping off their biweekly newsletters on the topic. Arthur Holland Michel publishes Known Unknowns: Data Issues and Military Autonomous Systems, which clearly identifies the known issues in autonomous systems that cause problems. The short story of the week comes from Asimov in 1956, with “Someday.” And the Naval Institute Press publishes a collection of essays in AI at War: How big data, AI, and machine learning are changing naval warfare. Finally, Diana Gehlhaus from Georgetown's Center for Security and Emerging Technology (CSET), joins Andy and Dave to preview an upcoming event, “Requirements for Leveraging AI.” Interview with Diana Gehlhaus: 33:32 Click here to visit our website and explore the links mentioned in the episode.

chicago ai interview truth war russia lies security code panel ibm berkeley nets requirements cornell biden administration georgetown autonomy gpt libya someday pile 3b 7b nist leveraging ai asimov un security council cna to rsvp naval institute press cset arthur holland michel larry lewis eleutherai gpt neo

#5 - OpenAI GPT-3 Competition (Cohere.AI, Google LaMDA, Google MUM, EleutherAI GPT-J)

Multimodal by Bakz T. Future

Play Episode Listen Later Jun 10, 2021 31:05

Discussing recent GPT-3 language model competition from Cohere, Google, and EleutherAI. What does it mean for the language model and multimodal space? Cohere: https://cohere.ai/ Google: https://blog.google/technology/ai/lamda/ https://blog.google/products/search/introducing-mum/ EleutherAI: https://twitter.com/arankomatsuzaki/status/1402446954550874116 Subscribe to the Multimodal Podcast! Spotify - https://open.spotify.com/show/7qrWSE7ZxFXYe8uoH8NIFV Apple Podcasts - https://podcasts.apple.com/us/podcast/multimodal-by-bakz-t-future/id1564576820 Google Podcasts - https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkLnBvZGJlYW4uY29tL2Jha3p0ZnV0dXJlL2ZlZWQueG1s Stitcher - https://www.stitcher.com/show/multimodal-by-bakz-t-future Other Podcast Apps (RSS Link) - https://feed.podbean.com/bakztfuture/feed.xml Connect with me: YouTube - https://www.youtube.com/bakztfuture Substack Newsletter - https://bakztfuture.substack.com Twitter - https://www.twitter.com/bakztfuture Instagram - https://www.instagram.com/bakztfuture Github - https://www.github.com/bakztfuture

spotify google competition openai gpt github lamda cohere openai gpt eleutherai google lamda gpt j

Clément Delangue, CEO of Hugging Face, on the power of the open source community

Gradient Dissent - A Machine Learning Podcast by W&B

Play Episode Listen Later Jun 10, 2021 46:34

Clem explains the virtuous cycles behind the creation and success of Hugging Face, and shares his thoughts on where NLP is heading. --- Clément Delangue is co-founder and CEO of Hugging Face, the AI community building the future. Hugging Face started as an open source NLP library and has quickly grown into a commercial product used by over 5,000 companies. Connect with Clem: Twitter: https://twitter.com/ClementDelangue LinkedIn: https://www.linkedin.com/in/clementdelangue/ --- Topics Discussed: 0:00 Sneak peek and intro 0:56 What is Hugging Face? 4:15 The success of Hugging Face Transformers 7:53 Open source and virtuous cycles 10:37 Working with both TensorFlow and PyTorch 13:20 The "Write With Transformer" project 14:36 Transfer learning in NLP 16:43 BERT and DistilBERT 22:33 GPT 26:32 The power of the open source community 29:40 Current applications of NLP 35:15 The Turing Test and conversational AI 41:19 Why speech is an upcoming field within NLP 43:44 The human challenges of machine learning Transcript: http://wandb.me/gd-clement-delangue Links Discussed: Write With Transformer, Hugging Face Transformer's text generation demo: https://transformer.huggingface.co/ "Attention Is All You Need" (Vaswani et al., 2017): https://arxiv.org/abs/1706.03762 EleutherAI and GPT-Neo: https://github.com/EleutherAI/gpt-neo] Rasa, open source conversational AI: https://rasa.com/ --- Get our podcast on these platforms: Apple Podcasts: http://wandb.me/apple-podcasts Spotify: http://wandb.me/spotify Google Podcasts: http://wandb.me/google-podcasts YouTube: http://wandb.me/youtube Soundcloud: http://wandb.me/soundcloud Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more: https://wandb.ai/fully-connected

ceo spotify ai open current soundcloud google podcasts nlp ml ment sneak amas rasa deep learning hugging turing test tensorflow open source community eleutherai gpt neo kagglers

Currents 033: Connor Leahy on Deep Learning

The Jim Rutt Show

Play Episode Listen Later Jun 4, 2021 53:25

Connor Leahy has a wide-ranging chat with Jim about the state & future of Deep Learning. They cover the history of EleutherAI, how GPT-3 works, the dynamics & power of scaling laws, ideal sampling rates & sizes for models, data sets, EleutherAI’s opensource GTP-Neo & GTP-NeoX, PyTorch vs TensorFlow, TPU’s vs GPU’s, the challenge of benchmarking & evaluations, quadradic bottlenecks, broad GTP-3 applications, … Continue reading Currents 033: Connor Leahy on Deep Learning → The post Currents 033: Connor Leahy on Deep Learning appeared first on The Jim Rutt Show.

gpu deep learning currents leahy gtp tpu connor leahy eleutherai jim rutt show

2. Connor Leahy on GPT3, EleutherAI and AI Alignment