Pigeon Hour

The gang from Episode 10 is back, with yet another Consistently Candid x Pigeon Hour crossoverAs Sarah from Consistently Candid describes:In this episode, Aaron Bergman and Max Alexander are back to battle it out for the philosophy crown, while I (attempt to) moderate. They discuss the Very Repugnant Conclusion, which, in the words of Claude, "posits that a world with a vast population living lives barely worth living could be considered ethically inferior to a world with an even larger population, where most people have extremely high quality lives, but a significant minority endure extreme suffering." Listen to the end to hear my uninformed opinion on who's right.- Listen to Consistently Candid on Apple Podcasts, Spotify, or via RSS- My blog post on suffering-focused utilitarianism- Follow Max on Twitter and check out his blog- Follow Sarah on Twitter Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

Please follow Arthur on Twitter and check out his blog! Thank you for just summarizing my point in like 1% of the words-Aaron, to Arthur, circa 34:45Summary(Written by Claude Opus aka Clong)* Aaron and Arthur introduce themselves and discuss their motivations for starting the podcast. Arthur jokingly suggests they should "solve gender discourse".* They discuss the benefits and drawbacks of having a public online persona and sharing opinions on Twitter. Arthur explains how his views on engaging online have evolved over time.* Aaron reflects on whether it's good judgment to sometimes tweet things that end up being controversial. They discuss navigating professional considerations when expressing views online.* Arthur questions Aaron's views on cause prioritization in effective altruism (EA). Aaron believes AI is one of the most important causes, while Arthur is more uncertain and pluralistic in his moral philosophy.* They debate whether standard EA global poverty interventions are likely to be the most effective ways to help people from a near-termist perspective. Aaron is skeptical, while Arthur defends GiveWell's recommendations.* Aaron makes the case that even from a near-termist view focused only on currently living humans, preparing for the impacts of AI could be highly impactful, for instance by advocating for a global UBI. Arthur pushes back, arguing that AI is more likely to increase worker productivity than displace labor.* Arthur expresses skepticism of long-termism in EA, though not due to philosophical disagreement with the basic premises. Aaron suggests this is a well-trodden debate not worth rehashing.* They discuss whether old philosophical texts have value or if progress means newer works are strictly better. Arthur mounts a spirited defense of engaging with the history of ideas and reading primary sources to truly grasp nuanced concepts. Aaron contends that intellectual history is valuable but reading primary texts is an inefficient way to learn for all but specialists.* Arthur and Aaron discover a shared passion for rock climbing, swapping stories of how they got into the sport as teenagers. While Aaron focused on indoor gym climbing and competitions, Arthur was drawn to adventurous outdoor trad climbing. They reflect on the mental challenge of rationally managing fear while climbing.* Discussing the role of innate talent vs training, Aaron shares how climbing made him viscerally realize the limits of hard work in overcoming genetic constraints. He and Arthur commiserate about the toxic incentives for competitive climbers to be extremely lean, while acknowledging the objective physics behind it.* They bond over falling out of climbing as priorities shifted in college and lament the difficulty of getting back into it after long breaks. Arthur encourages Aaron to let go of comparisons to his past performance and enjoy the rapid progress of starting over.TranscriptVery imperfect - apologies for the errors.AARONHello, pigeon hour listeners. This is Aaron, as it always is with Arthur Wright of Washington, the broader Washington, DC metro area. Oh, also, we're recording in person, which is very exciting for the second time. I really hope I didn't screw up anything with the audio. Also, we're both being really awkward at the start for some reason, because I haven't gotten into conversation mode yet. So, Arthur, what do you want? Is there anything you want?ARTHURYeah. So Aaron and I have been circling around the idea of recording a podcast for a long time. So there have been periods of time in the past where I've sat down and been like, oh, what would I talk to Aaron about on a podcast? Those now elude me because that was so long ago, and we spontaneously decided to record today. But, yeah, for the. Maybe a small number of people listening to this who I do not personally already know. I am Arthur and currently am doing a master's degree in economics, though I still know nothing about economics, despite being two months from completion, at least how I feel. And I also do, like, housing policy research, but I think have, I don't know, random, eclectic interests in various EA related topics. And, yeah, I don't. I feel like my soft goal for this podcast was to, like, somehow get Aaron cancelled.AARONI'm in the process.ARTHURWe should solve gender discourse.AARONOh, yeah. Is it worth, like, discussing? No, honestly, it's just very online. It's, like, not like there's, like, better, more interesting things.ARTHURI agree. There are more. I was sort of joking. There are more interesting things. Although I do think, like, the general topic that you talked to max a little bit about a while ago, if I remember correctly, of, like, kind of. I don't know to what degree. Like, one's online Persona or, like, being sort of active in public, sharing your opinions is, like, you know, positive or negative for your general.AARONYeah. What do you think?ARTHURYeah, I don't really.AARONWell, your. Your name is on Twitter, and you're like.ARTHURYeah. You're.AARONYou're not, like, an alt.ARTHURYeah, yeah, yeah. Well, I. So, like, I first got on Twitter as an alt account in, like, 2020. I feel like it was during my, like, second to last semester of college. Like, the vaccine didn't exist yet. Things were still very, like, hunkered down in terms of COVID And I feel like I was just, like, out of that isolation. I was like, oh, I'll see what people are talking about on the Internet. And I think a lot of the, like, sort of more kind of topical political culture war, whatever kind of stuff, like, always came back to Twitter, so I was like, okay, I should see what's going on on this Twitter platform. That seems to be where all of the chattering classes are hanging out. And then it just, like, made my life so much worse.AARONWait, why?ARTHURWell, I think part of it was that I just, like, I made this anonymous account because I was like, oh, I don't want to, like, I don't want to, like, have any reservations about, like, you know, who I follow or what I say. I just want to, like, see what's going on and not worry about any kind of, like, personal, like, ramifications. And I think that ended up being a terrible decision because then I just, like, let myself get dragged into, like, the most ultimately, like, banal and unimportant, like, sort of, like, culture war shit as just, like, an observer, like, a frustrated observer. And it was just a huge waste of time. I didn't follow anyone interesting or, like, have any interesting conversations. And then I, like, deleted my Twitter. And then it was in my second semester of my current grad program. We had Caleb Watney from the Institute for Progress come to speak to our fellowship because he was an alumni of the same fellowship. And I was a huge fan of the whole progress studies orientation. And I liked what their think tank was doing as, I don't know, a very different approach to being a policy think tank, I think, than a lot of places. And one of the things that he said for, like, people who are thinking about careers in, like, policy and I think sort of applies to, like, more ea sort of stuff as well, was like, that. Developing a platform on Twitter was, like, opened a lot of doors for him in terms of, like, getting to know people in the policy world. Like, they had already seen his stuff on Twitter, and I got a little bit, like, more open to the idea that there could be something constructive that could come from, like, engaging with one's opinions online. So I was like, okay, fuck it. I'll start a Twitter, and this time, like, I won't be a coward. I won't get dragged into all the worst topics. I'll just, like, put my real name on there and, like, say things that I think. And I don't actually do a lot of that, to be honest.AARONI've, like, thought about gotta ramp it.ARTHUROff doing more of that. But, like, you know, I think when it's not eating too much time into my life in terms of, like, actual deadlines and obligations that I have to meet, it's like, now I've tried to cultivate a, like, more interesting community online where people are actually talking about things that I think matter.AARONNice. Same. Yeah, I concur. Or, like, maybe this is, like, we shouldn't just talk about me, but I'm actually, like, legit curious. Like, do you think I'm an idiot or, like, cuz, like, hmm. I. So this is getting back to the, like, the current, like, salient controversy, which is, like, really just dumb. Not, I mean, controversy for me because, like, not, not like an actual, like, event in the world, but, like, I get so, like, I think it's, like, definitely a trade off where, like, yeah, there's, like, definitely things that, like, I would say if I, like, had an alt. Also, for some reason, I, like, really just don't like the, um, like, the idea of just, like, having different, I don't know, having, like, different, like, selves. Not in, like, a. And not in, like, any, like, sort of actual, like, philosophical way, but, like, uh, yeah, like, like, the idea of, like, having an online Persona or whatever, I mean, obviously it's gonna be different, but, like, in. Only in the same way that, like, um, you know, like, like, you're, like, in some sense, like, different people to the people. Like, you're, you know, really close friend and, like, a not so close friend, but, like, sort of a different of degree. Like, difference of, like, degree, not kind. And so, like, for some reason, like, I just, like, really don't like the idea of, like, I don't know, having, like, a professional self or whatever. Like, I just. Yeah. And you could, like, hmm. I don't know. Do you think I'm an idiot for, like, sometimes tweeting, like, things that, like, evidently, like, are controversial, even if they, like, they're not at all intent or, like, I didn't even, you know, plan, like, plan on them being.ARTHURYeah, I think it's, like, sort of similar to the, like, decoupling conversation we had the other night, which is, like, I totally am sympathetic to your sense of, like, oh, it's just nice to just, like, be a person and not have to, like, as consciously think about, like, dividing yourself into these different buckets of, like, what sort of, you know, Persona you want to, like, present to different audiences. So, like, I think there's something to that. And I, in some ways, I have a similar intuition when it comes to, like, I try to set a relatively strong principle for myself to not lie. And, like, it's not that I'm, like, a Kantian, but I just, like, I think, like, just as a practical matter, the problem with lying for me at least, is then, like, you have to keep these sorts of two books, sets of books in your head of, like, oh, what did I tell to whom? And, like, how do I now say new things that are, like, consistent with the information that I've already, like, you know, falsely or not, like, divulge to this person. Right. And I think, in a similar way, there's something appealing about just, like, being fully honest and open, like, on the Internet with your real name and that you don't have to, like, I don't know, jump through all of those hoops in your mind before, like, deciding whether or not to say something. But at the same time, to the, like, conversation we had the other night about decoupling and stuff, I think. I think there's, like, it is an unfortunate reality that, like, you will be judged and, like, perhaps unfairly on the things that you say on the Internet, like, in a professional sphere. And, like, I don't know, at some level, you can't just, like, wish your.AARONWay out of it. Yeah, no, no, that's, like, a. Okay, so I. This is actually, like, I, like, totally agree. I think, like, one thing is just. I, like, really, honestly, like, don't know how, like, empirically, like, what is the actual relationship between saying, like, say, you get, like, I don't know, like, ten, like, quote tweets, people who are, like, misunderstanding your point, like, and, like, I don't know, say, like, 30 comments or whatever replies or whatever. And, like, it is, like, not at all clear to me, like, what that corresponds to in the real world. And, like, I think I may have erred too much in the direction of, like, oh, that's, like, no evidence at all because, sorry, we should really talk about non twitter stuff. But, like, this is actually, like, on my mind. And this is something like, I didn't. Like, I thought about tweeting, like, but didn't, which is that, like, oh, yeah, I had, like, the building underground tweet, which, like, I think that's a good example. Like, anybody who's, like, reasonably charitable can, like, tell that. It's, like, it was, like, I don't know, it was, like, a reasonable question. Like, and we've mentioned this before, like, this is, like, I don't want to just, like, yeah, it's, like, sort of been beaten to death or whatever, but, like, I feel like maybe, like, I came away from that thinking that, like, okay, if people are mad at you on the Internet, that is, like, no evidence whatsoever about, like, how it, like, how a reasonable person will judge you and or, like, what will happen, like. Like, in real life and, like, yeah, maybe I, like, went too hard in that direction or something.ARTHURYeah, yeah, yeah. I mean, to, like, agree, maybe move on to non twitter, but, yeah, like, to close this loop. I think that, like, I agree that any. Any one instance of individuals being mad at you online, like, it's very easy to, like, over react or extrapolate from that. That, like, oh, people in the real world are gonna, like, judge me negatively because of this. Right. I think in any isolated instance, that's true, but I just. I also get the sense that in the broad world of sort of, like, think tanks and nonprofits and things where, like, your position would. Especially if you're, like, in a research position, like, to some degree, like, representative the opinions of an employer. Right. That there's a kind of, like, character judgment that goes into someone's overall Persona. So, like, the fact that you have, like, one controversial tweet where people are saying, like, oh, you think, you know, like, poor people don't deserve natural light or something like that. Like, that. Any one instance, like, might not matter very much, but if you, like, strongly cultivate a Persona online of, like, being a bit of a loose cannon and, like, oh, I'm gonna say, like, whatever controversial thing comes to mind, I can see any organization that has, like, communicating to a broader audience is, like, an important part of their mission. Like, being hesitant to, like, take a chance on a young person who, like, is prone to, you know, getting into those kinds of controversies on, like, a regular basis.AARONYeah, yeah. And actually, like, maybe this is, like, sort of meta, but, like, I think that is totally correct. Like, you should 100% up. Like, up if you're an employer listening to this. And, like, I don't know. Who knows? There's, like, a non zero chance that, like, I don't know, maybe, like, not low. Lower than, like, 0.1% or something like that. That will be the case. And, like, no, it is totally true that, like, my. I have, like, subpart. Wait, you better. I'm gonna, like. No, no quoting out of context here, please. Or, like, not know, like, clipping the quote out of, like, so it becomes out of context. But, like, it is, like, I have definitely poor judgment about how things will be, um, like, taken, uh, by the people of the Internet, people of the world. I, like, legitimately, I think I'm below not. Probably not first percentile, probably below 50th percentile, at least among broadly western educated, liberal ish people. And so, yes, it's hiring me for head of communication. I mean, there's a reason I'm not. I wouldn't say that I'm not applying to be a communications person anywhere, but I don't know, it's not crazy that I would. If you want to. Yeah, you should. Like, it is, like, correct information. Like, I'm not trying to trick anybody here. Well, okay. Is there anything else that's on your mind? Like, I don't know, salient or, like.ARTHURThat'S what I should have done before I came over here, but nothing, like, on the top of my head, but I feel like there's, I don't know, there's all kinds of, well, like, there's something you've, like, wandered into.AARONYeah, like, I think you have bad cause prioritization takes.ARTHUROh, right.AARONLike, maybe we shouldn't just, like, have the AI versus, like, I don't know, it's like my, like, the AI is a big deal. Tribe is like, yeah, not only winning, but, like, pretty obviously and for obvious reasons. So, like, I don't know, I don't, like, really need to have, like, the, you know, the 70th, like, debate ever about, like, oh, it's like, AI.ARTHURWait, sorry. You mean they're winning for obvious reasons insofar as, like, the victories are apparent or that you think, like, the actual arguments leading to them.AARONOh, yeah.ARTHURBecoming more prominent are obvious.AARONYeah. Setting aside the. In the abstract, what, non, like, empirical or empirical, but, like, only using data, like, pre chat, GPT release, like, setting aside that whole cluster of arguments, there is the fact that, like, I don't know, it seems very, very apparent to, like, the chattering classes of people who care about this stuff that, like, AI is, like, both the overt has expanded tremendously, like, also moved. It seems like the AI is as big of a deal, like, as the Internet is, like, the lower bound and, like, much, much more important than that. Is, like, the upper bound. And so, like, and, like, that's a. That's like, a significant shift, I guess. One thing is just, like, there have a lot been a lot of conversations, like, in EA spaces, and, like, I'm just, like, thinking about the AdK podcast. I feel like I've heard it multiple times, but maybe I'm making that up where it's like, one person is, like, makes the case for, like, I don't know, taking AI or, like, thinking that, like, yeah, AI broadly is, like, the most important altruistic area, right? And then the other person says no, and then they do the same, like, five discussion points back and forth.ARTHURYeah.AARONSo, like, I don't think we should do that.ARTHURSure.AARONThat was a really long winded way of saying that.ARTHURI see. So, so, but, but you're, you're trying to emphasize that, like, the kind of, like, reality of the pace of, you know, improvement in artificial intelligence and the fact that it is going to be, like, an incredibly important technology. Like you said, the lower bound being, like, as important as the Internet, I think, of the upper bound is like, I don't know, something like electricity provided we're not gonna, you know, all die or something. Or maybe more transformational extra. But. But I guess we're trying to say is that, like, the Overton window has, like, shifted so much that, like, everyone kind of agrees this is a really transformative technology. And, like, you know, therefore.AARONWell, I guess I. Sorry, wait, I interrupted. I'm an interrupting person. I'm sorry.ARTHURThat's good. It's a natural part of conversation, so I don't feel bad.AARONContinue.ARTHUROh, oh, no, no. I just. I like, like, yeah, maybe we don't need to rehash the, like, whether or not AI is important, but I'm curious, like, what you think. Yeah, like, what do you think is sort of wrong about my.AARONNo, I was just about to ask that, like, when I interrupted you. I actually don't fully know what you believe. I know we, like, go into different, like, vibe camps or, like, there's another. There's like, a proper noun, vibe camp. This is like a lowercase letters.ARTHURVibe count, vibe sphere.AARONYeah, yeah. And, like. But, like, I don't know, do you have, like, a thesis?ARTHURYeah, see, okay. I don't. I think in many ways, like, maybe just to lay out, like, I think my lack of a thesis is probably the biggest distinction between the two of us when it comes to these kind of cause prioritization things.AARONRight.ARTHURBecause, like, I think I, like, over the years have, as I became more interested in the effect of altruism, have sort of changed my views in many different directions and iterations in terms of, like, my basic moral philosophy and, like, what I think the role of EA is. And I think over time, like, I've generally just become, like, more kind of pluralistic. I know it's a bit of a hand wavy word, but, like, I think I have sufficient uncertainty about, like, my basic moral framework towards the world that, like, this is just a guess. Maybe we'll discover this through conversation. But I think, like, perhaps the biggest disagreement between you and I that, like, leads us in different directions is just that I am, like, much more willing to do some kind of, like, worldview diversification sort of move where like, just, you know, going from, like, a set of assumptions, you know, something like hedonistic utilitarianism and, like, viewing ea as, like, how can I as an individual make the greatest, like, marginal contribution to, like, maximizing this global hedonistic welfare function, right. I think, like, I hold that entire project with a little bit of, like, distance and a little bit of uncertainty. So, like, even if, you know, like, granting the assumptions of that project that spits out, like, okay, AI and animals are, like, the only things that we should care about. I think, like, I'm willing to, like, grant that that might follow from those premises. But I think, like, I hold the premise itself about, like, what the kind of EA project is or what I, as an individual who's, like, interested in these ideas should do with my career at, like, sufficient, you know, distance that I'm, like, willing to kind of, like, entertain other sets of assumptions about, like, what is valuable. And, like, therefore, I'm just, like, far less certain in committing to any particular cause area. I think before, before we get deeper into the weeds about this, just to put like, a sharper point on the, like, more meta point that I'm trying to make is that, like, so I think, like, I don't know if there was this ADK episode from like, a long time ago about solutions to the Fermi paradox. And I know this sounds unrelated, but I'm gonna try.AARONNo, no, that's cool.ARTHURAnd one of the things he talked about was like, you know, basically, like, the Fermi paradox isn't actually a paradox if you, like, understand the ways that, like, essentially, like, when you have uncertainty in, like, a bunch of different point estimates, those uncertainties, like, when combined, should yield like, a probability distribution rather than just like, the headline is often, like, the point estimate of, like, oh, we should expect there to be like, so many aliens, right? But it's like when you have uncertainties on, like, each decision, you know, like, each assumption that you're making in all of the parameters of the equation, right? Like, so I think, like, I guess to apply that a little bit to kind of my, like, sort of moral philosophy is, like, I think, like, the reason why I just am very kind of, like, waffly on my cost prioritization and I'm, like, open to many different things is just that, like, I start from the basic assumption that, like, the, you know, the grounding principle of the EA project, which is like, we should try to do good in the world and we should try to do, like, you know, good in the world in ways that are, like, effective and actually, like, you know, have the consequences that we, we want. Right. That, like, I am very bought into that, like, broad assumption, but I think, like, I have sufficient uncertainty at, like, every chain of reasoning from, like, what does good mean? Like, what, you know, what is the role of, like, me as an individual? Like, what is my comparative advantage? What does it mean to be cause neutral, like, at all of these points of decision? I feel like I just have, like, sufficiently high level of uncertainty that, like, when you get to the end of that chain of reasoning and you arrive at some answer of, like, what you ought to do. Like, I think I hold it sort of very lightly, and I think I have, like, very low credence on any, like, one, you know, conclusion from that chain of research.AARONYeah, yeah. That's what cut you off too much. But, like, but, like, I think there's, like, a very, like, paradigmatic conversation which is like, oh, like, should we be pluralistic? And it's happened seven bazillion times. And so, like, I know I want to claim something different. So. Sorry. I guess there's two separate claims. One is like. And you can tell if, like, I sort of. I was sort of assuming, like, you would disagree with this, but I'm not sure is, um. Yeah, like, even if you just, like, purely restrict, um, you're, like, philosophizing or, like, restrict your ethics just to, like, um, humans who are alive right now and, like, like, basically, like, have the worldview that, like, implies malaria nets. Yeah, um, I, like, think it's, like, very unlikely that, like, actually, like, the best guess intervention right now, like, is the set of, like, standard yay interventions or whatever. And, like, another, like, very related, but, like, some, I guess, distinct claim is, like, I don't know exactly. I don't. Yeah, I really don't know at all what this would look like. But, like, it seems very plausible to me that even under that worldview, so not a long term is worldview at all, like, probably doing something related to, like, artificial intelligence. Like, is, like, checks out under. Yeah, under, like, the most, like, norm, like, normal person version, like, restricted version of EA. And, like, I don't know.ARTHURI think I. Yeah, so I think I am inclined to agree with the first part and disagree with the second part. And that's why I want you to spell this out for me, because I. I actually am sympathetic to the idea that, like, under sort of near termist restricting our class of individuals that we want to help to human beings, like, who are alive today. Right. Under that set of assumptions, I similarly think that there's, like, relatively low likelihood that, like, the standard list of sort of, like, give well, interventions are the best. Right.AARONWell, not.ARTHUROr.AARONYeah, yeah, or, like, I'm telling you, like, yeah, if you think. Sorry. Um, yeah, my claim was, like, stronger than, like, that. That. Or, like, what one would interpret that as, like, if you just, like, take it like, super literally. So, like, I think that, like, um, not only expose, like, they're not even our, like, real best guesses, like, like, an actual effort would, like, yield other best guesses. Not only like, oh, yeah. Like, this is our, like, this is like a minority, but like, a plurality of the distribution, if that makes sense.ARTHUROkay, then. Then I do think we disagree because I think where I was going to go from that is that I think to me, like, I'm not as informed on these arguments as I should be. So, like, I will fully admit, like, huge degree of, like, epistemic limitation here, but, like, I think my response was just going to be that I think, like, the case for AI would be sort of even weaker than those givewell style interventions. So even though they're, like, unlikely to be, you know, the best, like, you know, like x post in some, like, future where we, like, have more information about other kinds of ways that we could be helping people. Right. They're like, still, you know, better than the existing alternatives and.AARONYeah, yeah, I'm gonna.ARTHURSo what is the case for, like, near termist case for AI? Like, what if you could.AARONYeah, yeah. Just to, sorry. I like, promise I will answer that. But like, just to clarify. Yeah, so I'm like, more confident about, like, the give world charities are, like, not the ex ante best guess than I am that the better, like one of the best. Like, in fact, ways to help only humans alive right now would involve AI. So, like, these are related, but like, distinctive and the AI one I'm like, much less confident in and haven't, I guess, in some sense, just because it's so much more specific.ARTHURActually, let's do both parts because I realized earlier also what I meant was not ex ante, but ex post. Like, with much larger amount of information about other potential interventions, we might determine that something is better than Givewell. Right. But nonetheless, in the world that we actually live in, with the information that we currently have, the evidence is sufficiently strong for impact under the kinds of assumptions we're saying we're operating under. Right. That, like, you know, other, other competing interventions, like, have a very high bar to click. Like, maybe they're worthwhile in, like, a hit space giving kind of way. Like, in that, like, it's worth, like, trying a bunch of them to, like, see if one of them would outperform givewell. But, like, for the time being, you know, that whatever givewell spreadsheet says at any current time, I think is pretty, like, is pretty compelling in terms of, like, you know, higher certainty ways to help individuals.AARONYeah. So, um.ARTHURSo, so one, I want to hear, like, why you disagree with that. And then two, I want to hear, like, your case for, like, AI.AARONYeah, okay. I think I'm responding to this. Like, you can cut me off or whatever. Um, so, like, fundamentally, I want to, like, decouple. Haha. Or. Yeah, this is something I like doing and decouple, um, the, like, uh, yeah, who we care about. And, like, um, how, like, how aesthetically normal are we gonna be? So, like, I want to say, like, okay, even, yeah, if you. If you're, like, still in the realm of, like, doing analytic philosophy about the issue. And, like, you just, like, say, like, okay, we're just, like, gonna restrict, like, who we care about to, like, humans alive right now. There's, like, still a lot of weird shit that can, like, come out of that. And so, like, my claim, I think actually, like, what's what. Maybe this is, like, somewhat of a hot take, whatever. But I think, like, actually what's happening is, like, there is, like, a, quote, unquote like, worldview that, like, vibe associates and to some extent, like, explicitly endorses, like, only just like, for whatever reason, like, trying to help humans who are alive right now, or, like, maybe, like, who will become alive in the near future or something. But, like, this is always paired with, like, a default, like, often non explicit assumption that, like, we have to do things that look normal. Or, like. And to some extent you can. Some extent you can, like, formalize this by just, like, saying you, like, care about certain deep impact. I think there's, like, not even that technical, but, like, mildly, like, technical reasons why. Like, if you're still in the realm of, like, doing analytical philosophy about the issue, like, that doesn't check out, like, for example, you don't actually know, like, which specific person you're gonna help. I'm, like, a big fan of, like, the recent reaping priorities report. So I spent, like, five minutes, like, rambling and, like, doing a terrible job of explaining what I. What I mean. And so the idea that I'm getting at is that I think there's like a natural, like, tendency to think of risk aversion in like, an EA or just like generally, like, altruistic context. That basically means, like, we like, understand like, a chain of causality. And there are like, professional economists, like, doing RCT's and they like, know what works and what doesn't. And, like, this isn't, like, there's like, something there that is valuable. Like, doing good is hard. And so, like, you know, careful analysis is actually really important. But I think this, like, doesn't, there's a tendency to, like, ignore the fact that, like, these type of, like, give well style, like charities and give well, style, like, analysis to identify the top charities. Basically to, as far as I know, almost exclusively, like, looks at just one of, like one of like, the, the most salient or like, intended, like, basically first order effects of an intervention. So we, like, it's just not true that we know what the impact of like, giving $3,000 to the gens malaria foundation is. And, like, it's like, you know, maybe there are, like, compelling, compelling reasons to, like, think that basically it all washes out or whatever. And, like, in fact, like, you know, reducing deaths from malaria and sickness is like the absolute, like the single core effect. But, like, as far as I know, there's, like, not, that seems to be mostly just like taken as a given. And I don't think this is justified. And so I don't think this, like, really checks out as like, a type of risk aversion that stands up to scrutiny. And I found this tweet. Basically, I think this is like, good wording. The way to formalize this conception is just have narrow confidence intervals on the magnitude of one first order effect of an intervention. And that's an awfully specific type of risk aversion. This is not generally what people mean in all walks of life. And then I mentioned this rethink priorities report written by Laura Duffy first, Pigeonhauer Guest. And she basically lists three different types of risk aversion that she does in some rating priorities, like analysis. So, yeah, number one, avoiding the worst. Basically, this is the s risk style or modality of thinking. The risk. The thing we really, really want to avoid is the worst states of the world happening to me. And I think to many people, that means a lot suffering. And then number two, difference making risk aversion. Basically, we want to avoid not doing anything or causing harm. But this focus is on, like, not on the state of the world that results from some action, like, but like your causal effect. And then finally, number three, ambiguity aversion. Basically, we don't like uncertain probabilities. And for what it's worth, I think, like, yeah, the givewell style, like, leaning, I think, can be sort of understood as an attempt to get to, like, addressed, like, two and three difference making an ambiguity aversion. But like, yeah, for reasons that, like, are not immediately, like, coming to my head and like, verbalize, I, like, don't think. Yeah, basically for the reasons I said before that, like, there's really no comprehensive analysis there. Like, might seem like there is. And like, we do have like, decent point estimates and like, uncertainty ranges for like, one effect. One. But like, I. That doesn't, as far as I can tell, like, that is not like, the core. The core desire isn't just to have like, one narrow, like, nobody. I don't think anyone thinks that we, like, should intrinsically value, like, small confidence intervals. You know what I mean? And this stands in contrast to, as I said before, also s risk of french organizations, which are also, in a very real sense, doing risk aversion. In fact, they use the term risk a lot. So it makes sense. The givewell vibe and the s risk research organization vibes are very different, but they, in a real sense that they're at least both attempting to address some kind of risk aversion, although these kinds are very different. And I think the asterisk one is the most legitimate, honestly. Yeah. Okay, so that is a. There was sort of like a lemma or whatever and then. Yeah. So, like, the case for AI in like, near term only affecting humans. Yes. So, like, here's one example. Like, this is not the actual, like, full claim that I have, but like, one example of like, a type of intervention is like, seeing if you can make it basically like, what institutions need to be in place for, like, world UBI. And let's actually try to get that policy. Let's set up the infrastructure to get that in place. Like, even now, even if you don't care about, like, you think long termism is false, like, don't care about animals, don't care about future people at all, it seems like there is work we can do now, like, within, you know, in like, the realm of like, writing PDF's and like, building. Yeah, building like, like, political institutions or like, at least. Sorry, not building institutions, but like, affecting political institutions. Like via. Like, via, like, I guess like both like, domestic and like, international politics or whatever that, like, still. And sorry, I like, kind of lost like, the grammatical structure of that sentence, but it seems plausible that, like, this actually is like better than the givewell interventions, just like if you actually do like an earnest, like best guess, like point estimate. But the reason that I think this is plausible is that all the people who are willing to do that kind of analysis are like, aren't, aren't restricting themselves to like, only helping humans in like the near future. They're like, I don't know. So there's like a weird, like missing middle of sorts, which, depending on what the counterfactual is, maybe bad or good. But I'm claiming that it exists and there's at least a plausible gap that hasn't really been ruled out in any explicit sense.ARTHUROkay, yeah, great. No, no, that's all very useful. So I think, I guess setting x risky things aside, because I think this is a usual way to get at the crux of our disagreement. Like, it's funny, on the one hand, I'm very sympathetic to your claim that sort of like the kinds of things that give, well, sort of interventions and, you know, RCT's coming out of like development economics are interested in, like, I'm sympathetic to the idea that that's not implied by the kind of like basic near termist EA, philosophical presupposition.AARONThank you for just summarizing my point in like 1% of the words.ARTHURYeah, yeah. So I'm like, I actually strongly agree with that. And it's precisely why, like, I'm more open to things that aren't like givewell style interventions, why I'm very sympathetic to the economic growth side of the growth versus RCT perennial debate, all that. That's maybe interesting side discussion. But to stay on the AI point, I guess putting existential risk aside, I want to make the standard economist argument for AI optimism and against what you were just trying to say. So like, to me, like, I think it is like plausible enough that we should be concerned that, like, increasing AI progress and dissemination of AI technologies decreases returns to labor in the global economy. I think it's plausible enough that we should care about that and not dismiss it out of hand. But, like, I think it's far less likely that, like, or sorry, not. Well, I want to be careful with. I think it's potentially more likely that almost exactly the opposite is true. So, like, if I look at like the big picture history of like global economic growth, like the classic, you know, hockey stick graph where like GDP per capita for the world is like totally flat until, you know, like about 200 years ago. Right? Like, I think the standard, like this is a super interesting rich topic that I've been like learning a lot more about over the last few years. And I think, like the devil is very much in the details. But nonetheless, I think the kind of like classic, you know, postcard length summary is basically correct that like, why did that happen? That happened because like productivity of individual workers, like dramatically increased, like orders of magnitude due to technological progress, right? And like, whether that to what degree that technological progress is sort of like political institutional technologies versus like direct, like labor augmenting technologies is like, you know, whatever, way too deep to get into in this discussion. I don't have like good informed takes on that. But like, nonetheless, I think that, like, the basic like, sort of lump of labor fallacy, like, is strongly at play at these worries that AI is going to displace workers. Like, I think if like, you look at all these, you know, previous technologies, like the, you know, Luddites destroying the power looms or they weren't really power looms, but they were like this more like, you know, better kinds of handlers or whatever. Right, right. Like, I think the worry that people have always had, and again, I get, I'm giving the standard economists soapbox thing that everyone has heard before, but like, I just don't see why AI is categorically different from these other technological advancements. And that, like, at a glance, like, for me as an individual, like trying to build a research career and like get a job and stuff, my ability to access GPT four and Claude, like has I think, like dramatically increased my marginal productivity and like would presumably also increase my wage in the long term because I can just do a lot more in the same amount of time. So, like, it seems to me like just as if not more likely that the better AI technology gets, you have people that are able to produce more in economic value with the same amount of labor and therefore are going to increase economic growth and increase their wages rather than just somehow displace them out of the labor market. And I think there is something that I think EA should maybe paying more attention to, but like maybe they're too concerned with existential risk. There is some interesting experimental economics research already, like looking at this question, which is like having people who work in kind of like standard sort of like, you know, operations and middle management sort of office jobs, like using AI in their work. And I think one of the interesting findings seems to be like a lot of these experiments are finding that it has sort of an equalizing effect, which is like for the most productive employees at a given task, their productivity is like only very modestly improved by having access to large language models. But like, the least productive employees see like very large improvement in their productivity from these technologies. So, like, in my opinion, it seems plausible that like, you know, better access to these sorts of technologies would, if anything, make your, like, standard, you know, employee in the global economy, like, you know, not only more productive, but have this sort of like leveling of the playing field effect. Right. Where like people who, who do not have the current capacities to like produce a lot of value are sort of, you know, brought up to the same level as like.AARONYeah. So, like, I think these are all reasonable points. I also think, um, sorry, I think I have like three, like points, I guess. Yeah. On the object level, I like, don't think I have anything to like, add to this discussion. The one thing I would point out is that it seems like there's, as far as I can tell, like no disagreement that like in principle you can imagine a system that is better than all humans at all tasks that does not have the effect you're talking about in principle, better than humans, better than, better and cheaper than all humans at all tasks.ARTHURRight. With no human input required.AARONYeah, in principle, yeah.ARTHUROkay.AARONYeah, yeah. Like, I don't think this is a radical claim. So like, then there's like the now moving away from the object level. Like, okay, so we've like set this now. Like the normal, like default thing is to like have an debate where, oh, you make some more points in the direction you just said. And I said makes more points. I just said. But like, the thing I want to point out is that like this discussion is like absent from near termist EA because all the people who are taking ideas seriously have already moved on to other areas. And there was one more, but just.ARTHURTo jump on that for a second. But I think I totally take your point that then maybe a lot more people should be thinking about this. Right. But to me, like, whether that's possible in principle, like, like, and I think you're obviously going to agree with me on this. Like, to what degree that's relevant depends on like whether we are living in a world where like those systems are on the horizon or are going to exist in the near term future. Right. And like, to what degree that, you know, imprincible possibility, like, represents the actual path we're heading on is like sort of the real crux of the issue.AARONOh, yeah. Okay. Maybe I actually wasn't sure. Yes, because we're living in a more.ARTHURStandard story where like this just increases the marginal product of labor because everyone gets more productive when they, like, learn how to use these technologies, and it doesn't mean it's not going to be disruptive, because I think there's a lot of interesting IO research on how, with the implementation of computer technologies in a lot of workplaces, it was very difficult to train older employees to use the new systems. So really, the only solution for a lot of firms was essentially just, like, fire all of their old employees and hire people who actually knew how to use these technologies. But presuming we get past the disruptive transition where the old people get screwed or have to learn how to adapt, and then the young people who grew up learning how to use AI technologies enter the workforce, it seems very possible to me that those people are just going to be the most productive generation of workers ever. Accordingly.AARONYeah. Yeah. Again, I think there's, like, sorry, I, like, don't. I guess I was about to just, like, make this. It make the same point that I. That I was before. I guess, like, put a little bit more, like, yeah, be a little bit clearer about, like, what I mean by, like, this debate isn't happening. It is, like, it doesn't seem. Maybe I'm wrong, but, like, like, reasonably confident that, um, givewell isn't doing the thing that, like, the long term esteem on open philanthropy is where they're like, try to answer this question because it's really fucking important and really informs and really informs what kind of, like, what the best near term interventions are. And, like, maybe that's, like, I don't want to pick on Givewell because, like, maybe it's in, you know, givewell is, like, maybe it's, like, in their charter or, like, in some sense, just like, everybody assumes that, like, yeah, they're going to do, like, the econ RCT stuff or whatever, but, like, well, but there'd be value.ARTHURThat, like, that would be my defensive give. Well, like, is that, like, you know, you, like, comparative advantage is real, and, like, you know, having an organization that's like, we're just not gonna worry about these. Like, they don't even do animal stuff, you know? And I think that's a good decision. Like, I care a lot about animal stuff, but I'm glad that there's an organization that's, like, defined their mission narrowly enough such that they're like, we are going to, like, do the best sort of econ development rct kind of stuff. And if you're, like, into this project, like, we're gonna tell you the best way to use your.AARONYeah, I think that like, I don't know, in the abstract. Like, I think. I guess I'm, like, pretty, pretty 50 50 on, like, whether I think it's good. I don't think they should, like, if anybody's deciding, like, whether to, like, give a dollar to give well or, like, not give well with Nea, I think, like, yeah, it's like, don't give a dollar to give well. Like, I don't think they should get any funding, EA funding or whatever. And I can defend that, but, like, so, yeah, maybe that particular organization, but I. Insofar as we're willing to treat, like, near term sea as, like, an institution, like, my stronger claim is, like, it's not happening anywhere.ARTHURYeah, well, I mean, I, like, you're right. At one level, I think I more or less agree with you that it should be happening within that institution. But I think what, at least to me, like, your broad sketch of this sort of near termist case for AI, like, where that discussion and debate is really happening, is in, like, labor economics. You know what I mean? Like, it's not that aren't people interested in this. I just think the people who are interested in this, like, and I don't think this is a coincidence are the people that, like, don't think, you know, the paperclip bots are going to kill us. All, right? They're like, the people who are just, like, have a much more, like, normie set of priors about, like, what this technology is going to look like.AARONYeah, I do.ARTHURAnd, like, they're the ones who are, like, having the debate about, like, what is the impact of AI going to be on the workforce, on inequality, on, you know, global economic growth, like, and I think, like, but, like, I guess in a funny way, it seems like what you're advocating for is, like, actually a much more, like, Normie research project. Like, where you just have, like, a bunch of economists, like, being funded by open philanthropy or something to, like, answer these questions.AARONI think the answer is, like, sort of, um. Does some extent. Yeah, actually, I think, like, I. Like, I don't know. I'm, like, not. Yeah, I actually just, like, don't know. Like, I don't, like, follow econ, like, as a discipline. Like, enough to, like, I, like, believe you or whatever. And, like, obviously it's, like, it's, like, pretty clearly, like, both. I guess I've seen, like, examples I've thrown around of, like, papers or whatever. Yeah, there's, like, clearly, like, some, like, empirical research. I, like, don't know how much research is, like, dedicated to the question, like, yeah, I guess there's a question that's like, if you know, like, yeah. Is anybody, like, trying to. With, like, reasonable. With, like, reasonable parameters, estimate the share of, like, how, like, late the share or the returns to, like, labor or whatever will, like, change in, like, the next, like, ten years or five? Not. Not only. Not only, like, with GPT-3 or, like, not. Not assuming that, like, GPT four is going to be, like, the status quo.ARTHURYeah, I mean, to my knowledge, like, I have no idea. Like, basically I don't have an. All the stuff that I'm thinking of is from, like, you know, shout out Eric Brynjolfsson. Everyone should follow him on Twitter. But, like, like, there's some economists who are in the kind of, like, I o and, like, labor econ space that are doing, like, much more, like, micro level stuff about, like, existing LLM technologies. Like, what are their effects on, sort of, like, the, like, you know, I don't know, knowledge work for lack of a better word, like, workforce, but that, yeah, I grant that, like, that is a very much more, like, narrow and tangible project than, like, trying to have some kind of macroeconomic model that, like, makes certain assumptions about, like, the future of artificial.AARONYeah, and, like, which maybe someone is doing.ARTHURAnd, I mean, I.AARONNo, yeah, I'm interested. People should comment slash dm me on Twitter or whatever. Like, yeah, I mean, I think we're just, like, in agreement that. I mean, I mean, like, I think I have some, like, pretty standard concerns about, like, academic, like, academia, incentives, which are, like, also been, like, rehashed everywhere. But, like, I mean, it's an empirical question that we, like, both just, like, agree is an empirical question that we don't know the answer to. Like, I would be pretty surprised if, like, labor economics has, like, a lot to say about. About fundamentally non empirical questions because, like, it doesn't. Yeah, I guess, like, the claim I'm making is, like, that class of research where you, like, look at, like, yeah. Like, how does chat GPG, like, affect the productivity of workers in 2023? 2024? Really? Just like, I mean, it's not zero evidence, but it's really not very strong evidence for, like, what the share of labor income will be in, like, five to ten years. Like, yeah, and it's, like, relevant. I think it's, like, relevant. The people who are actually building this technology think it's going to be, like, true, at least as far as I can tell. Broadly, it is a consensus opinion among people working on building frontier AI systems that it is going to be more transformative or substantially more transformative than the Internet, probably beyond electricity as well. And if you take that assumption, like, premise on that assumption, it seems like the current. I would be very surprised if there's much academic, like, labor economics that, like, really has a lot to say about, like, what the world would be, like in five to ten years.ARTHURYeah, I think I was just gonna say that I'm, like, sufficiently skeptical that people, like, working on these technologies directly are, like, well positioned to, like, make those kinds of. I'm not saying the labor econ people are, like, better positioned than them to make those progress, but, like, I think.AARONNo, that's totally fair. Yeah, that is really to be fair.ARTHURAlso that, like. Like, I think some of this is coming from, like, coming from a prior that I, like, definitely should, like, you know, completely change with, like, the recent, you know, post GPT-3 like, explosion these technologies. But I just think, like, for, like, just if you look at the history, like, I'm not. I'm not saying I endorse this, but, like, if you look at the history of, like, you know, sort of AI, like, not like optimism per se, but, like, enthusiasm about, like, the pace of progress and all this, like, historically, like, it had a, like, many, many decade track record of, like, promising a lot and failing that, like, was only, like, very recently falsified by, like, GPT-3 and.AARONI mean, like, I think this is basically just, like, wrong. It's like, a common misconception. Not like you're. I think this is, like, totally reasonable. This is, like, what I would have. Like, it seems like the kind of thing that happened. I'm pretty sure, like, there have been some, like, actually, like, looking back analyses, but it's, like, not wouldn't. It's not like there's zero instances, but, like, there's been a real qu. It is not, like, the same level of AI enthusiasm, like, as persistent forever. And, like, now we're like, um. Yeah, now it seems like. Oh. Like we're getting some, like, you know, results that, like, that, like, maybe justify. It seems like, um. Yeah, the consent, like, people are way. Hmm. What am I. Sorry. The actual thing that I'm trying to say here is I basically think this is just not true.ARTHURMeaning, like, the consensus was like, that.AARONLike, people didn't think Agi was ten years away in 1970 or 1990.ARTHURWell, I mean, some people did. Come on.AARONYeah. So I can't.ARTHURYou mean, just like, the consensus of the field as a whole was not as I like.AARONSo all I like, I have, this is, like, this is the problem with, like, arguing a cast opinion. Like, my cashed opinion is, like, I've seen good, convincing evidence that, like, the very common sense thing, which is like, oh, but AI, there's always been AI hype is, like, at least misleading and, like, more or less, like, wrong and that, like, there's been a, like, a, yeah, and like, I don't actually remember the object level evidence for this. So, like, I can try to, like, yeah, that's fine.ARTHURAnd I also like to be clear, like, I don't have a strong, like, strongly informed take, like, for the, like, AI. Yeah, hype is overblown thing. But, like, putting that aside, I think the other thing that I would wonder is, like, even if individuals, like, who work on these technologies, like, correctly have certain predictions about the future that are pretty outside the window or that people aren't sufficiently taking seriously in terms of what they think progress is going to be. And maybe this is some lingering, more credentialist intuitions or whatever, but I think that. I am skeptical that those people would also be in a good position to make kinds of economic forecasts about what the impacts of those technologies.AARONYeah, I basically agree. I like, yeah, it's, I guess, like, the weak claim I want to make is, like, you don't have to have that high a percentage on, like, oh, maybe there's, like, some, like, maybe these people are broadly right. You don't have to think it's above 50% to, like, think that. Like, I think the original claim I was, like, making is, like, um, is like, why probably, like, standard, like, labor economics, like, as a subfield. Like, isn't really doing a ton to, like, answer the core questions that would inform my, like, original thing of, like, oh, like, is ubi, like, like, a better use of money than, like, I give you against malaria foundation or whatever? Um, I, like, yeah, I just, like, don't. Yeah, maybe I'll be, like, pleasantly surprised. But, like, yeah, we could, we could also, I don't know. Do you want to move on to.ARTHURA. Yeah, yeah, sure.AARONSorry. So I didn't mean to. You can have the last word on.ARTHURNo, no, I don't. I don't think I have the last word. I mean, I think it's funny, like, just how this has progressed in that, like, I think, like, I, I don't completely, like, I think, I don't completely disagree, but I also don't feel like my, like, mind has been, like, changed in a big way, if that makes sense. It's just like, maybe we're in one of these weird situations where, like, we kind of, like, do broadly agree on the, like, actual object level, like, questions or whatever, but then there's just some, like, slight difference in, like, almost, like, personality or disposition or, like, some background beliefs that we, like, haven't fully fleshed out that, like, that, like, at least in terms of how we, like, present and emphasize our positions. Like, we end up still being in different places, even if we're not actually that.AARONNo, something I was thinking about bringing up earlier was, like, oh, no. Yeah, basically this point. And then, like. But, like, my. My version of, like, your defensive of the, like, I guess, like, the give. Well, class is, like, my defense of donating to, like, the humane league or whatever, and, like, maybe it doesn't check. And, like, I don't know. I just. Yeah, it's for whatever reason, like, I. I, like, yes, something. I'm still, um. I guess I still don't. Sorry. I just did, like, a bunch of, like, episodic, like, jumps in my head, and I, like, I always forget, like, oh, they can't see my thought patterns in the podcast. Yeah, it seems, like, pretty possible that a formal analysis would say that even under a suffering focused worldview, yet donating to s risk prevention organizations, beats, for example, are at least beats like the Humane League or the Animal Welfare Fund, which we recently raised funds for.ARTHURDo you want to talk? So, there's many things we could talk about. One potential thing that comes to mind is, like, I have a not very well worked out, but just, like, sort of lingering skepticism of long termism in general, which, like, I think doesn't actually come from any, like, philosophical objection to long termist premises. So, like, I think the.AARONYeah, I think.ARTHURI don't know what you want to talk about.AARONI mean, if you really want. If you're, like, really enthusiastic about it.ARTHURI'm not.AARONHonestly, I feel like this has been beaten to death in, like, on 80k. There's cold takes. Like, there have been a. Sorry. I feel like we're not gonna add anything. Like, I'm not gonna add anything either.ARTHUROkay. I don't feel like I would.AARONI mean, we can come. Another thing is, like, yeah, this doesn't have to be super intellectual. Talk about climbing. We're talking about having a whole episode on climbing, so, like, maybe we should do that. Like, also anything. Like, I don't know. It doesn't have to be, like, these super, like, totally.ARTHURNo, no. That was something that came to mind, too, and I was like, oh, the long term isn't thing, but like, it would be fun to just like, talk about something that's like, much less related to any of these topics. And in some ways, given both of our limitations in terms of contributing to these object level EA things, that's not a criticism of either of us, but just in terms of our knowledge and expertise, it could be fun to talk about something more personal.AARONYeah, I need to forget that it's. Yeah. I don't know what is interesting to you.ARTHURI'm trying to think if we should talk about some other area of disagreement because I feel like, like, I'm like, this is, this is random and maybe we'll cut this from the podcast. This is a weird thing to say, but I feel like Laura Duffy is one of the few people that I've, like, met where we just have like a weird amount of the same opinions on like, many different topics that wouldn't seem to like, correlate with one another, like, whatsoever. And it's funny, like, I remember ages ago, like, listening to y'all's discussion on this podcast and just being like, God, Laura is so right. What the fuck does Aaron believe about all these things?AARONAnd I'm willing to relitigate some if it. If it's like something that hasn't been beaten to death elsewhere.ARTHURSo I think we should either talk about like, something more personal, like we should talk about like rock climbing or something, or we should, like, now I.AARONHave to defend myself. You can't just say, you know, yeah. Was it the, like, oh, old philosophy is bad.ARTHUROld philosophy.AARONOld philosophy is fucking terrible. And I'm guessing you don't like this take.ARTHURI do not. Well, I find this take entertaining and, like, I find this take, like, actually, like, I mean, this totally, like, this sounds like a huge backhand compliment, but, like, I actually think it's, like, super useful to hear something that you just, like, think is, like, so deeply wrong, but then you, like, take for granted when you, like, surround yourself with people who, like, would also think it's so deeply wrong. So I think it's, like, actually, like, very useful and interesting for me to, like, understand why one would hold this opinion.AARONAlso, I should. I guess I should clarify. So, like, I. It's like, this is like, the kind of thing that, like, oh, it's like, kind of is like, in my vibe disposition or whatever. Yeah. And, like, also, it is not like, the most high stakes thing in the world, like, talking in the abstract. So, like, when I said, like, oh, there, it's fucking terrible. I was, like, I was, like, being hyperbolic.ARTHUROh, I know.AARONI know. No, but, like, in all serious, like, not in all seriousness, but just, like, like, without being, I don't know, using any figurative language at all, I'm, like, not over. There are definitely things that I'm, like, much more confident about than this. So, like, I wouldn't say I'm, like, 90. Oh, it's a. Me too.ARTHURI'm, like, pretty open to being wrong on this. Like, I don't think I have, like, a deep personal vested stake.AARONYeah, no, I don't think.ARTHURIt's just, I think. Okay, so this is something. Or actually maybe. Maybe an interesting topic that we are by no means experts on but could be interesting to get into is, I think, like, a lot of the debates about, like, what is, like, the role of, like, kind of higher education in general or, like, somewhat hard to separate from these questions of, like, oh, yeah, old text. Because I'm sort of have two minds of this, which is, like, on the one hand, I think I buy a lot of the, like, criticisms of, like, the higher ed, like, sort of model. And that I think, like, this general story which is not novel to me in any way, shape or form, that, like, we have this weird system where, like, universities used to be a kind of, like, the american university system, like, you know, comes in a lot of ways from, like, the british university system, which. Which, if you look at it historically, is sort of like a finishing school for elites, right? Like, you have this, like, elite class of society, and you kind of, like, go to these institutions because you have a certain social position where you, like, learn how to, like, be this, like, educated, erudite, like, member of the, like, elite class in your society. And there's, like, no pretense that it's any kind of, like, practical, you know, skills based education that'll help prepare you for the labor force. It's just like, you're just, like, learning how to be, like, a good, you know, a good, like, member of the upper class, essentially. Right? And then that model was, like, very, like, successful and, like, I think, in many ways, like, actually important to, like, the development of, like, lots of institutions and ideas that, like, matter today. So it's not like, it's not like, you know, it's, like, worth, like, taking seriously, I suppose. But, like, I think there's some truth to, like, why the hell is this now how we, like, certify and credential, like, in a more kind of, like, merit, like, meritocratic sort of, like, world with more social mobility and stuff. Like, why is this sort of, like, liberal arts model of, like, you go to, like, learn how to be this, like, erudite person that, like, knows about the world and, like, the great texts of the western tradition or whatever. Like, I think there's something to the, like, this whole thing is weird. And, like, if what college is now supposed to do is, like, to train one to be, like, a skilled worker in the labor force, like, we ought to seriously rethink this. But at the same time, I think I do have some, like, emotional attachment to, like, the, like, more flowery

IntroAround New Years, Max Alexander, Laura Duffy, Matt and I tried to raise money for animal welfare (more specifically, the EA Animal Welfare Fund) on Twitter. We put out a list of incentives (see the pink image below), one of which was to record a drunk podcast episode if the greater Very Online Effective Altruism community managed to collectively donate $10,000.To absolutely nobody's surprise, they did ($10k), and then did it again ($20k) and then almost did it a third time ($28,945 as of March 9, 2024). To everyone who gave or helped us spread the word, and on behalf of the untold number of animals these dollars will help, thank you.And although our active promotion on Twitter has come to an end, it is not too late to give! I give a bit more context in a short monologue intro I recorded (sober) after the conversation, so without further ado, Drunk Pigeon Hour:Transcript(Note: very imperfect - sorry!)MonologueHi, this is Aaron. This episode of Pigeon Hour is very special for a couple of reasons.The first is that it was recorded in person, so three of us were physically within a couple feet of each other. Second, it was recorded while we were drunk or maybe just slightly inebriated. Honestly, I didn't get super drunk, so I hope people forgive me for that.But the occasion for drinking was that this, a drunk Pigeon Hour episode, was an incentive for a fundraiser that a couple of friends and I hosted on Twitter, around a little bit before New Year's and basically around Christmas time. We basically said, if we raise $10,000 total, we will do a drunk Pigeon Hour podcast. And we did, in fact, we are almost at $29,000, just shy of it. So technically the fundraiser has ended, but it looks like you can still donate. So, I will figure out a way to link that.And also just a huge thank you to everyone who donated. I know that's really cliche, but this time it really matters because we were raising money for the Effective Altruism Animal Welfare Fund, which is a strong contender for the best use of money in the universe.Without further ado, I present me, Matt, and Laura. Unfortunately, the other co-host Max was stuck in New Jersey and so was unable to participate tragically. Yeah so here it is!ConversationAARONHello, people who are maybe listening to this. I just, like, drank alcohol for, like, the first time in a while. I don't know. Maybe I do like alcohol. Maybe I'll find that out now.MATTUm, All right, yeah, so this is, this is Drunk Pigeon Hour! Remember what I said earlier when I was like, as soon as we are recording, as soon as we press record, it's going to get weird and awkward.LAURAI am actually interested in the types of ads people get on Twitter. Like, just asking around, because I find that I get either, like, DeSantis ads. I get American Petroleum Institute ads, Ashdale College.MATTWeirdly, I've been getting ads for an AI assistant targeted at lobbyists. So it's, it's like step up your lobbying game, like use this like tuned, I assume it's like tuned ChatGPT or something. Um, I don't know, but it's, yeah, it's like AI assistant for lobbyists, and it's like, like, oh, like your competitors are all using this, like you need to buy this product.So, so yeah, Twitter thinks I'm a lobbyist. I haven't gotten any DeSantis ads, actually.AARONI think I might just like have personalization turned off. Like not because I actually like ad personalization. I think I'm just like trying to like, uh, this is, this is like a half-baked protest of them getting rid of circles. I will try to minimize how much revenue they can make from me.MATTSo, so when I, I like went through a Tumblr phase, like very late. In like 2018, I was like, um, like I don't like, uh, like what's happening on a lot of other social media.Like maybe I'll try like Tumblr as a, as an alternative.And I would get a lot of ads for like plus-sized women's flannels.So, so like the Twitter ad targeting does not faze me because I'm like, oh, okay, like, I can, hold on.AARONSorry, keep going. I can see every ad I've ever.MATTCome across, actually, in your giant CSV of Twitter data.AARONJust because I'm a nerd. I like, download. Well, there's actually a couple of things. I just download my Twitter data once in a while. Actually do have a little web app that I might try to improve at some point, which is like, you drop it in and then it turns them. It gives you a csV, like a spreadsheet of your tweets, but that doesn't do anything with any of the other data that they put in there.MATTI feel like it's going to be hard to get meaningful information out of this giant csv in a short amount of time.AARONIt's a giant JSON, actually.MATTAre you just going to drop it all into c long and tell it to parse it for you or tell it to give you insights into your ads.AARONWait, hold on. This is such a.MATTWait. Do people call it “C-Long” or “Clong”?AARONWhy would it be long?MATTWell, because it's like Claude Long.LAURAI've never heard this phrase.MATTThis is like Anthropic's chat bot with a long context with so like you can put. Aaron will be like, oh, can I paste the entire group chat history?AARONOh yeah, I got clong. Apparently that wasn't acceptable so that it.MATTCan summarize it for me and tell me what's happened since I was last year. And everyone is like, Aaron, don't give our data to Anthropic, is already suss.LAURAEnough with the impressions feel about the Internet privacy stuff. Are you instinctively weirded out by them farming out your personal information or just like, it gives me good ads or whatever? I don't care.MATTI lean a little towards feeling weird having my data sold. I don't have a really strong, and this is probably like a personal failing of mine of not having a really strong, well formed opinion here. But I feel a little sketched out when I'm like all my data is being sold to everyone and I don't share. There is this vibe on Twitter that the EU cookies prompts are like destroying the Internet. This is regulation gone wrong. I don't share that instinct. But maybe it's just because I have average tolerance for clicking no cookies or yes cookies on stuff. And I have this vibe that will.AARONSketch down by data. I think I'm broadly fine with companies having my information and selling it to ad targeting. Specifically. I do trust Google a lot to not be weird about it, even if it's technically legal. And by be weird about it, what do mean? Like, I don't even know what I mean exactly. If one of their random employees, I don't know if I got into a fight or something with one of their random employees, it would be hard for this person to track down and just see my individual data. And that's just a random example off the top of my head. But yeah, I could see my view changing if they started, I don't know, or it started leaching into the physical world more. But it seems just like for online ads, I'm pretty cool with everything.LAURAHave you ever gone into the ad personalization and tried see what demographics they peg you?AARONOh yeah. We can pull up mine right now.LAURAIt's so much fun doing that. It's like they get me somewhat like the age, gender, they can predict relationship status, which is really weird.AARONThat's weird.MATTDid you test this when you were in and not in relationships to see if they got it right?LAURANo, I think it's like they accumulate data over time. I don't know. But then it's like we say that you work in a mid sized finance. Fair enough.MATTThat's sort of close.LAURAYeah.AARONSorry. Keep on podcasting.LAURAOkay.MATTDo they include political affiliation in the data you can see?AARONOkay.MATTI would have been very curious, because I think we're all a little bit idiosyncratic. I'm probably the most normie of any of us in terms of. I can be pretty easily sorted into, like, yeah, you're clearly a Democrat, but all of us have that classic slightly. I don't know what you want to call it. Like, neoliberal project vibe or, like, supply side. Yeah. Like, some of that going on in a way that I'm very curious.LAURAThe algorithm is like, advertising deSantis.AARONYeah.MATTI guess it must think that there's some probability that you're going to vote in a republican primary.LAURAI live in DC. Why on earth would I even vote, period.MATTWell, in the primary, your vote is going to count. I actually would think that in the primary, DC is probably pretty competitive, but I guess it votes pretty. I think it's worth.AARONI feel like I've seen, like, a.MATTI think it's probably hopeless to live. Find your demographic information from Twitter. But, like.AARONAge 13 to 54. Yeah, they got it right. Good job. I'm only 50, 99.9% confident. Wait, that's a pretty General.MATTWhat's this list above?AARONOh, yeah. This is such a nerd snipe. For me, it's just like seeing y'all. I don't watch any. I don't regularly watch any sort of tv series. And it's like, best guesses of, like, I assume that's what it is. It thinks you watch dune, and I haven't heard of a lot of these.MATTWait, you watch cocaine there?AARONBig bang theory? No, I definitely have watched the big Bang theory. Like, I don't know, ten years ago. I don't know. Was it just, like, random korean script.MATTOr whatever, when I got Covid real bad. Not real bad, but I was very sick and in bed in 2022. Yeah, the big bang theory was like, what I would say.AARONThese are my interest. It's actually pretty interesting, I think. Wait, hold on. Let me.MATTOh, wait, it's like, true or false for each of these?AARONNo, I think you can manually just disable and say, like, oh, I'm not, actually. And, like, I did that for Olivia Rodrigo because I posted about her once, and then it took over my feed, and so then I had to say, like, no, I'm not interested in Olivia Rodrigo.MATTWait, can you control f true here? Because almost all of these. Wait, sorry. Is that argentine politics?AARONNo, it's just this.MATTOh, wait, so it thinks you have no interest?AARONNo, this is disabled, so I haven't. And for some reason, this isn't the list. Maybe it was, like, keywords instead of topics or something, where it was the.MATTGot it.AARONYes. This is interesting. It thinks I'm interested in apple stock, and, I don't know, a lot of these are just random.MATTWait, so argentine politics was something it thought you were interested in? Yeah. Right.AARONCan.MATTDo you follow Maya on Twitter?AARONWho's Maya?MATTLike, monetarist Maya? Like, neoliberal shell two years ago.AARONI mean, maybe. Wait, hold on. Maybe I'm just like.MATTYeah, hardcore libertarianism.LAURAYeah. No, so far so good with him. I feel like.AARONMaia, is it this person? Oh, I am.MATTYeah.AARONOkay.MATTYeah, she was, like, neoliberal shell two years ago.AARONSorry, this is, like, such an errands. Like snipe. I got my gender right. Maybe. I don't know if I told you that. Yeah. English. Nice.MATTWait, is that dogecoin?AARONI assume there's, like, an explicit thing, which is like, we're going to err way on the side of false positives instead of false negatives, which is like. I mean, I don't know. I'm not that interested in AB club, which.MATTYou'Re well known for throwing staplers at your subordinate.AARONYeah.LAURAWait, who did you guys support in 2020 primary?MATTYou were a Pete stan.LAURAI was a Pete stan. Yes, by that point, definitely hardcore. But I totally get. In 2016, I actually was a Bernie fan, which was like, I don't know how much I was really into this, or just, like, everybody around me was into it. So I was trying to convince myself that he was better than Hillary, but I don't know, that fell apart pretty quickly once he started losing. And, yeah, I didn't really know a whole lot about politics. And then, like, six months later, I became, like, a Reddit libertarian.AARONWe think we've talked about your ideological evolution.MATTHave you ever done the thing of plotting it out on the political? I feel like that's a really interesting.LAURAExercise that doesn't capture the online. I was into Ben Shapiro.MATTReally? Oh, my God. That's such a funny lore fact.AARONI don't think I've ever listened to Ben Shapiro besides, like, random clips on Twitter that I like scroll?MATTI mean, he talks very fast. I will give him that.LAURAAnd he's funny. And I think it's like the fast talking plus being funny is like, you can get away with a lot of stuff and people just end up like, oh, sure, I'm not really listening to this because it's on in the background.AARONYeah.MATTIn defense of the Bernie thing. So I will say I did not support Bernie in 2016, but there was this moment right about when he announced where I was very intrigued. And there's something about his backstory that's very inspiring. This is a guy who has been just extraordinarily consistent in his politics for close to 50 years, was saying lots of really good stuff about gay rights when he was like, Burlington mayor way back in the day, was giving speeches on the floor of the House in the number one sound very similar to the things he's saying today, which reflects, you could say, maybe a very myopic, closed minded thing, but also an ideological consistency. That's admirable. And I think is pointing at problems that are real often. And so I think there is this thing that's, to me, very much understandable about why he was a very inspiring candidate. But when it came down to nitty gritty details and also to his decisions about who to hire subordinates and stuff, very quickly you look at the Bernie campaign alumni and the nuances of his views and stuff, and you're like, okay, wait, this is maybe an inspiring story, but does it actually hold up?AARONProbably not.LAURAYeah, that is interesting. It's like Bernie went woke in 2020, kind of fell apart, in my opinion.AARONI stopped following or not following on social media, just like following him in general, I guess. 2016 also, I was 16. You were not 16. You were.MATTYeah, I was in college at that time, so I was about 20.AARONSo that was, you can't blame it. Anything that I do under the age of 18 is like just a race when I turn 18.LAURAOkay, 2028 draft. Who do we want to be democratic nominee?AARONOh, Jesse from pigeonhole. I honestly think he should run. Hello, Jesse. If you're listening to this, we're going to make you listen to this. Sorry. Besides that, I don't know.MATTI don't have, like, an obvious front runner in mind.AARONWait, 2028? We might be dead by 2028. Sorry, we don't talk about AI.MATTYeah.AARONNo, but honestly, that is beyond the range of planability, I think. I don't actually think all humans are going to be dead by 2028. But that is a long way away. All I want in life is not all I want. This is actually what I want out of a political leader. Not all I want is somebody who is good on AI and also doesn't tells the Justice Department to not sue California or whatever about their gestation. Or maybe it's like New Jersey or something about the gestation crate.MATTOh, yeah. Top twelve.AARONYeah. Those are my two criteria.MATTCorey Booker is going to be right on the latter.AARONYeah.MATTI have no idea about his views on.AARONIf to some extent. Maybe this is actively changing as we speak, basically. But until recently it wasn't a salient political issue and so it was pretty hard to tell. I don't know. I don't think Biden has a strong take on it. He's like, he's like a thousand years old.LAURAWatch what Mitch should have possibly decided. That's real if we don't do mean.AARONBut like, but his executive order was way better than I would have imagined. And I, like, I tweeted about, know, I don't think I could have predicted that necessarily.MATTI agree. I mean, I think the Biden administration has been very reasonable on AI safety issues and that generally is reflective. Yeah, I think that's reflective of the.AARONTongue we know Joe Biden is listening to.MATTOkay.AARONOkay.MATTTopics that are not is like, this is a reward for the fundraiser. Do we want to talk about fundraiser and retrospective on that?AARONSure.MATTBecause I feel like, I don't know. That ended up going at least like one sigma above.AARONHow much? Wait, how much did we actually raise?MATTWe raised like 22,500.LAURAOkay. Really pissed that you don't have to go to Ava.AARONI guess this person, I won't name them, but somebody who works at a prestigious organization basically was seriously considering donating a good amount of his donation budget specifically for the shrimp costume. And, and we chatted about it over Twitter, DM, and I think he ended up not doing it, which I think was like the right call because for tax reasons, it would have been like, oh. He thought like, oh, yeah, actually, even though that's pretty funny, it's not worth losing. I don't know, maybe like 1000 out of $5,000 tax reasons or whatever. Clearly this guy is actually thinking through his donations pretty well. But I don't know, it brought him to the brink of donating several, I think, I don't know, like single digit thousands of dollars. Exactly.LAURAClearly an issue in the tax.AARONDo you have any tax take? Oh, wait, sorry.MATTYeah, I do think we should like, I mean, to the extent you are allowed by your employer too, in public space.AARONAll people at think tanks, they're supposed to go on podcast and tweet. How could you not be allowed to do that kind of thing?MATTSorry, keep going. But yeah, no, I mean, I think it's worth dwelling on it a little bit longer because I feel like, yeah, okay, so we didn't raise a billion dollars as you were interested in doing.AARONYeah. Wait, can I make the case for like. Oh, wait. Yeah. Why? Being slightly unhinged may have been actually object level. Good. Yeah, basically, I think this didn't end up exposed to. We learned this didn't actually end up happening. I think almost all of the impact money, because it's basically one of the same in this context. Sorry. Most of the expected money would come in the form of basically having some pretty large, probably billionaire account, just like deciding like, oh, yeah, I'll just drop a couple of mil on this funny fundraiser or whatever, or maybe less, honestly, listen, $20,000, a lot of money. It's probably more money than I have personally ever donated. On the other hand, there's definitely some pretty EA adjacent or broadly rationalist AI adjacent accounts whose net worth is in at least tens of millions of dollars, for whom $100,000 just would not actually affect their quality of life or whatever. And I think, yeah, there's not nontrivial chance going in that somebody would just decide to give a bunch of money.MATTI don't know. My view is that even the kinds of multimillionaires and billionaires that hang out on Twitter are not going to ever have dropped that much on a random fundraiser. They're more rational.AARONWell, there was proof of concept for rich people being insane. Is Balaji giving like a million dollars to James Medlock.MATTThat's true.AARONThat was pretty idiosyncratic. Sorry. So maybe that's not fair. On the other hand. On the other hand, I don't know, people do things for clout. And so, yeah, I would have, quote, tweeted. If somebody was like, oh yeah, here's $100,000 guys, I would have quote, tweeted the shit out of them. They would have gotten as much possible. I don't know. I would guess if you have a lot of rich people friends, they're also probably on Twitter, especially if it's broadly like tech money or whatever. And so there's that. There's also the fact that, I don't know, it's like object people, at least some subset of rich people have a good think. EA is basically even if they don't identify as an EA themselves, think like, oh yeah, this is broadly legit and correct or whatever. And so it's not just like a random.MATTThat's true. I do think the choice of the animal welfare fund made that harder. Right. I think if it's like bed nets, I think it's more likely that sort of random EA rich person would be like, yes, this is clearly good. And I think we chose something that I think we could all get behind.AARONBecause we have, there was a lot of politicking around.MATTYeah, we all have different estimates of the relative good of different cause areas and this was the one we could very clearly agree on, which I think is very reasonable and good. And I'm glad we raised money for the animal welfare fund, but I do think that reduces the chance of, yeah.LAURAI think it pushes the envelope towards the animal welfare fund being more acceptable as in mainstream ea.org, just like Givewell would be. And so by forcing that issue, maybe we have done more good for the.AARONThat there's like that second order effect. I do just think even though you're like, I think choosing this over AMF or whatever, global health fund or whatever decreased the chance of a random person. Not a random person, but probably decrease the total amount of expected money being given. I think that was just trumped by the fact that I think the animal welfare, the number I pull out of thin air is not necessarily not out of thin air, but very uncertain is like 1000 x or whatever relative to the standards you vote for. Quote, let it be known that there is a rabbit on the premises. Do they interact with other rodents?MATTOkay, so rabbits aren't rodents. We can put this on the pod. So rabbits are lagging wars, which is.AARONFuck is that?MATTIt's a whole separate category of animals.AARONI just found out that elk were part of it. Like a type of deer. This is another world shattering insight.MATTNo, but rabbits are evolutionarily not part of the same. I guess it's a family on the classification tree.AARONNobody, they taught us that in 7th grade.MATTYeah, so they're not part of the same family as rodents. They're their own thing. What freaks me out is that guinea pigs and rabbits seem like pretty similar, they have similar diet.AARONThat's what I was thinking.MATTThey have similar digestive systems, similar kind of like general needs, but they're actually like, guinea pigs are more closely related to rats than they are to rabbits. And it's like a convergent evolution thing that they ended up.AARONAll mammals are the same. Honestly.MATTYeah. So it's like, super weird, but they're not rodents, to answer your question. Rabbits do like these kinds of rabbits. So these are all pet rabbits are descended from european. They're not descended from american rabbits because.LAURAAmerican rabbits like cotton tails. Oh, those are different.MATTYeah. So these guys are the kinds of rabbits that will live in warrens. Warrens. So, like, tunnel systems that they like. Like Elizabeth Warren. Yeah. And so they'll live socially with other rabbits, and they'll dig warrens. And so they're used to living in social groups. They're used to having a space they need to keep clean. And so that's why they can be, like, litter box trained, is that they're used to having a warren where you don't just want to leave poop everywhere. Whereas american rabbits are more solitary. They live above ground, or in my understanding is they sometimes will live in holes, but only occupying a hole that another animal has dug. They won't do their hole themselves. And so then they are just not social. They're not easily litter box trained, that kind of stuff. So all the domestic rabbits are bred from european ones.AARONI was thinking, if you got a guinea pig, would they become friends? Okay.MATTSo apparently they have generally similar dispositions and it can get along, but people don't recommend it because each of them can carry diseases that can hurt the other one. And so you actually don't want to do it. But it does seem very cute to have rabbit.AARONNo, I mean, yeah. My last pet was a guinea pig, circa 20. Died like, a decade ago. I'm still not over it.MATTWould you consider another one?AARONProbably. Like, if I get a pet, it'll be like a dog or a pig. I really do want a pig. Like an actual pig.MATTWait, like, not a guinea pig? Like a full size pig?AARONYeah. I just tweeted about this. I think that they're really cool and we would be friends. I'm being slightly sarcastic, but I do think if I had a very large amount of money, then the two luxury purchases would be, like, a lot of massages and a caretaker and space and whatever else a pig needs. And so I could have a pet.MATTLike, andy organized a not EADC, but EADC adjacent trip to Rosie's farm sanctuary.AARONOh, I remember this. Yeah.MATTAnd we got to pet pigs. And they were very sweet and seems very cute and stuff. They're just like, they feel dense, not like stupid. But when you pet them, you're like, this animal is very large and heavy for its size. That was my biggest surprising takeaway, like, interacting with the hair is not soft either. No, they're pretty coarse, but they seem like sweeties, but they are just like very robust.LAURAHave you guys seen Dave?AARONYes.LAURAThat's like one of the top ten movies of all time.AARONYou guys watch movies? I don't know. Maybe when I was like four. I don't like.LAURAOkay, so the actor who played farmer Hoggett in this movie ended up becoming a vegan activist after he realized, after having to train all of the animals, that they were extremely intelligent. And obviously the movie is about not killing animals, and so that ended up going pretty well.AARONYeah, that's interesting. Good brown.MATTOkay, sorry. Yeah, no, this is all tracked. No, this is great. We are doing a drunk podcast rather than a sober podcast, I think, precisely because we are trying to give the people some sidetracks and stuff. Right. But I jokingly put on my list of topics like, we solved the two envelopes paradox once and for all.AARONNo, but it's two boxing.MATTNo. Two envelopes. No. So this is the fundamental challenge to questions about, I think one of the fundamental challenges to be like, you multiply out the numbers and the number.AARONYeah, I feel like I don't have like a cash take. So just like, tell me the thing.MATTOkay.AARONI'll tell you the correct answer. Yeah.MATTOkay, great. We were leading into this. You were saying, like, animal charity is 1000 x game, right?AARONConditional. Yeah.MATTAnd I think it's hard to easily get to 1000 x, but it is totally possible to get to 50 x if you just sit down and multiply out numbers and you're like, probability of sentience and welfare range.AARONI totally stand by that as my actual point estimate. Maybe like a log mean or something. I'm actually not sure, but. Sorry, keep going.MATTOkay, so one line of argument raised against this is the two envelopes problem, and I'm worried I'm going to do a poor job explaining this. Laura, please feel free to jump in if I say something wrong. So two envelopes is like, it comes from the thing of, like, suppose you're given two envelopes and you're told that one envelope has twice as much money in it as the other.AARONOh, you are going to switch back and forth forever.MATTExactly. Every time. You're like, if I switch the other envelope and it has half as much money as this envelope, then I lose 0.5. But if it has twice as much money as this envelope, then I gain one. And so I can never decide on which envelope because it always looks like it's positive ev to switch the other. So that's where the name comes from.AARONI like a part that you're like, you like goggles?MATTSo let me do the brief summary, which is that basically, depending on which underlying units you pick, whether you work in welfare range, units that are using one human as the baseline or one chicken as the baseline, you can end up with different outputs of the expected value calculation. Because it's like, basically, is it like big number of chickens times some fraction of the human welfare range that dominates? Or is it like some small probability that chickens are basically not sentient times? So then a human has like a huge human's welfare range is huge in chicken units, and which of those dominates is determined by which unit you work in.AARONI also think, yeah, this is not a good conducive to this problem. Is not conducive to alcohol or whatever. Or alcohol is not going to this issue. To this problem or whatever. In the maximally abstract envelope thing. I have an intuition that's something weird kind of probably fake going on. I don't actually see what the issue is here. I don't believe you yet that there's like an actual issue here. It's like, okay, just do the better one. I don't know.MATTOkay, wait, I'll get a piece of paper. Talk amongst yourselves, and I think I'll be able to show this is like.LAURAMe as the stats person, just saying I don't care about the math. At some point where it's like, look, I looked at an animal and I'm like, okay, so we have evolutionarily pretty similar paths. It would be insane to think that it's not feeling like, it's not capable of feeling hedonic pain to pretty much the same extent as me. So I'm just going to ballpark it. And I don't actually care for webs.AARONI feel like I've proven my pro animal bona fide. I think it's bona fide. But here, and I don't share that intuition, I still think that we can go into that megapig discourse. Wait, yeah, sort of. Wait, not exactly megapig discourse. Yeah, I remember. I think I got cyberbullyed by, even though they didn't cyberbully me because I was informed of offline bullying via cyber about somebody's, sorry, this is going to sound absolutely incoherent. So we'll take this part out. Yeah. I was like, oh, I think it's like some metaphysical appeal to neuron counts. You specifically told me like, oh, yeah, Mr. So and so didn't think this checked out. Or whatever. Do you know what I'm talking about?LAURAYeah.AARONOkay. No, but maybe I put it in dawn or Cringey or pretentious terms, but I do think I'm standing by my metaphysical neurons claim here. Not that I'm super confident in anything, but just that we're really radically unsure about the nature of sentience and qualia and consciousness. And probably it has something to do with neurons, at least. They're clearly related in a very boring sciency way. Yeah. It's not insane to me that, like, that, like. Like the unit of. Yeah, like the. The thing. The thing that, like, produces or like is, or like is directly, like one to one associated with, like, particular, like. Like, I guess, amount, for lack of better terms, of conscious experience, is some sort of physical thing. The neurons jumps out as the unit that might make sense. And then there's like, oh, yeah, do we really think all the neurons that control the tongue, like the motor function of the tongue, are those really make you quadrillion more important than a seal or whatever? And then I go back to, okay, even though I haven't done any research on this, maybe it's just like opiate. The neurons directly related neuron counts directly of. Sorry. Neurons directly involved in pretty low level hedonic sensations. The most obvious one would be literal opioid receptors. Maybe those are the ones that matter. This is like, kind of. I feel like we've sort of lost the plot a little.MATTOkay, this is like weird drunk math.AARONBut I think your handwriting is pretty good.MATTI think I have it. So suppose we work in human units. I have a hypothetical intervention that can help ten chickens or one human, and we assume that when I say help, it's like, help them. The same of it. So if I work in human units, I say maybe there is a 50% chance that a chicken is zero one to 1100 of a human and a 50% chance that a chicken and a human are equal. Obviously, this is a thought experiment. I'm not saying that this is my real world probabilities, but suppose that these are my credences. So I do out the EV. The thing that helps ten chickens. I say that, okay, in half of the world, chickens are one 100th of a human, so helping ten of them is worth, like, zero five. Sorry, helping ten of them is zero one. And so 0.5 times zero one times ten is zero five. And then in the other half of the world, I say that a chicken and a human are equal. So then my intervention helps ten chickens, which is like helping ten humans so my total credence, like the benefit in that set of worlds with my 0.5 probability, is five. And so in the end, the chicken intervention wins because it has, on net, an ev of 5.5 versus one for the human intervention. Because the human intervention always helps one human. I switch it around and I say my base unit of welfare range, or, like moral weight, or whatever you want to say, is chicken units. Like, one chicken's worth of moral weight. So in half of the world, a human is worth 100 chickens, and then in the other half of the world, a human is worth one chicken. So I do out the ev for my intervention that helps the one human. Now, in the chicken units, and in chicken units, like, half of the time, that human is worth 100 chickens. And so I get 0.5 times, 100 times one, which is 50. And then in the other half of the world, the chicken and the human are equal. And so then it's 0.5 times one, times one, because I'm helping one human, so that's 0.5. The ev is 50.5. And then I do have my ev for my chicken welfare thing. That's, like, ten chickens, and I always help ten chickens. And so it's ten as my units of good. So when I worked in human units, I said that the chickens won because it was 5.5 human units versus one human unit for helping the human. When I did it in chicken units, it was 50.5 to help the humans versus ten to help the chickens. And so now I'm like, okay, my ev is changing just based on which units I work in. And I think this is, like, the two envelopes problem that's applied to animals. Brian Tomasic has, like, a long post about this, but I think this is, like, this is a statement or an example of the problem.AARONCool.LAURACan I just say something about the moral weight project? It's like, really just. We ended up coming up with numbers, which I think may have been a bit of a mistake in the end, because I think the real value of that was going through the literature and finding out the similarities and the traits between animals and humans, and then there are a surprising number of them that we have in common. And so at the end of the day, it's a judgment call. And I don't know what you do with it, because that is, like, a legit statistical problem with things that arises when you put numbers on stuff.MATTSo I'm pretty sympathetic to what you're saying here of, like, the core insight of the moral weight project is, like, when we look at features that could plausibly determine capacity to experience welfare, we find that a pig and a human have a ton in common. Obviously, pigs cannot write poetry, but they do show evidence of grief behavior when another pig dies. And they show evidence of vocalizing in response to pain and all of these things. I think coming out of the moral waste project being like, wow. Under some form of utilitarianism, it's really hard to justify harms to, or like harms to pigs. Really. Morally matter makes complete sense. I think the challenge here is when you get to something like black soldier flies or shrimp, where when you actually look at the welfare range table, you see that the number of proxies that they likely or definitely have is remarkably low. The shrimp number is hinging on. It's not hinging on a ton. They share a few things. And because there aren't that many categories overall, that ends up being in the median case. Like, they have a moral weight, like one 30th of a human. And so I worry that sort of your articulation of the benefit starts to break down when you get to those animals. And we start to like, I don't know what you do without numbers there. And I think those numbers are really susceptible to this kind of 200.AARONI have a question.MATTYeah, go.AARONWait. This supposed to be like 5.5 versus one?MATTYeah.AARONAnd this is 50.5 versus ten? Yeah. It sound like the same thing to me.MATTNo, but they've inverted this case, the chickens one. So it's like when I'm working in human units, right? Like, half the time, I help.AARONIf you're working in human units, then the chicken intervention looks 5.5 times better. Yes. Wait, can I write this down over here?MATTYeah. And maybe I'm not an expert on this problem. This is just like something that tortures me when I try and sleep at night, not like a thing that I've carefully studied. So maybe I'm stating this wrong, but, yeah. When I work in human units, the 50% probability in this sort of toy example that the chickens and the humans are equal means that the fact that my intervention can help more chickens makes the ev higher. And then when I work in the chicken units, the fact that human might be 100 times more sentient than the chicken or more capable of realizing welfare, to be technical, that means the human intervention just clearly wins.AARONJust to check that I would have this right, the claim is that in human units, the chicken intervention looks 5.5 times better than the human intervention. But when you use chicken units, the human intervention looks 5.5 times better than the chicken intervention. Is that correct?MATTYes, that's right.AARONWait, hold on. Give me another minute.MATTThis is why doing this drunk was a bad idea.AARONIn human.LAURANo, I think that's actually right. And I don't know what to do about the flies and shrimp and stuff like this. This is like where I draw my line of like, okay, so lemonstone quote.MATTTweeted me, oh, my God.LAURAI think he actually had a point of, there's a type of ea that is like, I'm going to set my budget constraint and then maximize within that versus start with a blank slate and allow the reason to take me wherever it goes. And I'm definitely in the former camp of like, my budget constraint is like, I care about humans and a couple of types of animals, and I'm just like drawing the line there. And I don't know what you do with the other types of things.MATTI am very skeptical of arguments that are like, we should end Medicare to spend it all on shrimp.AARONNo one's suggesting that. No, there's like a lot of boring, prosaic reasons.MATTI guess what I'm saying is there's a sense in which, like, totally agreeing with you. But I think the challenge is that object level.AARONYeah, you set us up. The political economy, I like totally by double it.MATTI think that there is. This is great. Aaron, I think you should have to take another shot for.AARONI'm sorry, this isn't fair. How many guys, I don't even drink, so I feel like one drink is like, is it infinity times more than normalize it? So it's a little bit handle.MATTI think there has to be room for moral innovation in my view. I think that your line of thinking, we don't want to do radical things based on sort of out there moral principles in the short term. Right. We totally want to be very pragmatic and careful when our moral ideas sort of put us really far outside of what's socially normal. But I don't think you get to where we are. I don't know what we owe the future was like a book that maybe was not perfect, but I think it eloquently argues with the fact that the first person to be like, hey, slavery in the Americas is wrong. Or I should say really the first person who is not themselves enslaved. Because of course, the people who are actually victims of this system were like, this is wrong from the start. But the first people to be like, random white people in the north being like, hey, this system is wrong. Looks super weird. And the same is true for almost any moral innovation. And so you have to, I think saying, like, my budget constraint is totally fixed seems wrong to me because it leaves no room for being wrong about some of your fundamental morals.LAURAYeah, okay. A couple of things here. I totally get that appeal 100%. At the same time, a lot of people have said this about things that now we look back at as being really bad, like the USSR. I think communism ends up looking pretty bad in retrospect, even though I think there are a lot of very good moral intuitions underpinning it.AARONYeah, I don't know. It's like, mostly an empirical question in that case, about what government policies do to human preference satisfaction, which is like, pretty. Maybe I'm too econ. These seem like very different questions.LAURAIt's like we let our reason go astray, I think.MATTRight, we, as in some humans.AARONNo, I think. Wait, at first glance. At first glance, I think communism and things in that vicinity seem way more intuitively appealing than they actually, or than they deserve to be, basically. And the notion of who is it? Like Adam Smith? Something Smith? Yeah, like free hand of the market or whatever. Invisible hand. Invisible free hand of the bunny ear of the market. I think maybe it's like, field intuitive to me at this point, because I've heard it a lot. But no, I totally disagree that people's natural intuition was that communism can't work. I think it's like, isn't true.MATTI'm not sure you guys are disagreeing with one.AARONYeah.MATTLike, I think, Laura, if I can attempt to restate your point, is that to at least a subset of the people in the USSR at the time of the russian revolution, communism plausibly looked like the same kind of moral innovation as lots of stuff we looked back on as being really good, like the abolition of slavery or like, women's rights or any of those other things. And so you need heuristics that will defend against these false moral innovations.AARONWait, no, you guys are both wrong. Wait, hold on. No, the issue there isn't that we disregard, I guess, humans, I don't know exactly who's responsible for what, but people disregarded some sort of deserving heuristic that would have gardened against communism. The issue was that, like, it was that, like, we had, like, lots of empirical, or, like, it's not even necessarily. I mean, in this case, it is empirical evidence, but, like, like, after a couple years of, like, communism or whatever, we had, like, lots of good evidence to think, oh, no, books like that doesn't actually help people, and then they didn't take action on that. That's the problem. If we were sitting here in 1910 or whatever, and I think it's totally possible, I will be convinced communism is, in fact, the right thing to do. But the thing that would be wrong is if, okay, five years later, you have kids starving or people starving or whatever, and maybe you can find intellectuals who claim and seem reasonably correct that they can explain how this downstream of your policies. Then doubling down is the issue, not the ex ante hypothesis that communism is good. I don't even know if that made any sense, I think.LAURABut we're in the ex ante position right now.AARONYeah, totally. Maybe we'll find out some sort of, whether it's empirical or philosophical or something like maybe in five years or two years or whatever, there'll be some new insight that sheds light on how morally valuable shrimp are. And we should take that into account.LAURAI don't know. Because it's really easy to get good feedback when other fellow humans are starving to death versus. How are you supposed to judge? No, we've made an improvement.AARONYeah, I do think. Okay. Yes. That's like a substantial difference. Consciousness is, like, extremely hard. Nobody knows what the hell is going on. It kind of drives me insane.MATTWhomstemonga has not been driven insane by the hard problem of consciousness.AARONYeah. For real. I don't know. I don't have to say. It's like, you kind of got to make your best guess at some point.MATTOkay, wait, so maybe tacking back to how to solve it, did you successfully do math on this piece of paper?AARONMostly? No, mostly I was word selling.MATTI like the verb form there.AARONYeah. No, I mean, like, I don't have, like, a fully thought out thing. I think in part this might be because of the alcohol. I'm pretty sure that what's going on here is just that, like, in fact, like, there actually is an asymmetry between chicken units and human units, which is that. Which is that we have much better idea. The real uncertainty here is how valuable a chicken is. There's probably somebody in the world who doubts this, but I think the common sense thing and thing that everybody assumes is we basically have it because we're all humans and there's a lot of good reasons to think we have a decent idea of how valuable another human life is. And if we don't, it's going to be a lot worse for other species. And so just, like, taking that as a given, the human units are the correct unit because the thing with the unit is that you take it as given or whatever. The real uncertainty here isn't the relationship between chickens and humans. The real question is how valuable is a chicken? And so the human units are just like the correct one to use.LAURAYeah, there's something there, which is the right theory is kind of driving a lot of the problem in the two envelope stuff. Because if you just chose one theory, then the units wouldn't really matter which one. The equality theory is like, you've resolved all the inter theoretic uncertainty and so wouldn't that get rid of.AARONI don't know if you know, if there's, like. I'm not exactly sure what you mean by theory.LAURALike, are they equal, the equality theory versus are they 1100 theory? And we're assuming that each of them has five probabilities each end. So if we resolved that, it's like we decide upon the 1100 theory, then the problem goes away.AARONYeah, I mean, that's true, but you might not be able to.MATTYeah, I think it doesn't reflect our current state or, like.AARONNo, just like taking as given the numbers, like, invented, which I think is fine for the illustration of the problem. Maybe a better example is what's, like, another thing, chicken versus a rabbit. I don't know. Or like rabbits. I don't know.MATTChicken versus shrimp. I think it's like a real one. Because if you're the animal welfare fund, you are practically making that decision.AARONYeah. I think that becomes harder. But it's not, like, fundamentally different. And it's like the question of, like, okay, which actually makes sense, makes more sense to use as a unit. And maybe you actually can come up with two, if you can just come up with two different species for which, on the merits, they're equally valid as a unit and there's no issue anymore. It really is 50 50 in the end.MATTYeah. I don't know. I see the point you're making. With humans, we know in some sense we have much more information about how capable of realizing welfare a human is. But I guess I treat this as, like, man, I don't know. It's like why all of my confidence intervals are just, like, massive on all these things is I'm just very confused by these problems and how much that.AARONSeems like I'm confused by this one. Sorry, I'm, like, half joking. It is like maybe. I don't know, maybe I'll be less confident. Alcohol or so.MATTYeah, I don't know. I think it's maybe much more concerning to me the idea that working in a different unit changes your conclusion radically.AARONThan it is to you.LAURASometimes. I don't know if this is, like, too much of a stoner cake or something like that.AARONBring it on.LAURAI kind of doubt working with numbers at all.MATTOkay. Fit me well.LAURAIt's just like when he's.AARONStop doing that.LAURAI don't know what to do, because expected value theory. Okay, so one of the things that, when we hired a professional philosopher to talk about uncertainty.MATTPause for a sec. Howie is very sweetly washing his ears, which is very cute in the background. He's like, yeah, I see how he licks his paws and squeezes his ear.AARONIs it unethical for me to videotape?MATTNo, you're more than welcome to videotape it, but I don't know, he might be done.AARONYeah, that was out.MATTLaura, I'm very sorry. No, yeah, you were saying you hired the professional philosopher.LAURAYeah. And one of the first days, she's like, okay, well, is it the same type of uncertainty if we, say, have a one in ten chance of saving the life of a person we know for sure is conscious, versus we have a certain chance of saving the life of an animal that has, like, a one in ten probability of being sentient? These seem like different types.AARONI mean, maybe in some sense they're like different types. Sure. But what are the implications? It's not obviously the same.LAURAIt kind of calls into question as to whether we can use the same mathematical approach for analyzing each of these.AARONI think my main take is, like, you got a better idea? That was like, a generic.LAURANo, I don't.AARONYeah. It's like, okay, yeah, these numbers are, like, probably. It seems like the least bad option if you're going by intuition. I don't know. I think all things considered, sometimes using numbers is good because our brains aren't built to handle getting moral questions correct.MATTYeah, I mean, I think that there is a very strong piece of evidence for what you're saying, Aaron, which is.AARONThe whole paper on this. It's called the unreasonable efficacy of mathematics in the natural sciences.MATTOr this is. This is interesting. I was going to make sort of an easier or simpler argument, which is just like, I think the global health ea pitch of, like, we tend to get charity radically wrong.AARONOften.MATTCharities very plausibly do differ by 100 x or 1000 x in cost effectiveness. And most of the time, most people don't take that into account and end up helping people close to them or help an issue that's salient to them or help whatever they've heard about most and leave opportunities for doing what I think is very difficult to argue as not being radically more effective opportunities on the table as a result. Now, I led into this saying that I have this very profound uncertainty when it comes to human versus animal trade offs. So I'm not saying that, yes, we just should shut up and multiply. But I do think that is sort of like the intuition for why I think the stoner take is very hard for me to endorse is that we know in other cases, actually bringing numbers to the problem leads to saving many more lives of real people who have all of the same hopes and dreams and fears and feelings and experiences as the people who would have been saved in alternate options.LAURAIsn't that just like still underlying this is we're sure that all humans are equal. And that's like our theory that we have endorsed.AARONWait, what?MATTOr like on welfare ranges, the differences among different humans are sufficiently small in terms of capacity to realize welfare. That plausibly they are.AARONYeah, I don't think anyone believes that. Does anyone believe that? Wait, some people that everybody's hedonic range is the same.LAURARandomly select a person who lives in Kenya. You would think that they have the same welfare range, a priority as somebody.MATTWho lives in the description. The fundamental statistics of describing their welfare range are the same.AARONYeah, I think that's probably correct. It's also at an individual level, I think it's probably quite varied between humans.LAURASo I don't think we can say that we can have the same assumption about animals. And that's where it kind of breaks down, is we don't know the right theory to apply it.AARONWell, yeah, it's a hard question. Sorry, I'm being like kind of sarcastic.LAURAI think you have to have the theory right. And you can't easily average over theories with numbers.MATTYeah, no, I mean, I think you're right. I think this is the challenge of the two envelopes. Problem is exactly this kind of thing. I'm like four chapters into moral uncertainty. The book.AARONBy Will.MATTYeah. McCaskill, Ord and Bryke Fist. I'm probably getting that name. But they have a third co author who is not as much of like an.AARONYeah, I don't know. I don't have any super eloquent take except that to justify the use of math right now. Although I actually think I could. Yeah, I think mostly it's like, insofar as there's any disagreement, it's like we're both pointing at the issue, pointing at a question, and saying, look at that problem. It's, like, really hard. And then I'm saying like, yeah, I know. Shit. You should probably just do your best to answer it. Sorry, maybe I'm just not actually adding any insight here or whatever, but I agree with you that a lot of these problems are very difficult, actually. Sorry, maybe this is, like, a little bit of a nonsense. Whatever. Getting back to the hard problem of consciousness, I really do think it feels like a cruel joke that we have to implicitly, we have to make decisions about potentially gigantic numbers of digital lives or, like, digital sentience or, you know, whatever you want to call it, without having any goddamn idea, like, what the fuck is up with consciousness. And, I don't know, it doesn't seem fair. Okay.MATTYeah, wait, okay, so fundraiser. This is great. We've done all of these branching off things. So we talked about how much we raised, which was, like, amount that I was quite happy with, though. Maybe that's, like, selfish because I didn't have to wear a shrink costume. And we talked about. Cause prio. We haven't talked about the whole fake OpenAI thing.AARONFake open AI.MATTWait. Like the entire.AARONOh, well, shout out to I really. God damn it, Qualy. I hope you turn into a human at some point, because let it be known that Qualy made a whole ass Google Doc to plan out the whole thing and was, like, the driving. Yeah, I think it's fair to say Qualy was the driving force.MATTYeah, totally. Like, absolutely had the concept, did the Google Doc. I think everybody played their parts really well, and I think that was very fun.AARONYeah, you did. Good job, everybody.MATTBut, yeah, that was fun. It was very unexpected. Also, I enjoyed that. I was still seeing tweets and replies that were like, wait, this was a bit. I didn't get this after the end of it, which maybe suggests. But if you look at the graph I think I sent in, maybe we.AARONShould pull up my. We can analyze my Twitter data and find out which things got how many views have.MATTLike, you have your text here. I think the graph of donations by date is, like, I sent in the text chat between.AARONMaybe I can pull it like, media.MATTLike you and me and Max and Laura. And it's very clear that that correlated with a. I think it's probably pretty close to the end.AARONMaybe I just missed this. Oh, Laura, thank you for making.MATTYeah, the cards were amazing cards.AARONThey're beautiful.MATTOh, wait, okay, maybe it's not. I thought I said. Anyway, yeah, we got, like, a couple grand at the start, and then definitely at least five grand, maybe like, ten grand, somewhere in the five to ten range.AARONCan we get a good csv going? Do you have access to. You don't have to do this right now.MATTWait, yeah, let me grab that.AARONI want to get, like, aerospace engineering grade cpus going to analyze the causal interactions here based on, I don't know, a few kilobytes of data. It's a baby laptop.MATTYeah, this is what the charts looked like. So it's basically like there was some increase in the first. We raised, like, a couple of grand in the first couple of days. Then, yeah, we raised close to ten grand over the course of the quality thing, and then there was basically flat for a week, and then we raised another ten grand right at the end.AARONThat's cool. Good job, guys.MATTAnd I was very surprised by this.AARONMaybe I didn't really internalize that or something. Maybe I was sort of checked out at that point. Sorry.MATTI guess. No, you were on vacation because when you were coming back from vacation, it's when you did, like, the fake Sama.AARONYeah, that was on the plane.LAURAOkay, yeah, I remember this. My mom got there the next day. I'm like, I'm checking out, not doing anything.AARONYeah, whatever. I'll get rstudio revving later. Actually, I'm gradually turning it into my worst enemy or something like that.MATTWait, how so?AARONI just use Python because it's actually faster and catchy and I don't have to know anything. Also, wait, this is like a rant. This is sort of a totally off topic take, but something I was thinking about. No, actually, I feel like a big question is like, oh, are LLMs going to make it easy for people to do bad things that make it easier for me to do? Maybe not terrible things, but things that are, like, I don't know, I guess of dubious or various things that are mostly in the realm of copyright violation or pirating are not ever enforced, as far as I can tell. But, no, I just couldn't have done a lot of things in the past, but now I can, so that's my anecdote.MATTOkay, I have a whole python.AARONYou can give me a list of YouTube URLs. I guess Google must do, like, a pretty good job of policing how public websites do for YouTube to md three sites, because nothing really just works very well very fast. But you can just do that in python, like, five minutes. But I couldn't do that before, so.MATTI feel like, to me, it's obvious that LLMs make it easier for people to do bad stuff. Exactly as you said because they let make in general make it easier for people to do stuff and they have some protections on this, but those protections are going to be imperfect. I think the much more interesting question in some sense is this like a step change relative to the fact that Google makes it way easier for you to do stuff and including bad stuff and the printing press made it way easier for you to do?AARONI wouldn't even call it a printing press.MATTI like think including bad stuff. So it's like, right, like every invention that generally increases people's capability to do stuff and share information also has these bad effects. And I think the hard question is, are LLMs, wait, did I just x.AARONNo, I don't think, wait, did I just like, hold on. I'm pretty sure it's like still wait, how do I have four things?LAURAWhat is the benefit of LLMs versus.AARONYou can ask it something and it tells you the answer.LAURAI know, but Google does this too.AARONI don't mean, I don't know if I have like a super, I don't think I have any insightful take it just in some sense, maybe these are all not the same, but maybe they're all of similar magnitude, but like object level. Now we live in a world with viruses CRISPR. Honestly, I think to the EA movement's credit, indefinite pause, stop. AI is just not, it's not something that I support. It's not something like most people support, it's not like the official EA position and I think for good reason. But yeah, going back to whatever it was like 1416 or whatever, who knows? If somebody said somebody invented the printing press and somebody else was like, yeah, we should, well I think there's some pretty big dis analysis just because of I guess, biotech in particular, but just like how destructive existing technologies are now. But if somebody had said back then, yeah, let's wait six months and see if we can think of any reason not to release the printing press. I don't think that would have been a terrible thing to do. I don't know, people. I feel like I'm saying something that's going to get coded as pretty extreme. But like x ante hard ex ante. People love thinking, exposed nobody. Like I don't know. I don't actually think that was relevant to anything. Maybe I'm just shit faced right now.MATTOn one shot of vodka.AARON$15 just to have one shot.MATTI'll have a little.AARONYeah. I think is honestly, wait. Yeah, this is actually interesting. Every time I drink I hope that it'll be the time that I discover that I like drinking and it doesn't happen, and I think that this is just because my brain is weird. I don't hate it. I don't feel, like, bad. I don't know. I've used other drugs, which I like. Alcohol just doesn't do it for me. Yeah, screw you, alcohol.MATTYes. And you're now 15.99 cheaper or 50. 99 poorer.AARONYeah, I mean, this will last me a lifetime.MATTYou can use it for, like, cleaning your sink.AARONWait, this has got to be the randomest take of all time. But, yeah, actually, like, isopropyl alcohol, top tier, disinfected. Because you don't have to do anything with it. You leave it there, it evaporates on its own.MATTHonestly. Yeah.AARONI mean, you don't want to be in an enclosed place or whatever. Sorry. To keep. Forget. This is like.MATTNo, I mean, it seems like a good take to me.AARONThat's all.MATTYeah, this is like a very non sequitur.AARONBut what are your guys' favorite cleaning suppliers?MATTOkay, this is kind of bad. Okay, this is not that bad. But I'm, like, a big fan of Clorox wipes.AARONScandalous.MATTI feel like this gets looked down on a little bit because it's like, in theory, I should be using a spray cleaner and sponge more.AARONIf you're like, art porn, what theories do you guys.MATTIf you're very sustainable, very like, you shouldn't just be buying your plastic bucket of Clorox infused wet wipes and you're killing the planet.AARONWhat I thought you were going to say is like, oh, this is like germaphobe coating.MATTNo, I think this is fine. I don't wipe down my groceries with Clorox wipes. This is like, oh, if I need to do my deep clean of the kitchen, what am I going to reach for? I feel like my roommate in college was very much like, oh, I used to be this person. No, I'm saying he was like an anti wet wipe on sustainability reasons person. He was like, oh, you should use a rag and a spray cleaner and wash the rag after, and then you will have not used vast quantities of resources to clean your kitchen.AARONAt one point, I tweeted that I bought regular. Actually, don't do this anymore because it's no longer practical. But I buy regularly about 36 packs of bottled water for like $5 or whatever. And people actually, I think it was like, this is like close to a scissor statement, honestly. Because object level, you know what I am, right. It's not bad. For anything. I'm sorry. It just checks out. But people who are normally pretty technocratic or whatever were kind of like, I don't know, they were like getting heated on.MATTI think this is an amazing scissor statement.AARONYeah.MATTBecause I do.AARONI used to be like, if I were to take my twelve year old self, I would have been incredibly offended, enraged.MATTAnd to be fair, I think in my ideal policy world, there would be a carbon tax that slightly increases the price of that bottled water. Because actually it is kind of wasteful to. There is something, something bad has happened there and you should internalize those.AARONYeah, I think in this particular, I think like thin plastic is just like not. Yeah, I don't think it would raise it like very large amount. I guess.MATTI think this is probably right that even a relatively high carbon tax would not radically change the price.LAURAIt's not just carbon, though. I think because there is land use implicated in this.AARONNo, there's not.LAURAYeah, you're filling up more landfills.AARONYeah, I'm just doing like hearsay right now. Heresy.MATTHearsay. Hearsay is going to be whatever. Well, wait, no, heresy is, if you're arguing against standardly accepted doctrine. Hearsay is like, well, it's both. Then you're just saying shit.AARONI'm doing both right now. Which is that actually landfills are usually like on the outskirts of town. It's like, fine.LAURAThey'Re on the outskirts of town until the town sprawls, and then the elementary school is on a phone.AARONYeah, no, I agree in principle. I don't have a conceptual reason why you're wrong. I just think basically, honestly, the actual heuristic operating here is that I basically outsource what I should pay attention to, to other people. And since I've never seen a less wrong post or gave Warren post about how actually landfills are filling up, it's like, fine, probably.LAURANo, this is me being devil's advocate. I really don't care that about personal waste.MATTYeah, I mean, I think plausibly here, there is, right? So I think object level, the things that matter, when we think about plastic, there is a carbon impact. There is a production impact of like, you need to think about what pollution happened when the oil was drilled and stuff. And then there is like a disposal impact. If you successfully get that bottle into a trash can, for what it's worth.AARONMy bottles are going into their goddamn trash can.MATTIdeally a recycling. No, apparently recycling, I mean, recycling is.AARONWell, I mean, my sense is like apparently recycling. Yeah, I recycle metal. I think I do paper out of convenience.MATTIf you successfully get that bottle handle a waste disposal system that is properly disposing of it, rather than like you're throwing it on a slap, then I think my guess is that the willingness to pay, or if you really crunch the numbers really hard, it would not be once again, a huge cost for the landfill costs. On the flip side, if you throw it in a river, that's very bad. My guess is that it would be right for everyone on Twitter to flame you for buying bottles and throwing them in a river if you did that.AARONWhat is an ed impact on wild animal welfare and equilibrium? No, just kidding. This is something. Yeah, don't worry, guys. No, I was actually the leave no trade coordinator for my Boy scout troop. It's actually kind of ironic because I think probably like a dumb ideology or.LAURAWhatever, it's a public good for the other people around you to not have a bunch of garbage around on that trail.AARONYeah, I do think I went to an overnight training for this. They're very hardcore, but basically conceptually incoherent people. I guess people aren't conceptually incoherent. Their concepts are incoherent who think it

Table of contentsNote: links take you to the corresponding section below; links to the original episode can be found there.* Laura Duffy solves housing, ethics, and more [00:01:16]* Arjun Panickssery solves books, hobbies, and blogging, but fails to solve the Sleeping Beauty problem because he's wrong on that one [00:10:47]* Nathan Barnard on how financial regulation can inform AI regulation [00:17:16]* Winston Oswald-Drummond on the tractability of reducing s-risk, ethics, and more [00:27:48]* Nathan Barnard (again!) on why general intelligence is basically fake [00:34:10]* Daniel Filan on why I'm wrong about ethics (+ Oppenheimer and what names mean in like a hardcore phil of language sense) [00:56:54]* Holly Elmore on AI pause, wild animal welfare, and some cool biology things I couldn't fully follow but maybe you can [01:04:00]* Max Alexander and I solve ethics, philosophy of mind, and cancel culture once and for all [01:24:43]* Sarah Woodhouse on discovering AI x-risk, Twitter, and more [01:30:56] * Pigeon Hour x Consistently Candid pod-crossover: I debate moral realism with Max Alexander and Sarah Hastings-Woodhouse [01:41:08]Intro [00:00:00]To wrap up the year of Pigeon Hour, the podcast, I put together some clips from each episode to create a best-of compilation. This was inspired by 80,000 Hours, a podcast that did the same with their episodes, and I thought it was pretty cool and tractable enough.It's important to note that the clips I chose range in length significantly. This does not represent the quality or amount of interesting content in the episode. Sometimes there was a natural place to break the episode into a five-minute chunk, and other times it wouldn't have made sense to take a five-minute chunk out of what really needed to be a 20-minute segment. I promise I'm not just saying that.So without further ado, please enjoy.#1: Laura Duffy solves housing, ethics, and more [00:01:16]In this first segment, Laura, Duffy, and I discuss the significance and interpretation of Aristotle's philosophical works in relation to modern ethics and virtue theory.AARON: Econ is like more interesting. I don't know. I don't even remember of all the things. I don't know, it seems like kind of cool. Philosophy. Probably would have majored in philosophy if signaling wasn't an issue. Actually, maybe I'm not sure if that's true. Okay. I didn't want to do the old stuff though, so I'm actually not sure. But if I could aristotle it's all wrong. Didn't you say you got a lot out of Nicomachi or however you pronounce that?LAURA: Nicomachian ethics guide to how you should live your life. About ethics as applied to your life because you can't be perfect. Utilitarians. There's no way to be that.AARON: But he wasn't even responding to utilitarianism. I'm sure it was a good work given the time, but like, there's like no other discipline in which we care. So people care so much about like, what people thought 2000 years ago because like the presumption, I think the justified presumption is that things have iterated and improved since then. And I think that's true. It's like not just a presumption.LAURA: Humans are still rather the same and what our needs are for living amongst each other in political society are kind of the same. I think America's founding is very influenced by what people thought 2000 years ago.AARON: Yeah, descriptively that's probably true. But I don't know, it seems like all the whole body of philosophers have they've already done the work of, like, compressing the good stuff. Like the entire academy since like, 1400 or whatever has like, compressed the good stuff and like, gotten rid of the bad stuff. Not in like a high fidelity way, but like a better than chance way. And so the stuff that remains if you just take the state of I don't know if you read the Oxford Handbook of whatever it is, like ethics or something, the takeaways you're going to get from that are just better than the takeaways you're going to get from a summary of the state of the knowledge in any prior year. At least. Unless something weird happened. And I don't know. I don't know if that makes sense.LAURA: I think we're talking about two different things, though. Okay. In terms of knowledge about logic or something or, I don't know, argumentation about trying to derive the correct moral theory or something, versus how should we think about our own lives. I don't see any reason as to why the framework of virtue theory is incorrect and just because it's old. There's many virtue theorists now who are like, oh yeah, they were really on to something and we need to adapt it for the times in which we live and the kind of societies we live in now. But it's still like there was a huge kernel of truth in at least the way of thinking that Aristotle put forth in terms of balancing the different virtues that you care about and trying to find. I think this is true. Right? Like take one virtue of his humor. You don't want to be on one extreme where you're just basically a meme your entire life. Everybody thinks you're funny, but that's just not very serious. But you don't want to be a boar and so you want to find somewhere in the middle where it's like you have a good sense of humor, but you can still function and be respected by other people.AARON: Yeah. Once again, I agree. Well, I don't agree with everything. I agree with a lot of what you just said. I think there was like two main points of either confusion or disagreement. And like, the first one is that I definitely think, no, Aristotle shouldn't be discounted or like his ideas or virtue ethics or anything like that shouldn't be discounted because they were canonical texts or something were written a long time ago. I guess it's just like a presumption that I have a pretty strong presumption that conditional on them being good, they would also be written about today. And so you don't actually need to go back to the founding texts and then in fact, you probably shouldn't because the good stuff will be explained better and not in weird it looks like weird terms. The terms are used differently and they're like translations from Aramaic or whatever. Probably not Aramaic, probably something else. And yeah, I'm not sure if you.LAURA: Agree with this because we have certain assumptions about what words like purpose mean now that we're probably a bit richer in the old conception of them like telos or happiness. Right. Udaimnia is much better concept and to read the original text and see how those different concepts work together is actually quite enriching compared to how do people use these words now. And it would take like I don't know, I think there just is a lot of value of looking at how these were originally conceived because popularizers of the works now or people who are seriously doing philosophy using these concepts. You just don't have the background knowledge that's necessary to understand them fully if you don't read the canonical text.AARON: Yeah, I think that would be true. If you are a native speaker. Do you know Greek? If you know Greek, this is like dumb because then you're just right.LAURA: I did take a quarter of it.AARON: Oh God. Oh my God. I don't know if that counts, but that's like more than anybody should ever take. No, I'm just kidding. That's very cool. No, because I was going to say if you're a native speaker of Greek and you have the connotations of the word eudaimonia and you were like living in the temper shuttle, I would say. Yeah, that's true actually. That's a lot of nuanced, connotation and context that definitely gets lost with translation. But once you take the jump of reading English translations of the texts, not you may as well but there's nothing super special. You're not getting any privileged knowledge from saying the word eudaimonia as opposed to just saying some other term as a reference to that concept or something. You're absorbing the connotation in the context via English, I guess, via the mind of literally the translators who have like.LAURA: Yeah, well see, I tried to learn virtue theory by any other route than reading Aristotle.AARON: Oh God.LAURA: I took a course specifically on Plato and Aristotle.AARON: Sorry, I'm not laughing at you. I'm just like the opposite type of philosophy person.LAURA: But keep going. Fair. But she had us read his physics before we read Nicomachi.AARON: Think he was wrong about all that.LAURA: Stuff, but it made you understand what he meant by his teleology theory so much better in a way that I could not get if I was reading some modern thing.AARON: I don't know, I feel like you probably could. No, sorry, that's not true. I don't think you could get what Aristotle the man truly believed as well via a modern text. But is that what you? Depends. If you're trying to be a scholar of Aristotle, maybe that's important. If you're trying to find the best or truest ethics and learn the lessons of how to live, that's like a different type of task. I don't think Aristotle the man should be all that privileged in that.LAURA: If all of the modern people who are talking about virtue theory are basically Aristotle, then I don't see the difference.AARON: Oh, yeah, I guess. Fair enough. And then I would say, like, oh, well, they should probably start. Is that in fact the state of the things in virtue theory? I don't even know.LAURA: I don't know either.#2 Arjun Panickssery solves books, hobbies, and blogging, but fails to solve the Sleeping Beauty problem because he's wrong on that one [00:10:47]All right, next, Arjun Panixery and I explore the effectiveness of reading books in retaining and incorporating knowledge, discussing the value of long form content and the impact of great literary works on understanding and shaping personal worldviews.ARJUN: Oh, you were in the book chat, though. The book rant group chat, right?AARON: Yeah, I think I might have just not read any of it. So do you want to fill me in on what I should have read?ARJUN: Yeah, it's group chat of a bunch of people where we were arguing about a bunch of claims related to books. One of them is that most people don't remember pretty much anything from books that they read, right? They read a book and then, like, a few months later, if you ask them about it, they'll just say one page's worth of information or maybe like, a few paragraphs. The other is that what is it exactly? It's that if you read a lot of books, it could be that you just incorporate the information that's important into your existing models and then just forget the information. So it's actually fine. Isn't this what you wrote in your blog post or whatever? I think that's why I added you to that.AARON: Oh, thank you. I'm sorry I'm such a bad group chat participant. Yeah, honestly, I wrote that a while ago. I don't fully remember exactly what it says, but at least one of the things that it said was and that I still basically stand by, is that it's basically just like it's increasing the salience of a set of ideas more so than just filling your brain with more facts. And I think this is probably true insofar as the facts support a set of common themes or ideas that are kind of like the intellectual core of it. It would be really hard. Okay, so this is not a book, but okay. I've talked about how much I love an 80,000 hours podcast, and I've listened to, I don't think every episode, but at least 100 of the episodes. And no, you're just, like, not going to definitely I've forgotten most of the actual almost all of the actual propositional pieces of information said, but you're just not going to convince me that it's completely not affecting either model of the world or stuff that I know or whatever. I mean, there are facts that I could list. I think maybe I should try.ARJUN: Sure.AARON: Yeah. So what's your take on book other long form?ARJUN: Oh, I don't know. I'm still quite confused or I think the impetus for the group chat's creation was actually Hanania's post where he wrote the case against most books or most was in parentheses or something. I mean, there's a lot of things going on in that post. He just goes off against a bunch of different categories of books that are sort of not closely related. Like, he goes off against great. I mean, this is not the exact take he gives, but it's something like the books that are considered great are considered great literature for some sort of contingent reason, not because they're the best at getting you information that you want.AARON: This is, like, another topic. But I'm, like, anti great books. In fact, I'm anti great usually just means old and famous. So insofar as that's what we mean by I'm like, I think this is a bad thing, or, like, I don't know, aristotle is basically wrong about everything and stuff like that.ARJUN: Right, yeah. Wait, we could return to this. I guess this could also be divided into its component categories. He spends more time, though, I think, attacking a certain kind of nonfiction book that he describes as the kind of book that somebody pitches to a publisher and basically expands a single essay's worth of content into with a bunch of anecdotes and stuff. He's like, most of these books are just not very useful to read, I guess. I agree with that.AARON: Yeah. Is there one that comes to mind as, like, an? Mean, I think of Malcolm Gladwell as, like, the kind of I haven't actually read any of his stuff in a while, but I did, I think, when I started reading nonfiction or with any sort of intent, I read. A bunch of his stuff or whatever and vaguely remember that this is basically what he like for better or.ARJUN: Um yeah, I guess so. But he's almost, like, trying to do it on purpose. This is the experience that you're getting by reading a Malcolm Gladwell book. It's like talib. Right? It's just him just ranting. I'm thinking, I guess, of books that are about something. So, like, if you have a book that's know negotiation or something, it'll be filled with a bunch of anecdotes that are of dubious usefulness. Or if you get a book that's just about some sort of topic, there'll be historical trivia that's irrelevant. Maybe I can think of an example.AARON: Yeah. So the last thing I tried to read, maybe I am but haven't in a couple of weeks or whatever, is like, the Derek Parfit biography. And part of this is motivated because I don't even like biographies in general for some reason, I don't know. But I don't know. He's, like, an important guy. Some of the anecdotes that I heard were shockingly close to home for me, or not close to home, but close to my brain or something. So I was like, okay, maybe I'll see if this guy's like the smarter version of Aaron Bergman. And it's not totally true.ARJUN: Sure, I haven't read the book, but I saw tweet threads about it, as one does, and I saw things that are obviously false. Right. It's the claims that he read, like, a certain number of pages while brushing his teeth. That's, like, anatomically impossible or whatever. Did you get to that part? Or I assumed no, I also saw.AARON: That tweet and this is not something that I do, but I don't know if it's anatomically impossible. Yeah, it takes a little bit of effort to figure out how to do that, I guess. I don't think that's necessarily false or whatever, but this is probably not the most important.ARJUN: Maybe it takes long time to brush his teeth.#3: Nathan Barnard on how financial regulation can inform AI regulation [00:17:16]In this next segment, Nathan Barnard and I dive into the complexities of AI regulation, including potential challenges and outcomes of governing AI in relation to economic growth and existential security. And we compare it to banking regulation as well.AARON: Yeah, I don't know. I just get gloomy for, I think justified reasons when people talk about, oh yeah, here's the nine step process that has to take place and then maybe there's like a 20% chance that we'll be able to regulate AI effectively. I'm being facetious or exaggerating, something like that, but not by a gigantic amount.NATHAN: I think this is pretty radically different to my mainline expectation.AARON: What's your mainline expectation?NATHAN: I suppose I expect like AI to come with an increasing importance past economy and to come up to really like a very large fraction of the economy before really crazy stuff starts happening and this world is going very anonymous. Anonymous, anonymous, anonymous. I know the word is it'd be very unusual if this extremely large sector economy which was impacted like a very large number of people's lives remains like broadly unregulated.AARON: It'll be regulated, but just maybe in a stupid way.NATHAN: Sure, yes, maybe in a stupid way. I suppose critically, do you expect the stupid way to be like too conservative or too like the specific question of AI accenture it's basically too conservative or too lenient or I just won't be able to interact with this.AARON: I guess generally too lenient, but also mostly on a different axis where just like I don't actually know enough. I don't feel like I've read learned about various governance proposals to have a good object level take on this. But my broad prior is that there are just a lot of ways to for anything. There's a lot of ways to regulate something poorly. And the reason insofar as anything isn't regulated poorly it's because of a lot of trial and error.NATHAN: Maybe.AARON: I mean, there's probably exceptions, right? I don't know. Tax Americana is like maybe we didn't just kept winning wars starting with World War II. I guess just like maybe like a counterexample or something like that.NATHAN: Yeah, I think I still mostly disagree with this. Oh, cool. Yeah. I suppose I see a much like broader spectrum between bad regulation and good regulation. I agree it's like very small amount. The space of optimal regulation is very small. But I think we have to hit that space for regulation to be helpful. Especially in this especially if you consider that if you sort of buy the AI extension safety risk then the downsides of it's not this quite fine balancing act between too much whether consumer protection and siphoning competition and cycling innovation too much. It's like trying to end this quite specific, very bad outcome which is maybe much worse than going somewhat slowering economic growth, at least somewhat particularly if we think we're going to get something. This is very explosive rates for economic growth really quite soon. And the cost of slowing down economic growth by weather even by quite a large percentage, very small compared to the cost of sort of an accidental catastrophe. I sort of think of Sony iconic growth as the main cost of main way regulation goes wrong currently.AARON: I think in an actual sense that is correct. There's the question of like okay, Congress in the states like it's better than nothing. I'm glad it's not anarchy in terms of like I'm glad we have a legislature.NATHAN: I'm also glad the United States.AARON: How reasons responsive is Congress? I don't think reasons responsive enough to make it so that the first big law that gets passed insofar as there is one or if there is one is on the pareto frontier trading off between economic growth and existential security. It's going to be way inside of that production frontier or whatever. It's going to suck on every action, maybe not every act but at least like some relevant actions.NATHAN: Yeah that doesn't seem like obviously true to me. I think Dodge Frank was quite a good law.AARON: That came after 2008, right?NATHAN: Yeah correct. Yeah there you go. No, I agree. I'm not especially confident about doing regulation before there's some quite bad before there's a quite bad warning shot and yes, if we're in world where we have no warning shots and we're just like blindsided by everyone getting turned into everyone getting stripped their Athens within 3 seconds, this is not good. Both in law we do have one of those shots and I think Glass Seagull is good law. Not good law is a technical term. I think Glass Steagall was a good piece of legislation. I think DoD Frank was a good piece of legislation. I think the 2008 Seamless Bill was good piece of legislation. I think the Troubled Assets Relief Program is a good piece of piece of legislation.AARON: I recognize these terms and I know some of them and others I do not know the contents of.NATHAN: Yeah so Glass Eagle was the financial regulation passed in 1933 after Great Depression. The Tropical Asset Relief Program was passed in I think 2008, moved 2009 to help recapitalize banks. Dodge Frank was the sort of landmark post financial cris piece of legislation passed in 2011. I think these are all good pieces of legislation now. I think like financial regulation is probably unusually good amongst US legislation. This is like a quite weak take, I guess. It's unusually.AARON: So. I don't actually know the pre depression financial history at all but I feel like the more relevant comparison to the 21st century era is what was the regulatory regime in 1925 or something? I just don't know.NATHAN: Yeah, I know a bit. I haven't read this stuff especially deeply and so I don't want to don't want to be so overcompensant here but sort of the core pieces which were sort of important for the sort of the Great Depression going very badly was yeah, no distinction between commercial banks and investment banks. Yes, such a bank could take much riskier. Much riskier. Things with like custom deposits than they could from 1933 until the Peel Glass Eagle. And combine that with no deposit insurance and if you sort of have the combination of banks being able to do quite risky things with depositors money and no deposit insurance, this is quite dangerously known. And glassy repeal.AARON: I'm an expert in the sense that I have the Wikipedia page up. Well, yeah, there was a bunch of things. Basically. There's the first bank of the United States. There's the second bank of the United States. There's the free banking era. There was the era of national banks. Yada, yada, yada. It looks like 19. Seven was there was some panic. I vaguely remember this from like, AP US history, like seven years ago or.NATHAN: Yes, I suppose in short, I sort of agree that the record of sort of non post Cris legislation is like, not very good, but I think record of post Cris legislation really, at least in the financial sector, really is quite good. I'm sure lots of people disagree with this, but this is my take.#4 Winston Oswald-Drummond on the tractability of reducing s-risk, ethics, and more [00:27:48]Up next, Winston Oswald Drummond and I talk about the effectiveness and impact of donating to various research organizations, such as suffering-focused S-risk organizations. We discuss tractability, expected value, and essentially where we should give our money.AARON: Okay, nice. Yeah. Where to go from here? I feel like largely we're on the same page, I feel like.WINSTON: Yeah. Is your disagreement mostly tractability? Then? Maybe we should get into the disagreement.AARON: Yeah. I don't even know if I've specified, but insofar as I have one, yes, it's trapped ability. This is the reason why I haven't donated very much to anywhere for money reasons. But insofar as I have, I have not donated to Clrcrs because I don't see a theory of change that connects the research currently being done to actually reducing s risks. And I feel like there must be something because there's a lot of extremely smart people at both of these orgs or whatever, and clearly they thought about this and maybe the answer is it's very general and the outcome is just so big in magnitude that anything kind.WINSTON: Of that is part of it, I think. Yeah, part of it is like an expected value thing and also it's just very neglected. So it's like you want some people working on this, I think, at least. Even if it's unlikely to work. Yeah, even that might be underselling it, though. I mean, I do think there's people at CRS and Clr, like talking to people at AI labs and some people in politics and these types of things. And hopefully the research is a way to know what to try to get done at these places. You want to have some concrete recommendations and I think obviously people have to also be willing to listen to you, but I think there is some work being done on that and research is partially just like a community building thing as well. It's a credible signal that you were smart and have thought about this, and so it gives people reason to listen to you and maybe that mostly pays off later on in the future.AARON: Yeah, that all sounds like reasonable. And I guess one thing is that I just don't there's definitely things I mean, first of all, I haven't really stayed up to date on what's going on, so I haven't even done I've done zero research for this podcast episode, for example. Very responsible and insofar as I've know things about these. Orgs. It's just based on what's on their website at some given time. So insofar as there's outreach going on, not like behind the scenes, but just not in a super public way, or I guess you could call that behind the scenes. I just don't have reason to, I guess, know about that. And I guess, yeah, I'm pretty comfortable. I don't even know if this is considered biting a bullet for the crowd that will be listening to this, if that's anybody but with just like yeah, saying a very small change for a very large magnitude, just, like, checks out. You can just do expected value reasoning and that's basically correct, like a correct way of thinking about ethics. But even I don't know how much you know specifically or, like, how much you're allowed want to reveal, but if there was a particular alignment agenda that I guess you in a broad sense, like the suffering focused research community thought was particularly promising and relative to other tractable, I guess, generic alignment recommendations. And you were doing research on that and trying to push that into the alignment mainstream, which is not very mainstream. And then with the hope that that jumps into the AI mainstream. Even if that's kind of a long chain of events. I think I would be a lot more enthusiastic about I don't know that type of agenda, because it feels like there's like a particular story you're telling where it cashes out in the end. You know what I mean?WINSTON: Yeah, I'm not the expert on this stuff, but I do think you just mean I think there's some things about influencing alignment and powerful AI for sure. Maybe not like a full on, like, this is our alignment proposal and it also handles Sris. But some things we could ask AI labs that are already building, like AGI, we could say, can you also implement these sort of, like, safeguards so if you failed alignment, you fail sort of gracefully and don't cause lots of suffering.AARON: Right?WINSTON: Yeah. Or maybe there are other things too, which also seem potentially more tractable. Even if you solve alignment in some sense, like aligning with whatever the human operator tells the AI to do, then you can also get the issue that malevolent actors can take control of the AI and then what they want also causes lots of suffering that type of alignment wouldn't. Yeah, and I guess I tend to be somewhat skeptical of coherent extrapolated volition and things like this, where the idea is sort of like it'll just figure out our values and do the right thing. So, yeah, there's some ways to push on this without having a full alignment plan, but I'm not sure if that counts as what you were saying.AARON: No, I guess it does. Yeah, it sounds like it does. And it could be that I'm just kind of mistaken about the degree to which that type of research and outreach is going on. That sounds like it's at least partially true.#5: Nathan Barnard (again!) on why general intelligence is basically fake [00:34:10]Up next, Nathan Barnard is back for his second episode. And we talked about the nature of general intelligence, its relationship with language and the implications of specialized brain functions on the understanding of human cognitive abilities.NATHAN: Yes. This like symbolic like symbolic, symbolic reasoning stuff. Yeah. So I think if I was, like, making the if I was, like, making the case for general intelligence being real, I wouldn't have symbolic reasoning, but I would have language stuff. I'd have this hierarchical structure thing, which.AARON: I would probably so I think of at least most uses of language and central examples as a type of symbolic reasoning because words mean things. They're like yeah. Pointers to objects or something like that.NATHAN: Yeah, I think it's like, pretty confidence isn't where this isn't a good enough description of general intelligence. So, for instance so if you bit in your brain called, I'm using a checklist, I don't fuck this up vernacular, I'm not making this cool. Lots of connects to use words like pointers as these arbitrary signs happens mostly in this area of the brain called Berkeley's area. But very famously, you can have Berkeley's epaxics who lose the ability to do language comprehension and use the ability to consistently use words as pointers, as signs to point to things, but still have perfect good spatial reasoning abilities. And so, conversely, people with brokers of fascia who fuck up, who have the broker's reason their brain fucks up will not be able to form fluent sentences and have some problems like unsigned syntax, and they'll still be able to have very good spatial reasoning. It could still, for instance, be like, good engineers. Would you like many problems which, like, cost engineering?AARON: Yeah, I totally buy that. I don't think language is the central thing. I think it's like an outgrowth of, like I don't know, there's like a simplified model I could make, which is like it's like an outgrowth of whatever general intelligence really is. But whatever the best spatial or graphical model is, I don't think language is cognition.NATHAN: Yes, this is a really big debate in psycholinguistics as to whether language is like an outgrowth of other abilities like the brain has, whether language whether there's very specialized language modules. Yeah, this is just like a very live debate in psycholinguistics moments. I actually do lean towards the reason I've been talking about this actually just going to explain this hierarchical structure thing? Yeah, I keep talking about it. So one theory for how you can comprehend new sentences, like, the dominant theory in linguistics, how you can comprehend new sentences, um, is you break them up into, like you break them up into, like, chunks, and you form these chunks together in this, like, tree structure. So something like, if you hear, like, a totally novel sentence like the pit bull mastiff flopped around deliciously or something, you can comprehend what the sentence means despite the fact you've never heard it. Theory behind this is you saw yes, this can be broken up into this tree structure, where the different, like, ah, like like bits of the sentence. So, like like the mastiff would be like, one bit, and then you have, like, another bit, which is like, the mastiff I can't remember I said rolled around, so that'd be like, another bit, and then you'd have connectors to our heart.AARON: Okay.NATHAN: So the massive rolling around one theory of one of the sort of distinctive things that humans have disabilities is like, this quite general ability to break things up into these these tree structures. This is controversial within psycholinguistics, but it's broadly an area which I broadly buy it because we do see harms to other areas of intelligence. You get much worse at, like, Ravens Progressive Matrices, for instance, when you have, like, an injury to brokers area, but, like, not worse at, like, tests like tests of space, of, like, spatial reasoning, for instance.AARON: So what is like, is there, like, a main alternative to, like, how humans.NATHAN: Understand language as far as this specificity of how we pass completely novel sentences, as far as where this is just like this is just like the the academic consensus. Okay.AARON: I mean, it sounds totally like right? I don't know.NATHAN: Yeah. But yeah, I suppose going back to saying, how far is language like an outgrowth of general intelligence? An outgrowth like general intelligence versus having much more specialized language modules? Yeah, I lean towards the latter, despite yeah, I still don't want to give too strong of a personal opinion here because I'm not a linguistic this is a podcast.AARON: You're allowed to give takes. No one's going to say this is like the academic we want takes.NATHAN: We want takes. Well, gone to my head is.AARON: I.NATHAN: Think language is not growth of other abilities. I think the main justification for this, I think, is that the loss of other abilities we see when you have damage to broker's area and verca's area.AARON: Okay, cool. So I think we basically agree on that. And also, I guess one thing to highlight is I think outgrowth can mean a couple of different things. I definitely think it's plausible. I haven't read about this. I think I did at some point, but not in a while. But outgrowth could mean temporarily or whatever. I think I'm kind of inclined to think it's not that straightforward. You could have coevolution where language per se encourages both its own development and the development of some general underlying trait or something.NATHAN: Yeah. Which seems likely.AARON: Okay, cool. So why don't humans have general intelligence?NATHAN: Right. Yeah. As I was sort of talking about previously.AARON: Okay.NATHAN: I think I think I'd like to use go back to like a high level like a high level argument is there appears to be very surprised, like, much higher levels of functional specialization in brains than you expect. You can lose much more specific abilities than you expect to be able to lose. You can lose specifically the ability a famous example is like facebindness, actually. You probably lose the ability to specifically recognize things which you're, like, an expert in.AARON: Who does it or who loses this ability.NATHAN: If you've damaged your fuse inform area, you'll lose the ability to recognize faces, but nothing else.AARON: Okay.NATHAN: And there's this general pattern that your brain is much more you can lose much more specific abilities than you expect. So, for instance, if you sort of have damage to your ventral, medial, prefrontal cortex, you can say the reasoning for why you shouldn't compulsively gamble but still compulsively gamble.AARON: For instance okay, I understand this not gambling per se, but like executive function stuff at a visceral level. Okay, keep going.NATHAN: Yeah. Some other nice examples of this. I think memory is quite intuitive. So there's like, a very famous patient called patient HM who had his hippocampus removed and so as a result, lost all declarative memory. So all memory of specific facts and things which happened in his life. He just couldn't remember any of these things, but still perfectly functioning otherwise. I think at a really high level, I think this functional specialization is probably the strongest piece of evidence against the general intelligence hypothesis. I think fundamentally, general intelligence hypothesis implies that, like, if you, like yeah, if you was, like, harm a piece of your brain, if you have some brain injury, you might like generically get worse at tasks you like, generically get worse at, like at like all task groups use general intelligence. But I think suggesting people, including general intelligence, like the ability to write, the ability to speak, maybe not speak, the ability to do math, you do have.AARON: This it's just not as easy to analyze in a Cogsy paper which IQ or whatever. So there is something where if somebody has a particular cubic centimeter of their brain taken out, that's really excellent evidence about what that cubic centimeter does or whatever, but that non spatial modification is just harder to study and analyze. I guess we'll give people drugs, right? Suppose that set aside the psychometric stuff. But suppose that general intelligence is mostly a thing or whatever and you actually can ratchet it up and down. This is probably just true, right? You can probably give somebody different doses of, like, various drugs. I don't know, like laughing gas, like like, yeah, like probably, probably weed. Like I don't know.NATHAN: So I think this just probably isn't true. Your working memory corrects quite strongly with G and having better working memory generic can make you much better at lots of tasks if you have like.AARON: Yeah.NATHAN: Sorry, but this is just like a specific ability. It's like just specifically your working memory, which is improved if you go memory to a drugs. Improved working memory. I think it's like a few things like memory attention, maybe something like decision making, which are all like extremely useful abilities and improve how well other cognitive abilities work. But they're all separate things. If you improved your attention abilities, your working memory, but you sort of had some brain injury, which sort of meant you sort of had lost ability to pass syntax, you would not get better at passing syntax. And you can also use things separately. You can also improve attention and improve working memory separately, which just it's not just this one dial which you can turn up.AARON: There's good reason to expect that we can't turn it up because evolution is already sort of like maximizing, given the relevant constraints. Right. So you would need to be looking just like injuries. Maybe there are studies where they try to increase people's, they try to add a cubic centimeter to someone's brain, but normally it's like the opposite. You start from some high baseline and then see what faculties you lose. Just to clarify, I guess.NATHAN: Yeah, sorry, I think I've lost the you still think there probably is some general intelligence ability to turn up?AARON: Honestly, I think I haven't thought about this nearly as much as you. I kind of don't know what I think at some level. If I could just write down all of the different components and there are like 74 of them and what I think of a general intelligence consists of does that make it I guess in some sense, yeah, that does make it less of an ontologically legit thing or something. I think I think the thing I want to get the motivating thing here is that with humans yet you can like we know humans range in IQ, and there's, like, setting aside a very tiny subset of people with severe brain injuries or development disorders or whatever. Almost everybody has some sort of symbolic reasoning that they can do to some degree. Whereas the smartest maybe I'm wrong about this, but as far as I know, the smartest squirrel is not going to be able to have something semantically represent something else. And that's what I intuitively want to appeal to, you know what I mean?NATHAN: Yeah, I know what you're guessing at. So I think there's like two interesting things here. So I think one is, could a squirrel do this? I'm guessing a squirrel couldn't do this, but a dog can, or like a dog probably can. A chimpanzee definitely can.AARON: Do what?NATHAN: Chimpanzees can definitely learn to associate arbitrary signs, things in the world with arbitrary signs.AARON: Yes, but maybe I'm just adding on epicentercles here, but I feel like correct me if I'm wrong, but I think that maybe I'm just wrong about this, but I would assume that Chicken Tees cannot use that sign in a domain that is qualitatively different from the ones they've been in. Right. So, like, a dog will know that a certain sign means sit or whatever, but maybe that's not a good I.NATHAN: Don'T know think this is basically not true.AARON: Okay.NATHAN: And we sort of know this from teaching.AARON: Teaching.NATHAN: There's like a famously cocoa de guerrilla. Also a bonobo whose name I can't remember were taught sign language. And the thing they were consistently bad at was, like, putting together sentences they could learn quite large vocabularies learning to associate by large, I mean in the hundreds of words, in the low hundreds of words which they could consistently use consistently use correctly.AARON: What do you mean by, like, in what sense? What is bonobo using?NATHAN: A very famous and quite controversial example is like, coco gorilla was like, saw a swan outside and signed water bird. That's like, a controversial example. But other things, I think, which are controversial here is like, the syntax part of putting water and bird together is the controversial part, but it's not the controversial part that she could see a swan and call that a bird.AARON: Yeah, I mean, this is kind of just making me think, okay, maybe the threshold for D is just like at the chimp level or something. We are like or whatever the most like that. Sure. If a species really can generate from a prefix and a suffix or whatever, a concept that they hadn't learned before.NATHAN: Yeah, this is a controversial this is like a controversial example of that the addition to is the controversial part. Yeah, I suppose maybe brings back to why I think this matters is will there be this threshold which AIS cross such that their reasoning after this is qualitatively different to their reasoning previously? And this is like two things. One, like a much faster increase in AI capabilities and two, alignment techniques which worked on systems which didn't have g will no longer work. Systems which do have g. Brings back to why I think this actually matters. But I think if we're sort of accepting it, I think elephants probably also if you think that if we're saying, like, g is like a level of chimpanzees, chimpanzees just, like, don't don't look like quantitatively different to, like, don't look like that qualitatively different to, like, other animals. Now, lots of other animals live in similar complex social groups. Lots of other animals use tools.AARON: Yeah, sure. For one thing, I don't think there's not going to be a discontinuity in the same way that there wasn't a discontinuity at any point between humans evolution from the first prokaryotic cells or whatever are eukaryotic one of those two or both, I guess. My train of thought. Yes, I know it's controversial, but let's just suppose that the sign language thing was legit with the waterbird and that's not like a random one off fluke or something. Then maybe this is just some sort of weird vestigial evolutionary accident that actually isn't very beneficial for chimpanzees and they just stumbled their way into and then it just enabled them to it enables evolution to bootstrap Shimp genomes into human genomes. Because at some the smartest or whatever actually, I don't know. Honestly, I don't have a great grasp of evolutionary biology or evolution at all. But, yeah, it could just be not that helpful for chimps and helpful for an extremely smart chimp that looks kind of different or something like that.NATHAN: Yeah. So I suppose just like the other thing she's going on here, I don't want to keep banging on about this, but you can lose the language. You can lose linguistic ability. And it's just, like, happens this happens in stroke victims, for instance. It's not that rare. Just, like, lose linguistic ability, but still have all the other abilities which we sort of think of as like, general intelligence, which I think would be including the general intelligence, like, hypothesis.AARON: I agree that's, like, evidence against it. I just don't think it's very strong evidence, partially because I think there is a real school of thought that says that language is fundamental. Like, language drives thought. Language is, like, primary to thought or something. And I don't buy that. If you did buy that, I think this would be, like, more damning evidence.#6 Daniel Filan on why I'm wrong about ethics (+ Oppenheimer and what names mean in like a hardcore phil of language sense) [00:56:54][Note: I forgot to record an intro segment here. Sorry!]AARON: Yeah. Yes. I'm also anti scam. Right, thank you. Okay, so I think that thing that we were talking about last time we talked, which is like the thing I think we actually both know stuff about instead of just like, repeating New York Times articles is my nuanced ethics takes and why you think about talk about that and then we can just also branch off from there.DANIEL: Yeah, we can talk about that.AARON: Maybe see where that did. I luckily I have a split screen up, so I can pull up things. Maybe this is kind of like egotistical or something to center my particular view, but you've definitely given me some of the better pushback or whatever that I haven't gotten that much feedback of any kind, I guess, but it's still interesting to hear your take. So basically my ethical position or the thing that I think is true is that which I think is not the default view. I think most people think this is wrong is that total utilitarianism does not imply that for some amount of suffering that could be created there exists some other extremely large arbitrarily, large amount of happiness that could also be created which would morally justify the former. Basically.DANIEL: So you think that even under total utilitarianism there can be big amounts of suffering such that there's no way to morally tip the calculus. However much pleasure you can create, it's just not going to outweigh the fact that you inflicted that much suffering on some people.AARON: Yeah, and I'd highlight the word inflicted if something's already there and you can't do anything about it, that's kind of neither here nor there as it pertains to your actions or something. So it's really about you increasing, you creating suffering that wouldn't have otherwise been created. Yeah. It's also been a couple of months since I've thought about this in extreme detail, although I thought about it quite a bit. Yeah.DANIEL: Maybe I should say my contrary view, I guess, when you say that, I don't know, does total utilitarianism imply something or not? I'm like, well, presumably it depends on what we mean by total utilitarianism. Right. So setting that aside, I think that thesis is probably false. I think that yeah. You can offset great amounts of suffering with great amounts of pleasure, even for arbitrary amounts of suffering.AARON: Okay. I do think that position is like the much more common and even, I'd say default view. Do you agree with that? It's sort of like the implicit position of people who are of self described total utilitarians who haven't thought a ton about this particular question.DANIEL: Yeah, I think it's probably the implicit default. I think it's the implicit default in ethical theory or something. I think that in practice, when you're being a utilitarian, I don't know, normally, if you're trying to be a utilitarian and you see yourself inflicting a large amount of suffering, I don't know. I do think there's some instinct to be like, is there any way we can get around this?AARON: Yeah, for sure. And to be clear, I don't think this would look like a thought experiment. I think what it looks like in practice and also I will throw in caveats as I see necessary, but I think what it looks like in practice is like, spreading either wild animals or humans or even sentient digital life through the universe. That's in a non as risky way, but that's still just maybe like, say, making the earth, making multiple copies of humanity or something like that. That would be an example that's probably not like an example of what an example of creating suffering would be. For example, just creating another duplicate of earth. Okay.DANIEL: Anything that would be like so much suffering that we shouldn't even the pleasures of earth outweighs.AARON: Not necessarily, which is kind of a cop out. But my inclination is that if you include wild animals, the answer is yes, that creating another earth especially. Yeah, but I'm much more committed to some amount. It's like some amount than this particular time and place in human industry is like that or whatever.DANIEL: Okay, can I get a feel of some other concrete cases to see?AARON: Yeah.DANIEL: So one example that's on my mind is, like, the atomic bombing of Hiroshima and Nagasaki, right? So the standard case for this is, like, yeah, what? A hundred OD thousand people died? Like, quite terrible, quite awful. And a lot of them died, I guess a lot of them were sort of some people were sort of instantly vaporized, but a lot of people died in extremely painful ways. But the countercase is like, well, the alternative to that would have been like, an incredibly grueling land invasion of Japan, where many more people would have died or know regardless of what the actual alternatives were. If you think about the atomic bombings, do you think that's like the kind of infliction of suffering where there's just not an offsetting amount of pleasure that could make that okay?AARON: My intuition is no, that it is offsettable, but I would also emphasize that given the actual historical contingencies, the alternative, the implicit case for the bombing includes reducing suffering elsewhere rather than merely creating happiness. There can definitely be two bad choices that you have to make or something. And my claim doesn't really pertain to that, at least not directly.#7: Holly Elmore on AI pause, wild animal welfare, and some cool biology things I couldn't fully follow but maybe you can [01:04:00]Up next, Holly Elmore and I discuss the complexities and implications of AI development and open sourcing. We talk about protests and ethical considerations around her, um, uh, campaign to pause the development of frontier AI systems until, until we can tell that they're safe.AARON: So what's the plan? Do you have a plan? You don't have to have a plan. I don't have plans very much.HOLLY: Well, right now I'm hopeful about the UK AI summit. Pause AI and I have planned a multi city protest on the 21 October to encourage the UK AI Safety Summit to focus on safety first and to have as a topic arranging a pause or that of negotiation. There's a lot of a little bit upsetting advertising for that thing that's like, we need to keep up capabilities too. And I just think that's really a secondary objective. And that's how I wanted to be focused on safety. So I'm hopeful about the level of global coordination that we're already seeing. It's going so much faster than we thought. Already the UN Secretary General has been talking about this and there have been meetings about this. It's happened so much faster at the beginning of this year. Nobody thought we could talk about nobody was thinking we'd be talking about this as a mainstream topic. And then actually governments have been very receptive anyway. So right now I'm focused on other than just influencing opinion, the targets I'm focused on, or things like encouraging these international like, I have a protest on Friday, my first protest that I'm leading and kind of nervous that's against Meta. It's at the Meta building in San Francisco about their sharing of model weights. They call it open source. It's like not exactly open source, but I'm probably not going to repeat that message because it's pretty complicated to explain. I really love the pause message because it's just so hard to misinterpret and it conveys pretty clearly what we want very quickly. And you don't have a lot of bandwidth and advocacy. You write a lot of materials for a protest, but mostly what people see is the title.AARON: That's interesting because I sort of have the opposite sense. I agree that in terms of how many informational bits you're conveying in a particular phrase, pause AI is simpler, but in some sense it's not nearly as obvious. At least maybe I'm more of a tech brain person or whatever. But why that is good, as opposed to don't give extremely powerful thing to the worst people in the world. That's like a longer everyone.HOLLY: Maybe I'm just weird. I've gotten the feedback from open source ML people is the number one thing is like, it's too late, there's already super powerful models. There's nothing you can do to stop us, which sounds so villainous, I don't know if that's what they mean. Well, actually the number one message is you're stupid, you're not an ML engineer. Which like, okay, number two is like, it's too late, there's nothing you can do. There's all of these other and Meta is not even the most powerful generator of models that it share of open source models. I was like, okay, fine. And I don't know, I don't think that protesting too much is really the best in these situations. I just mostly kind of let that lie. I could give my theory of change on this and why I'm focusing on Meta. Meta is a large company I'm hoping to have influence on. There is a Meta building in San Francisco near where yeah, Meta is the biggest company that is doing this and I think there should be a norm against model weight sharing. I was hoping it would be something that other employees of other labs would be comfortable attending and that is a policy that is not shared across the labs. Obviously the biggest labs don't do it. So OpenAI is called OpenAI but very quickly decided not to do that. Yeah, I kind of wanted to start in a way that made it more clear than pause AI. Does that anybody's welcome something? I thought a one off issue like this that a lot of people could agree and form a coalition around would be good. A lot of people think that this is like a lot of the open source ML people think know this is like a secret. What I'm saying is secretly an argument for tyranny. I just want centralization of power. I just think that there are elites that are better qualified to run everything. It was even suggested I didn't mention China. It even suggested that I was racist because I didn't think that foreign people could make better AIS than Meta.AARON: I'm grimacing here. The intellectual disagreeableness, if that's an appropriate term or something like that. Good on you for standing up to some pretty bad arguments.HOLLY: Yeah, it's not like that worth it. I'm lucky that I truly am curious about what people think about stuff like that. I just find it really interesting. I spent way too much time understanding the alt. Right. For instance, I'm kind of like sure I'm on list somewhere because of the forums I was on just because I was interested and it is something that serves me well with my adversaries. I've enjoyed some conversations with people where I kind of like because my position on all this is that look, I need to be convinced and the public needs to be convinced that this is safe before we go ahead. So I kind of like not having to be the smart person making the arguments. I kind of like being like, can you explain like I'm five. I still don't get it. How does this work?AARON: Yeah, no, I was thinking actually not long ago about open source. Like the phrase has such a positive connotation and in a lot of contexts it really is good. I don't know. I'm glad that random tech I don't know, things from 2004 or whatever, like the reddit source code is like all right, seems cool that it's open source. I don't actually know if that was how that right. But yeah, I feel like maybe even just breaking down what the positive connotation comes from and why it's in people's self. This is really what I was thinking about, is like, why is it in people's self interest to open source things that they made and that might break apart the allure or sort of ethical halo that it has around it? And I was thinking it probably has something to do with, oh, this is like how if you're a tech person who makes some cool product, you could try to put a gate around it by keeping it closed source and maybe trying to get intellectual property or something. But probably you're extremely talented already, or pretty wealthy. Definitely can be hired in the future. And if you're not wealthy yet I don't mean to put things in just materialist terms, but basically it could easily be just like in a yeah, I think I'll probably take that bit out because I didn't mean to put it in strictly like monetary terms, but basically it just seems like pretty plausibly in an arbitrary tech person's self interest, broadly construed to, in fact, open source their thing, which is totally fine and normal.HOLLY: I think that's like 99 it's like a way of showing magnanimity showing, but.AARON: I don't make this sound so like, I think 99.9% of human behavior is like this. I'm not saying it's like, oh, it's some secret, terrible self interested thing, but just making it more mechanistic. Okay, it's like it's like a status thing. It's like an advertising thing. It's like, okay, you're not really in need of direct economic rewards, or sort of makes sense to play the long game in some sense, and this is totally normal and fine, but at the end of the day, there's reasons why it makes sense, why it's in people's self interest to open source.HOLLY: Literally, the culture of open source has been able to bully people into, like, oh, it's immoral to keep it for yourself. You have to release those. So it's just, like, set the norms in a lot of ways, I'm not the bully. Sounds bad, but I mean, it's just like there is a lot of pressure. It looks bad if something is closed source.AARON: Yeah, it's kind of weird that Meta I don't know, does Meta really think it's in their I don't know. Most economic take on this would be like, oh, they somehow think it's in their shareholders interest to open source.HOLLY: There are a lot of speculations on why they're doing this. One is that? Yeah, their models aren't as good as the top labs, but if it's open source, then open source quote, unquote then people will integrate it llama Two into their apps. Or People Will Use It And Become I don't know, it's a little weird because I don't know why using llama Two commits you to using llama Three or something, but it just ways for their models to get in in places where if you just had to pay for their models too, people would go for better ones. That's one thing. Another is, yeah, I guess these are too speculative. I don't want to be seen repeating them since I'm about to do this purchase. But there's speculation that it's in best interests in various ways to do this. I think it's possible also that just like so what happened with the release of Llama One is they were going to allow approved people to download the weights, but then within four days somebody had leaked Llama One on four chan and then they just were like, well, whatever, we'll just release the weights. And then they released Llama Two with the weights from the beginning. And it's not like 100% clear that they intended to do full open source or what they call Open source. And I keep saying it's not open source because this is like a little bit of a tricky point to make. So I'm not emphasizing it too much. So they say that they're open source, but they're not. The algorithms are not open source. There are open source ML models that have everything open sourced and I don't think that that's good. I think that's worse. So I don't want to criticize them for that. But they're saying it's open source because there's all this goodwill associated with open source. But actually what they're doing is releasing the product for free or like trade secrets even you could say like things that should be trade secrets. And yeah, they're telling people how to make it themselves. So it's like a little bit of a they're intentionally using this label that has a lot of positive connotations but probably according to Open Source Initiative, which makes the open Source license, it should be called something else or there should just be like a new category for LLMs being but I don't want things to be more open. It could easily sound like a rebuke that it should be more open to make that point. But I also don't want to call it Open source because I think Open source software should probably does deserve a lot of its positive connotation, but they're not releasing the part, that the software part because that would cut into their business. I think it would be much worse. I think they shouldn't do it. But I also am not clear on this because the Open Source ML critics say that everyone does have access to the same data set as Llama Two. But I don't know. Llama Two had 7 billion tokens and that's more than GPT Four. And I don't understand all of the details here. It's possible that the tokenization process was different or something and that's why there were more. But Meta didn't say what was in the longitude data set and usually there's some description given of what's in the data set that led some people to speculate that maybe they're using private data. They do have access to a lot of private data that shouldn't be. It's not just like the common crawl backup of the Internet. Everybody's basing their training on that and then maybe some works of literature they're not supposed to. There's like a data set there that is in question, but metas is bigger than bigger than I think well, sorry, I don't have a list in front of me. I'm not going to get stuff wrong, but it's bigger than kind of similar models and I thought that they have access to extra stuff that's not public. And it seems like people are asking if maybe that's part of the training set. But yeah, the ML people would have or the open source ML people that I've been talking to would have believed that anybody who's decent can just access all of the training sets that they've all used.AARON: Aside, I tried to download in case I'm guessing, I don't know, it depends how many people listen to this. But in one sense, for a competent ML engineer, I'm sure open source really does mean that. But then there's people like me. I don't know. I knew a little bit of R, I think. I feel like I caught on the very last boat where I could know just barely enough programming to try to learn more, I guess. Coming out of college, I don't know, a couple of months ago, I tried to do the thing where you download Llama too, but I tried it all and now I just have like it didn't work. I have like a bunch of empty folders and I forget got some error message or whatever. Then I tried to train my own tried to train my own model on my MacBook. It just printed. That's like the only thing that a language model would do because that was like the most common token in the training set. So anyway, I'm just like, sorry, this is not important whatsoever.HOLLY: Yeah, I feel like torn about this because I used to be a genomicist and I used to do computational biology and it was not machine learning, but I used a highly parallel GPU cluster. And so I know some stuff about it and part of me wants to mess around with it, but part of me feels like I shouldn't get seduced by this. I am kind of worried that this has happened in the AI safety community. It's always been people who are interested in from the beginning, it was people who are interested in singularity and then realized there was this problem. And so it's always been like people really interested in tech and wanting to be close to it. And I think we've been really influenced by our direction, has been really influenced by wanting to be where the action is with AI development. And I don't know that that was right.AARON: Not personal, but I guess individual level I'm not super worried about people like you and me losing the plot by learning more about ML on their personal.HOLLY: You know what I mean? But it does just feel sort of like I guess, yeah, this is maybe more of like a confession than, like a point. But it does feel a little bit like it's hard for me to enjoy in good conscience, like, the cool stuff.AARON: Okay. Yeah.HOLLY: I just see people be so attached to this as their identity. They really don't want to go in a direction of not pursuing tech because this is kind of their whole thing. And what would they do if we weren't working toward AI? This is a big fear that people express to me with they don't say it in so many words usually, but they say things like, well, I don't want AI to never get built about a pause. Which, by the way, just to clear up, my assumption is that a pause would be unless society ends for some other reason, that a pause would eventually be lifted. It couldn't be forever. But some people are worried that if you stop the momentum now, people are just so luddite in their insides that we would just never pick it up again. Or something like that. And, yeah, there's some identity stuff that's been expressed. Again, not in so many words to me about who will we be if we're just sort of like activists instead of working on.AARON: Maybe one thing that we might actually disagree on. It's kind of important is whether so I think we both agree that Aipause is better than the status quo, at least broadly, whatever. I know that can mean different things, but yeah, maybe I'm not super convinced, actually, that if I could just, like what am I trying to say? Maybe at least right now, if I could just imagine the world where open eye and Anthropic had a couple more years to do stuff and nobody else did, that would be better. I kind of think that they are reasonably responsible actors. And so I don't k

IntroAt the gracious invitation of AI Safety Twitter-fluencer Sarah Hastings-Woodhouse, I appeared on the very first episode of her new podcast “Consistently Candid” to debate moral realism (or something kinda like that, I guess; see below) with fellow philosophy nerd and EA Twitter aficionado Max Alexander, alongside Sarah as moderator and judge of sorts.What I believeIn spite of the name of the episode and the best of my knowledge/understanding a few days ago, it turns out my stance may not be ~genuine~ moral realism. Here's my basic meta-ethical take:* Descriptive statements that concern objective relative goodness or badness (e.g., "it is objectively for Sam to donate $20 than to buy an expensive meal that costs $20 more than a similar, less fancy meal”) can be and sometimes are true; but* Genuinely normative claims like “Sam should (!) donate $20 and should not buy that fancy meal” are never objectively true.Of course the label per se doesn't really matter. But for a bunch of reasons it still seems wise to figure out which label really does work best.Some definitionsStanford Encyclopedia of Philosophy: Moral realists are those who think that, in these respects, things should be taken at face value—moral claims do purport to report facts and are true if they get the facts right. Moreover, they hold, at least some moral claims actually are true. That much is the common and more or less defining ground of moral realism (although some accounts of moral realism see it as involving additional commitments, say to the independence of the moral facts from human thought and practice, or to those facts being objective in some specified way).Wikipedia:The robust model of moral realism commits moral realists to three theses:* The semantic thesis: The primary semantic role of moral predicates (such as "right" and "wrong") is to refer to moral properties (such as rightness and wrongness), so that moral statements (such as "honesty is good" and "slavery is unjust") purport to represent moral facts, and express propositions that are true or false (or approximately true, largely false, and so on).* The alethic thesis: Some moral propositions are in fact true.* The metaphysical thesis: Moral propositions are true when actions and other objects of moral assessment have the relevant moral properties (so that the relevant moral facts obtain), where these facts and properties are robust: their metaphysical status, whatever it is, is not relevantly different from that of (certain types of) ordinary non-moral facts and properties.And to settle the matter, here's what some LLM have to say.My prompt:Suppose a person believes the following:1: Descriptive statements that concern objective relative goodness or badness (e.g., "it is objectively for Sam to donate $20 than to buy an expensive meal that costs $20 more than a similar, less fancy meal”) can be and sometimes are true; but 2: Genuinely normative claims like “Sam should donate $20 and should not buy that fancy meal” are never objectively true.Is this person best categorized as a moral realist?ChatGPTGiven these two beliefs, the person's stance doesn't align completely with traditional moral realism, which typically asserts that there are objective truths about both moral facts (like the goodness or badness of actions) and moral duties (what one should or should not do). However, it also doesn't fully align with moral anti-realism, since the person believes in some form of objective truth regarding moral value.One possible categorization for this stance could be a form of "value realism" combined with "normative anti-realism" or "normative subjectivism." This would mean that the person believes in objective truths about what is better or worse (value realism) but thinks that what one ought to do (normative claims) is not a matter of objective truth but perhaps of subjective judgment or societal conventions.This position is quite sophisticated and might be best understood as a hybrid or intermediate position in the landscape of moral philosophy, rather than fitting neatly into the category of moral realism or its standard alternatives.Great, so much for trying to fit things into nice little buckets.Claude, fortunately, is more definitive:No, the view described is not best categorized as moral realism. Here's why:Moral realism is the view that there are objective moral facts or truths. A moral realist believes that some moral claims, including normative claims about what one should or shouldn't do, can be objectively and mind-independently true.The view in the example accepts that there can be objective truths about relative degrees of goodness or badness (claim 1). However, it denies that there are objective normative truths about what one should or shouldn't do (claim 2). By rejecting that genuine normative 'should/shouldn't' claims can be objectively true, the view denies a key commitment of moral realism - namely that moral claims can state objective normative facts. At best it allows for objective evaluative facts about goodness/badness, not prescriptive facts about obligations.So this view would be more accurately categorized as a form of moral anti-realism like non-cognitivism or error theory. It does not qualify as moral realism because of its rejection of the existence of objective normative truths that could make strong normative claims justified or true. The commitment to objective evaluative facts alone does not sufficiently qualify the view as realist in the metaethical sense.In summary, the view accepts objective "good/bad" facts but denies objective "should/shouldn't" facts. This combination puts it at odds with core commitments of moral realism, so it is better seen as an anti-realist view.I'm gonna disagree with Claude here, at least in that the bolded claim above (objective normative facts are a key part of moral realism) just seems…wrong? Like all the definitions could replace the term “moral fact” or “moral claim” with “normative fact” and “normative claim” but they do not!So I really don't have a great answer to which label works best. One of two ~human level chatbots giving me a definitive-sounding “anti-realism” answer is too much evidence for me, whose only formal philosophy training is an undergrad minor, to rule that one out. There are also good arguments, I think, for the “realist label,” as well as for “neither” (i.e., ‘secret third thing'). In fact all of these seem pretty similar in terms of argument convincingness/correctness. So, in sum,

Note: I can't seem to edit or remove the “transcript” tab. I recommend you ignore that and just look at the much higher quality, slightly cleaned up one below. Most importantly, follow Sarah on Twitter! Summary (Written by chatGPT, as you can probably tell)In this episode of Pigeon Hour host Aaron delves deep into the world of AI safety with his guest, Sarah Woodhouse. Sarah shares her unexpected journey from fearing job automation to becoming a recognized voice on AI safety Twitter. Her story starts with a simple Google search that led her down a rabbit hole of existential dread and unexpected fame on social media. As she narrates her path from lurker to influencer, Sarah reflects on the quirky dynamics of the AI safety community, her own existential crisis, and the serendipitous tweet that resonated with thousands.Aaron and Sarah's conversation takes unexpected turns, discussing everything from the peculiarities of EA rationalists to the surprisingly serious topic of shrimp welfare. They also explore the nuances of AI doom probabilities, the social dynamics of tech Twitter, and Sarah's unexpected viral fame as a tween. This episode is a rollercoaster of insights and anecdotes, perfect for anyone interested in the intersection of technology, society, and the unpredictable journey of internet fame.Topics discussedDiscussion on AI Safety and Personal Journeys:* Aaron and Sarah discuss her path to AI safety, triggered by concerns about job automation and the realization that AI could potentially replace her work.* Sarah's deep dive into AI safety started with a simple Google search, leading her to Geoffrey Hinton's alarming statements, and eventually to a broader exploration without finding reassuring consensus.* Sarah's Twitter engagement began with lurking, later evolving into active participation and gaining an audience, especially after a relatable tweet thread about an existential crisis.* Aaron remarks on the rarity of people like Sarah, who follow the AI safety rabbit hole to its depths, considering its obvious implications for various industries.AI Safety and Public Perception:* Sarah discusses her surprise at discovering the AI safety conversation happening mostly in niche circles, often with a tongue-in-cheek attitude that could seem dismissive of the serious implications of AI risks.* The discussion touches on the paradox of AI safety: it's a critically important topic, yet it often remains confined within certain intellectual circles, leading to a lack of broader public engagement and awareness.Cultural Differences and Personal Interests:* The conversation shifts to cultural differences between the UK and the US, particularly in terms of sincerity and communication styles.* Personal interests, such as theater and musicals (like "Glee"), are also discussed, revealing Sarah's background and hobbies.Effective Altruism (EA) and Rationalist Communities:* Sarah points out certain quirks of the EA and rationalist communities, such as their penchant for detailed analysis, hedging statements, and the use of probabilities in discussions.* The debate around the use of "P(Doom)" (probability of doom) in AI safety discussions is critiqued, highlighting how it can be both a serious analytical tool and a potentially alienating jargon for outsiders.Shrimp Welfare and Ethical Considerations:* A detailed discussion on shrimp welfare as an ethical consideration in effective altruism unfolds, examining the moral implications and effectiveness of focusing on animal welfare at a large scale.* Aaron defends his position on prioritizing shrimp welfare in charitable giving, based on the principles of importance, tractability, and neglectedness.Personal Decision-Making in Charitable Giving:* Strategies for personal charitable giving are explored, including setting a donation cutoff point to balance moral obligations with personal needs and aspirations.TranscriptAARON: Whatever you want. Okay. Yeah, I feel like you said this on Twitter. The obvious thing is, how did you learn about AI safety? But maybe you've already covered that. That's boring. First of all, do you want to talk about that? Because we don't have to.SARAH: I don't mind talking about that.AARON: But it's sort of your call, so whatever. I don't know. Maybe briefly, and then we can branch out?SARAH: I have a preference for people asking me things and me answering them rather than me setting the agenda. So don't ever feel bad about just asking me stuff because I prefer that.AARON: Okay, cool. But also, it feels like the kind of thing where, of course, we have AI. Everyone already knows that this is just like the voice version of these four tweets or whatever. But regardless. Yes. So, Sarah, as Pigeon Hour guest, what was your path through life to AI safety Twitter?SARAH: Well, I realized that a chatbot could very easily do my job and that my employers either hadn't noticed this or they had noticed, but they were just being polite about it and they didn't want to fire me because they're too nice. And I was like, I should find out what AI development is going to be like over the next few years so that I know if I should go and get good at some other stuff.SARAH: I just had a little innocent Google. And then within a few clicks, I'd completely doom pilled myself. I was like, we're all going to die. I think I found Geoffrey Hinton because he was on the news at the time, because he just quit his job at Google. And he was there saying things that sounded very uncertain, very alarming. And I was like, well, he's probably the pessimist, but I'm sure that there are loads of optimists to counteract that because that's how it usually goes. You find a doomer and then you find a bunch of more moderate people, and then there's some consensus in the middle that everything's basically fine.SARAH: I was like, if I just keep looking, I'll find the consensus because it's there. I'm sure it's there. So I just kept looking and looking for it. I looked for it for weeks. I just didn't find it. And then I was like, nobody knows what's going on. This seems really concerning. So then I started lurking on Twitter, and then I got familiar with all the different accounts, whatever. And then at some point, I was like, I'm going to start contributing to this conversation, but I didn't think that anybody would talk back to me. And then at some point, they started talking back to me and I was like, this is kind of weird.SARAH: And then at some point, I was having an existential crisis and I had a couple of glasses of wine or something, and I just decided to type this big, long thread. And then I went to bed. I woke up the next morning slightly grouchy and hungover. I checked my phone and there were all these people messaging me and all these people replying to my thread being like, this is so relatable. This really resonated with me. And I was like, what is going on?AARON: You were there on Twitter before that thread right? I'm pretty sure I was following you.SARAH: I think, yeah, I was there before, but no one ever really gave me any attention prior to that. I think I had a couple of tweets that blew up before that, but not to the same extent. And then after that, I think I was like, okay, so now I have an audience. When I say an audience, like, obviously a small one, but more of an audience than I've ever had before in my life. And I was like, how far can I take this?SARAH: I was a bit like, people obviously started following me because I'm freFreaking out about AI, but if I post an outfit, what's going to happen? How far can I push this posting, these fit checks? I started posting random stuff about things that were completely unrelated. I was like, oh, people are kind of here for this, too. Okay, this is weird. So now I'm just milking it for all its worth, and I really don't know why anybody's listening to me. I'm basically very confused about the whole thing.AARON: I mean, I think it's kind of weird from your perspective, or it's weird in general because there aren't that many people who just do that extremely logical thing at the beginning. I don't know, maybe it's not obvious to people in every industry or whatever that AI is potentially a big deal, but there's lots of truckers or whatever. Maybe they're not the best demographic or the most conducive demographic, like, getting on Twitter or whatever, but there's other jobs that it would make sense to look into that. It's kind of weird to me that only you followed the rabbit hole all the way down.SARAH: I know! This is what I…Because it's not that hard to complete the circle. It probably took me like a day, it took me like an afternoon to get from, I'm worried about job automation to I should stop saving for retirement. It didn't take me that long. Do you know what I mean? No one ever looks. I literally don't get it. I was talking to some people. I was talking to one of my coworkers about this the other day, and I think I came up in conversation. She was like, yeah, I'm a bit worried about AI because I heard on the radio that taxi drivers might be out of a job. That's bad. And I was like, yeah, that is bad. But do you know what else? She was like, what are the AI companies up to that we don't know about? And I was like, I mean, you can go on their website. You can just go on their website and read about how they think that their technology is an extinction risk. It's not like they're hiding. It's literally just on there and no one ever looks. It's just crazy.AARON: Yeah. Honestly, I don't even know if I was in your situation, if I would have done that. It's like, in some sense, I am surprised. It's very few people maybe like one, but at another level, it's more rationality than most humans have or something. Yeah. You regret going down that rabbit hole?SARAH: Yeah, kind of. Although I'm enjoying the Twitter thing and it's kind of fun, and it turns out there's endless comedic material that you can get out of impending doom. The whole thing is quite funny. It's not funny, but you can make it funny if you try hard enough. But, yeah, what was I going to say? I think maybe I was more primed for doom pilling than your average person because I already knew what EA was and I already knew, you know what I mean. That stuff was on my radar.AARON: That's interesting.SARAH: I think had it not been on my radar, I don't think I would have followed the pipeline all the way.AARON: Yeah. I don't know what browser you use, but it would be. And you should definitely not only do this if you actually think it would be cool or whatever, but this could be in your browser history from that day and that would be hilarious. You could remove anything you didn't want to show, but if it's like Google Chrome, they package everything into sessions. It's one browsing session and it'll have like 10,000 links.SARAH: Yeah, I think for non-sketchy reasons, I delete my Google history more regularly than that. I don't think I'd be able to find that. But I can remember the day and I can remember my anxiety levels just going up and up somewhere between 01:00 p.m. and 07:00 p.m. And by the evening I'm like, oh, my God.AARON: Oh, damn, that's wild.SARAH: It was really stressful.AARON: Yeah, I guess props for, I don't know if props…Is the right word, I guess, impressed? I'm actually somewhat surprised to hear that you said you regret it. I mean, that sucks though, I guess. I'm sorry.SARAH: If you could unknow this, would you?AARON: No, because I think it's worth maybe selfishly, but not overall because. Okay, yeah, I think that would plausibly be the selfish thing to do. Actually. No, actually, hold on. No, I actually don't think that's true. I actually think there's enough an individual can do selfishly such that it makes sense. Even the emotional turmoil.SARAH: It would depend how much you thought that you were going to personally move the needle by knowing about it. I personally don't think that I'm going to be able to do very much. I was going to tip the scales. I wouldn't selfishly unknow it and sacrifice the world. But me being not particularly informed or intelligent and not having any power, I feel like if I forgot that AI was going to end the world, it would not make much difference.AARON: You know what I mean? I agree that it's like, yes, it is unlikely for either of us to tip the scales, but.SARAH: Maybe you can't.AARON: No, actually, in terms of, yeah, I'm probably somewhat more technically knowledgeable just based on what I know about you. Maybe I'm wrong.SARAH: No, you're definitely right.AARON: It's sort of just like a probabilities thing. I do think that ‘doom' - that word - is too simplified, often too simple to capture what people really care about. But if you just want to say doom versus no doom or whatever, AI doom versus no AI doom. Maybe there's like a one in 100,000 chance that one of us tips the scales. And that's important. Maybe even, like, one in 10,000. Probably not. Probably not.SARAH: One in 10,000. Wow.AARON: But that's what people do. People vote, even though this is old 80k material I'm regurgitating because they basically want to make the case for why even if you're not. Or in some article they had from a while ago, they made a case for why doing things that are unlikely to counterfactually matter can still be amazingly good. And the classic example, just voting if you're in a tight race, say, in a swing state in the United States, and it could go either way. Yeah. It might be pretty unlikely that you are the single swing vote, but it could be one in 100,000. And that's not crazy.SARAH: It doesn't take very much effort to vote, though.AARON: Yeah, sure. But I think the core justification, also, the stakes are proportionally higher here, so maybe that accounts for some. But, yes, you're absolutely right. Definitely different amounts of effort.SARAH: Putting in any effort to saving the world from AI. I wouldn't say that. I wouldn't say that I'm sacrificing.AARON: I don't even know if I like. No. Maybe it doesn't feel like a sacrifice. Maybe it isn't. But I do think there's, like, a lot. There's at least something to be. I don't know if this really checks out, but I would, like, bet that it does, which is that more reasonably, at least calibrated. I wanted to say reasonably well informed. But really what it is is, like, some level of being informed and, like, some level of knowing what you don't know or whatever, and more just like, normal. Sorry. I hope normal is not like a bat. I'm saying not like tech Bros, I guess so more like non tech bros. People who are not coded as tech bros. Talking about this on a public platform just seems actually, in fact, pretty good.SARAH: As long as we like, literally just people that aren't men as well. No offense.AARON: Oh, no, totally. Yeah.SARAH: Where are all the women? There's a few.AARON: There's a few that are super. I don't know, like, leaders in some sense, like Ajeya Cotra and Katja Grace. But I think the last EA survey was a third. Or I could be butchering this or whatever. And maybe even within that category, there's some variation. I don't think it's 2%.SARAH: Okay. All right. Yeah.AARON: Like 15 or 20% which is still pretty low.SARAH: No, but that's actually better than I would have thought, I think.AARON: Also, Twitter is, of all the social media platforms, especially mail. I don't really know.SARAH: Um.AARON: I don't like Instagram, I think.SARAH: I wonder, it would be interesting to see whether or not that's much, if it's become more male dominated since Elon Musk took.AARON: It's not a huge difference, but who knows?SARAH: I don't know. I have no idea. I have no idea. We'll just be interesting to know.AARON: Okay. Wait. Also, there's no scheduled time. I'm very happy to keep talking or whatever, but as soon as you want to take a break or hop off, just like. Yeah.SARAH: Oh, yeah. I'm in no rush.AARON: Okay, well, I don't know. We've talked about the two obvious candidates. Do you have a take or something? Want to get out to the world? It's not about AI or obesity or just a story you want to share.SARAH: These are my two pet subjects. I don't know anything else.AARON: I don't believe you. I know you know about house plants.SARAH: I do. A secret, which you can't tell anyone, is that I actually only know about house plants that are hard to kill, and I'm actually not very good at taking care of them.AARON: Well, I'm glad it's house plants in that case, rather than pets. Whatever.SARAH: Yeah. I mean, I have killed some sea monkeys, too, but that was a long time ago.AARON: Yes. So did I, actually.SARAH: Did you? I feel like everyone has. Everyone's got a little sea monkey graveyard in their past.AARON: New cause area.SARAH: Are there more shrimp or more sea monkeys? That's the question.AARON: I don't even know what even. I mean, are they just plankton?SARAH: No, they're not plankton.AARON: I know what sea monkeys are.SARAH: There's definitely a lot of them because they're small and insignificant.AARON: Yeah, but I also think we don't. It depends if you're talking about in the world, which I guess probably like sea monkeys or farmed for food, which is basically like. I doubt these are farmed either for food or for anything.SARAH: Yeah, no, you're probably right.AARON: Or they probably are farmed a tiny bit for this niche little.SARAH: Or they're farmed to sell in aquariums for kids.AARON: Apparently. They are a kind of shrimp, but they were bred specifically to, I don't know, be tiny or something. I'm just skimming that, Wikipedia. Here.SARAH: Sea monkeys are tiny shrimp. That is crazy.AARON: Until we get answers, tell me your life story in whatever way you want. It doesn't have to be like. I mean, hopefully not. Don't straight up lie, but wherever you want to take that.SARAH: I'm not going to lie. I'm just trying to think of ways to make it spicier because it's so average. I don't know what to say about it.AARON: Well, it's probably not that average, right? I mean, it might be average among people you happen to know.SARAH: Do you have any more specific questions?AARON: Okay, no. Yeah, hold on. I have a meta point, which is like, I think the people who are they have a thing on the top of their mind, and if I give any sort of open ended question whatsoever, they'll take it there and immediately just start giving slinging hot takes. But thenOther people, I think, this category is very EA. People who aren't, especially my sister, they're like, “No, I have nothing to talk about. I don't believe that.” But they're not, I guess, as comfortable.SARAH: No, I mean, I have. Something needs to trigger them in me. Do you know what I mean? Yeah, I need an in.AARON: Well, okay, here's one. Is there anything you're like, “Maybe I'll cut this. This is kind of, like narcissistic. I don't know. But is there anything you want or curious to ask?” This does sound kind of weird. I don't know. But we can cut it if need be.SARAH: What does the looking glass in your Twitter name mean? Because I've seen a bunch of people have this, and I actually don't know what it means, but I was like, no.AARON: People ask this. I respond to a tweet that's like, “What does that like?” At least, I don't know, once every month or two. Or know basically, like Spencer Greenberg. I don't know if you're familiar with him. He's like a sort of.SARAH: I know the know.AARON: He literally just tweeted, like a couple years ago. Put this in your bio to show that you really care about finding the truth or whatever and are interested in good faith conversations. Are you familiar with the scout mindset?SARAH: Yeah.AARON: Julia Galef. Yeah. That's basically, like the short version.SARAH: Okay.AARON: I'm like, yeah, all right. And there's at least three of us who have both a magnifying glass. Yeah. And a pause thing, which is like, my tightest knit online community I guess.SARAH: I think I've followed all the pause people now. I just searched the emoji on Twitter, and I just followed everyone. Now I can't find. And I also noticed when I was doing this, that some people, if they've suspended their account or they're taking time off, then they put a pause in their thing. So I was, like, looking, and I was like, oh, these are, like, AI people. But then they were just, like, in their bio, they were, like, not tweeting until X date. This is a suspended account. And I was like, I see we have a messaging problem here. Nice. I don't know how common that actually.AARON: Was. I'm glad. That was, like, a very straightforward question. Educated the masses. Max Alexander said Glee. Is that, like, the show? You can also keep asking me questions, but again, this is like.SARAH: Wait, what did he say? Is that it? Did he just say glee? No.AARON: Not even a question mark. Just the word glee.SARAH: Oh, right. He just wants me to go off about Glee.AARON: Okay. Go off about. Wait, what kind of Glee are we? Vaguely. This is like a show or a movie or something.SARAH: Oh, my God. Have you not seen it?AARON: No. I mean, I vaguely remember, I think, watching some TV, but maybe, like, twelve years ago or something. I don't know.SARAH: I think it stopped airing in, like, maybe 2015?AARON: 16. So go off about it. I don't know what I. Yeah, I.SARAH: Don't know what to say about this.AARON: Well, why does Max think you might have a take about Glee?SARAH: I mean, I don't have a take about. Just see the thing. See? No, not even, like, I am just transparently extremely lame. And I really like cheesy. I'm like. I'm like a musical theater kid. Not even ironically. I just like show tunes. And Glee is just a show about a glee club at a high school where they sing show tunes and there's, like, petty drama, and people burst into song in the hallways, and I just think it's just the most glorious thing on Earth. That's it. There are no hot takes.AARON: Okay, well, that's cool. I don't have a lot to say, unfortunately, but.SARAH: No, that's totally fine. I feel like this is not a spicy topic for us to discuss. It's just a good time.AARON: Yeah.SARAH: Wait.AARON: Okay. Yeah. So I do listen to Hamilton on Spotify.SARAH: Okay.AARON: Yeah, that's about it.SARAH: I like Hamilton. I've seen it three times. Oh.AARON: Live or ever. Wow. Cool. Yeah, no, that's okay. Well, what do people get right or wrong about theater kids?SARAH: Oh, I don't know. I think all the stereotypes are true.AARON: I mean, that's generally true, but usually, it's either over moralized, there's like a descriptive thing that's true, but it's over moralized, or it's just exaggerated.SARAH: I mean, to put this in more context, I used to be in choir. I went every Sunday for twelve years. And then every summer we do a little summer school and we go away and put on a production. So we do a musical or something. So I have been. What have I been? I was in Guys and Dolls. I think I was just in the chorus for that. I was the reverend in Anything Goes. But he does unfortunately get kidnapped in like the first five minutes. So he's not a big presence. Oh, I've been Tweedle dumb in Alice in Wonderland. I could go on, but right now as I'm saying this, I'm looking at my notice board and I have two playbills from when I went to Broadway in April where I saw Funny Girl and Hadestown.SARAH: I went to New York.AARON: Oh, cool. Oh yeah. We can talk about when you're moving to the United States. However.SARAH: I'm not going to do that. Okay.AARON: I know. I'm joking. I mean, I don't know.SARAH: I don't think I'm going to do that. I don't know. It just seems like you guys have got a lot going on over there. It seems like things aren't quite right with you guys. Things aren't quite right with us either.AARON: No, I totally get this. I think it would be cool. But also I completely relate to not wanting to. I've lived within 10 miles of one. Not even 10 miles, 8 miles in one location. Obviously gone outside of that. But my entire life.SARAH: You've just always lived in DC.AARON: Yeah, either in DC or. Sorry. But right now in Maryland, it's like right next to DC on the Metro or at Georgia University, which is in the trying to think would I move to the UK. Like I could imagine situations that would make me move to the UK. But it would still be annoying. Kind of.SARAH: Yeah, I mean, I guess it's like they're two very similar places, but there are all these little cultural things which I feel like kind of trip you up.AARON: I don't to. Do you want to say what?SARAH: Like I think people, I just like, I don't know. I don't have that much experience because I've only been to America twice. But people seem a lot more sincere in a way that you don't really get that. Like people are just never really being upfront. And in America, I just got the impression that people just have less of a veneer up, which is probably a good thing. But it's really hard to navigate if you're not used to it or something. I don't know how to describe that.AARON: Yeah, I've definitely heard this at least. And yeah, I think it's for better and for worse.SARAH: Yeah, I think it's generally a good thing.AARON: Yeah.SARAH: But it's like there's this layer of cynicism or irony or something that is removed and then when it's not there, it's just everything feels weak. I can't describe it.AARON: This is definitely, I think, also like an EA rationalist thing. I feel like I'm pretty far on the spectrum. Towards the end of surgical niceties are fine, but I don't know, don't obscure what you really think unless it's a really good reason to or something. But it can definitely come across as being rude.SARAH: Yeah. No, but I think it's actually a good rule of thumb to obscure what you. It's good to try not to obscure what you think most of the time, probably.Ably, I don't know, but I would love to go over temporarily for like six months or something and just hang out for a bit. I think that'd be fun. I don't know if I would go back to New York again. Maybe. I like the bagels there.AARON: I should have a place. Oh yeah. Remember, I think we talked at some point. We can cut this out if you like. Don't if either of us doesn't want it in. But we discussed, oh yeah, I should be having a place. You can. I emailed the landlord like an hour before this. Hopefully, probably more than 50%. That is still an offer. Yeah, probably not for all six months, but I don't know.SARAH: I would not come and sleep on your sofa for six months. That would be definitely impolite and very weird.AARON: Yeah. I mean, my roommates would probably grumble.SARAH: Yeah. They would be like.AARON: Although I don't know. Who knows? I wouldn't be shocked if people were actually like, whatever somebody asked for as a question. This is what he said. I might also be interested in hearing how different backgrounds. Wait, sorry. This is not good grammar. Let me try to parse this. Not having a super hardcore EA AI rationalist background shape how you think or how you view AI as rationality?SARAH: Oh, that's a good question. I think it's more happening the other way around, the more I hang around in these circles. You guys are impacting how I think.AARON: It's definitely true for me as well.SARAH: Seeping into my brain and my language as well. I've started talking differently. I don't know. That's a good question, though. Yeah. One thing that I will say is that there are certain things that I find irritating about the EA way of style of doing things. I think one specific, I don't know, the kind of like hand ring about everything. And I know that this is kind of the point, right? But it's kind of like, you know, when someone's like, I want to take a stance on something, but then whenever they want to take a stance on something, they feel the need to write like a 10,000 word blog post where they're thinking about the second and order and third and fifth order effects of this thing. And maybe this thing that seems good is actually bad for this really convoluted reason. That's just so annoying.AARON: Yeah.SARAH: Also understand that maybe that is a good thing to do sometimes, but it just seems like, I don't know how anyone ever gets anywhere. It seems like everyone must be paralyzed by indecision all the time because they just can't commit to ever actually just saying anything.AARON: I think this kind of thing is really good if you're trying to give away a billion dollars. Oh yes, I do want the billion dollar grantor to be thinking through second and third order effects of how they give away their billion dollars. But also, no, I am super. The words on the tip of my tongue, not overwhelmed but intimidated when I go on the EA forum because the posts, none of them are like normal, like five paragraph essays. Some of them are like, I think one of them I looked up for fun because I was going to make a meme about it and still will. Probably was like 30,000 words or something. And even the short form posts, which really gets me kind of not even annoyed. I don't know, maybe kind of annoyed is that the short form posts, which is sort of the EA forum version of Twitter, are way too high quality, way too intimidating. And so maybe I should just suck it up and post stuff anyway more often. It just feels weird. I totally agree.SARAH: I was also talking to someone recently about how I lurked on the EA forum and less wrong for months and months and I couldn't figure out the upvoting system and I was like, am I being stupid or why are there four buttons? And I was like, well, eventually I had to ask someone because I couldn't figure it out. And then he explained it to me and I was like, that is just so unnecessary. Like, just do it.AARON: No, I do know what you mean.SARAH: I just tI think it's annoying. It pisses me off. I just feel like sometimes you don't need to add more things. Sometimes less is good. Yeah, that's my hot take. Nice things.AARON: Yeah, that's interesting.SARAH: But actually, a thing that I like that EA's do is the constant hedging and caveatting. I do find it kind of adorable. I love that because it's like you're having to constantly acknowledge that you probably didn't quite articulate what you really meant and that you're not quite making contact with reality when you're talking. So you have to clarify that you probably were imprecise when you said this thing. It's unnecessary, but it's kind of amazing.AARON: No, it's definitely. I am super guilty of this because I'll give an example in a second. I think I've been basically trained to try pretty hard, even in normal conversation with anybody, to just never say anything that's literally wrong. Or at least if I do caveat it.AARON: I was driving home, me and my parents and I, unless visited, our grandparents were driving back, and we were driving back past a cruise ship that was in a harbor. And my mom, who was driving at the time, said, “Oh, Aaron, can you see if there's anyone on there?” And I immediately responded like, “Well, there's probably at least one person.” Obviously, that's not what she meant. But that was my technical best guess. It's like, yes, there probably are people on there, even though I couldn't see anybody on the decks or in the rooms. Yeah, there's probably a maintenance guy. Felt kind of bad.SARAH: You can't technically exclude that there are, in fact, no people.AARON: Then I corrected myself. But I guess I've been trained into giving that as my first reaction.SARAH: Yeah, I love that. I think it's a waste of words, but I find it delightful.AARON: It does go too far. People should be more confident. I wish that, at least sometimes, people would say, “Epistemic status: Want to bet?” or “I am definitely right about this.” Too rarely do we hear, "I'm actually pretty confident here.SARAH: Another thing is, people are too liberal with using probabilities. The meaning of saying there is an X percent chance of something happening is getting watered down by people constantly saying things like, “I would put 30% on this claim.” Obviously, there's no rigorous method that's gone into determining why it's 30 and not 35. That's a problem and people shouldn't do that. But I kind of love it.AARON: I can defend that. People are saying upfront, “This is my best guess. But there's no rigorous methodology.” People should take their word for that. In some parts of society, it's seen as implying that a numeric probability came from a rigorous model. But if you say, “This is my best guess, but it's not formed from anything,” people should take their word for that and not refuse to accept them at face value.SARAH: But why do you have to put a number on it?AARON: It depends on what you're talking about. Sometimes probabilities are relevant and if you don't use numbers, it's easy to misinterpret. People would say, “It seems quite likely,” but what does that mean? One person might think “quite reasonably likely” means 70%, the other person thinks it means 30%. Even though it's weird to use a single number, it's less confusing.SARAH: To be fair, I get that. I've disagreed with people about what the word “unlikely” means. Someone's pulled out a scale that the government uses, or intelligence services use to determine what “unlikely” means. But everyone interprets those words differently. I see what you're saying. But then again, I think people in AI safety talking about P Doom was making people take us less seriously, especially because people's probabilities are so vibey.AARON: Some people are, but I take Paul Cristiano's word seriously.SARAH: He's a 50/50 kind of guy.AARON: Yeah, I take that pretty seriously.Obviously, it's not as simple as him having a perfect understanding of the world, even after another 10,000 hours of investigation. But it's definitely not just vibes, either.SARAH: No, I came off wrong there. I don't mean that everyone's understanding is just vibes.AARON: Yeah.SARAH: If you were looking at it from the outside, it would be really difficult to distinguish between the ones that are vibes and the ones that are rigorous, unless you carefully parsed all of it and evaluated everyone's background, or looked at the model yourself. If you're one step removed, it looks like people just spitting out random, arbitrary numbers everywhere.AARON: Yeah. There's also the question of whether P doom is too weird or silly, or if it could be easily dismissed as such.SARAH: Exactly, the moment anyone unfamiliar with this discussion sees it, they're almost definitely going to dismiss it. They won't see it as something they need to engage with.AARON: That's a very fair point. Aside from the social aspect, it's also a large oversimplification. There's a spectrum of outcomes that we lump into doom and not doom. While this binary approach can be useful at times, it's probably overdone.SARAH: Yeah, because when some people say doom, they mean everyone dies, while others mean everyone dies plus everything is terrible. And no one specifies what they mean. It is silly. But, I also find it kind of funny and I kind of love it.AARON: I'm glad there's something like that. So it's not perfect. The more straightforward thing would be to say P existential risk from AI comes to pass. That's the long version, whatever.SARAH: If I was in charge, I would probably make people stop using PDOOm. I think it's better to say it the long way around. But obviously I'm not in charge. And I think it's funny and kind of cute, so I'll keep using it.AARON: Maybe I'm willing to go along and try to start a new norm. Not spend my whole life on it, but say, I think this is bad for X, Y, and Z reasons. I'll use this other phrase instead and clarify when people ask.SARAH: You're going to need Twitter premium because you're going to need a lot more characters.AARON: I think there's a shorthand which is like PX risk or P AiX risk.SARAH: Maybe it's just the word doom that's a bit stupid.AARON: Yeah, that's a term out of the Bay Area rationalists.SARAH: But then I also think it kind of makes the whole thing seem less serious. People should be indignant to hear that this meme is being used to trade probabilities about the likelihood that they're going to die and their families are going to die. This has been an in-joke in this weird niche circle for years and they didn't know about it. I'm not saying that in a way to morally condemn people, but if you explain this to people…People just go to dinner parties in Silicon Valley and talk about this weird meme thing, and what they really mean is the ODs know everyone's going to prematurely die. People should be outraged by that, I think.AARON: I disagree that it's a joke. It is a funny phrase, but the actual thing is people really do stand by their belief.SARAH: No, I totally agree with that part. I'm not saying that people are not being serious when they give their numbers, but I feel like there's something. I don't know how to put this in words. There's something outrageous about the fact that for outsiders, this conversation has been happening for years and people have been using this tongue-in-cheek phrase to describe it, and 99.9% of people don't know that's happening. I'm not articulating this very well.AARON: I see what you're saying. I don't actually think it's like. I don't know a lot of jargon.SARAH: But when I first found out about this, I was outraged.AARON: I honestly just don't share that intuition. But that's really good.SARAH: No, I don't know how to describe this.AARON: I think I was just a little bit indignant, perhaps.SARAH: Yeah, I was indignant about it. I was like, you guys have been at social events making small talk by discussing the probability of human extinction all this time, and I didn't even know. I was like, oh, that's really messed up, guys.AARON: I feel like I'm standing by the rational tier because, it was always on. No one was stopping you from going on less wrong or whatever. It wasn't behind closed.SARAH: Yeah, but no one ever told me about it.AARON: Yeah, that's like a failure of outreach, I suppose.SARAH: Yeah. I think maybe I'm talking more about. Maybe the people that I'm mad at is the people who are actually working on capabilities and using this kind of jargon. Maybe I'm mad at those people. They're fine.AARON: Do we have more questions? I think we might have more questions. We have one more. Okay, sorry, but keep going.SARAH: No, I'm going to stop making that point now because I don't really know what I'm trying to say and I don't want to be controversial.AARON: Controversy is good for views. Not necessarily for you. No, thank you for that. Yes, that was a good point. I think it was. Maybe it was wrong. I think it seems right.SARAH: It was probably wrong.Shrimp Welfare: A Serious DiscussionAARON: I don't know what she thinks about shrimp welfare. Oh, yeah. I think it's a general question, but let's start with that. What do you think about shrimp? Well, today.SARAH: Okay. Is this an actual cause area or is this a joke about how if you extrapolate utilitarianism to its natural conclusion, you would really care about shrimp?AARON: No, there's a charity called the Shrimp Welfare Initiative or project. I think it's Shrimp Welfare Initiative. I can actually have a rant here about how it's a meme that people find amusing. It is a serious thing, but I think people like the meme more than they're willing to transfer their donations in light of it. This is kind of wrong and at least distasteful.No, but there's an actual, if you Google, Shrimp Welfare Project. Yeah, it's definitely a thing, but it's only a couple of years old. And it's also kind of a meme because it does work in both ways. It sort of shows how we're weird, but in the sense that we are willing to care about things that are very different from us. Not like we're threatening other people. That's not a good description.SARAH: Is the extreme version of this position that we should put more resources into improving the lives of shrimp than into improving the lives of people just because there are so many more shrimp? Are there people that actually believe that?AARON: Well, I believe some version of that, but it really depends on who the ‘we' is there.SARAH: Should humanity be putting more resources?AARON: No one believes that as far as I know.SARAH: Okay. Right. So what is the most extreme manifestation of the shrimp welfare position?AARON: Well, I feel like my position is kind of extreme, and I'm happy to discuss it. It's easier than speculating about what the more extreme ones are. I don't think any of them are that extreme, I guess, from my perspective, because I think I'm right.SARAH: Okay, so what do you believe?AARON: I think that most people who have already decided to donate, say $20, if they are considering where to donate it and they are better morally, it would be better if they gave it to the shrimp welfare project than if they gave it to any of the commonly cited EA organizations.SARAH: Malaria nets or whatever.AARON: Yes. I think $20 of malaria nets versus $20 of shrimp. I can easily imagine a world where it would go the other way. But given the actual situation, the $20 of shrimp is much better.SARAH: Okay. Is it just purely because there's just more shrimp? How do we know how much shrimp suffering there is in the world?AARON: No, this is an excellent question. The numbers are a key factor, but no, it's not as simple. I definitely don't think one shrimp is worth one human.SARAH: I'm assuming that it's based on the fact that there are so many more shrimp than there are people that I don't know how many shrimp there are.AARON: Yeah, that's important, but at some level, it's just the margin. What I think is that when you're donating money, you should give to wherever it does the most good, whatever that means, whatever you think that means. But let's just leave it at that. The most good is morally best at the margin, which means you're not donating where you think the world should or how you think the world should expend its trillion dollar wealth. All you're doing is adding $20 at this current level, given the actual world. And so part of it is what you just said, and also including some new research from Rethink Priorities.Measuring suffering in reasonable ranges is extremely hard to do. But I believe it's difficult to do a better job than raising priorities on that, given what I've seen. I can provide some links. There are a few things to consider here: numbers, times, and the enormity of suffering. I think there are a couple of key elements, including tractability.Are you familiar with the three-pronged concept people sometimes discuss, which encompasses tractability, and neglectedness?SARAH: Okay.AARON: Importance is essentially what we just mentioned. Huge numbers and plausible amounts of suffering. When you try to do the comparison, it seems like they're a significant concern. Tractability is another factor. I think the best estimates suggest that a one-dollar donation could save around 10,000 shrimp from a very painful death.SARAH: In that sense…AARON: You could imagine that even if there were a hundred times more shrimp than there actually are, we have direct control over how they live and die because we're farming them. The industry is not dominated by wealthy players in the United States. Many individual farmers in developing nations, if educated and provided with a more humane way of killing the shrimp, would use it. There's a lot of potential for improvement here. This is partly due to the last prong, neglectedness, which is really my focus.SARAH: You're saying no one cares about the shrimp.AARON: I'm frustrated that it's not taken seriously enough. One of the reasons why the marginal cost-effectiveness is so high is because large amounts of money are donated to well-approved organizations. But individual donors often overlook this. They ignore their marginal impact. If you want to see even a 1% shift towards shrimp welfare, the thing to do is to donate to shrimp welfare. Not donate $19 to human welfare and one dollar to shrimp welfare, which is perhaps what they think the overall portfolio should be.SARAH: Interesting. I don't have a good reason why you're wrong. It seems like you're probably right.AARON: Let me put the website in the chat. This isn't a fair comparison since it's something I know more about.SARAH: Okay.AARON: On the topic of obesity, neither of us were more informed than the other. But I could have just made stuff up or said something logically fallacious.SARAH: You could have told me that there were like 50 times the number of shrimp in the world than there really are. And I would have been like, sure, seems right.AARON: Yeah. And I don't know, if I…If I were in your position, I would say, “Oh, yeah, that sounds right.” But maybe there are other people who have looked into this way more than me that disagree, and I can get into why I think it's less true than you'd expect in some sense.SARAH: I just wonder if there's like… This is like a deeply non-EA thing to say. So I don't know, maybe I shouldn't say it, but are there not any moral reasons? Is there not any good moral philosophy behind just caring more about your own species than other species? If you're sorry, but that's probably not right, is it? There's probably no way to actually morally justify that, but it seems like it feels intuitively wrong. If you've got $20 to be donating 19 of them to shrimp and one to children with malaria, that feels like there should be something wrong with that, but I can't tell you what it is.AARON: Yeah, no, there is something wrong, which is that you should donate all 20 because they're acting on the margin, for one thing. I do think that doesn't check out morally, but I think basically me and everybody I know in terms of real life or whatever, I do just care way more about humans. I don't know, for at least the people that it's hard to formalize or specify what you mean by caring about or something. But, yeah, I think you can definitely basically just be a normal human who basically cares a lot about other humans. And still that's not like, negated by changing your $20 donation or whatever. Especially because there's nothing else that I do for shrimp. I think you should be like a kind person or something. I'm like an honest person, I think. Yeah, people should be nice to other humans. I mean, you should be nice in the sense of not beating them. But if you see a pigeon on the street, you don't need to say hi or whatever, give it a pet, because. I don't know. But yeah, you should be basically like, nice.SARAH: You don't stop to say hi to every pigeon that you see on the way to anywhere.AARON: I do, but I know most normal people don't.SARAH: This is why I'm so late to everything, because I have to do it. I have to stop for every single one. No exceptions.AARON: Yeah. Or how I think about it is sort of like a little bit of compartmentalization, which I think is like… Which is just sort of like a way to function normally and also sort of do what you think really checks out at the end of the day, just like, okay, 99% of the time I'm going to just be like a normal person who doesn't care about shrimp. Maybe I'll refrain from eating them. But actually, even that is like, I could totally see a person just still eating them and then doing this. But then during the 1% of the time where you're deciding how to give money away and none of those, the beneficiaries are going to be totally out of sight either way. This is like a neutral point, I guess, but it's still worth saying, yeah, then you can be like a hardcore effective altruist or whatever and then give your money to the shrimp people.SARAH: Do you have this set up as like a recurring donation?AARON: Oh, no. Everybody should call me out as a hypocrite because I haven't donated much money, but I'm trying to figure out actually, given that I haven't had a stable income ever. And maybe, hopefully I will soon, actually. But even then, it's still a part-time thing. I haven't been able to do sort of standard 10% or more thing, and I'm trying to figure out what the best thing to do or how to balance, I guess, not luxury, not like consumption on things that I… Well, to some extent, yeah. Maybe I'm just selfish by sometimes getting an Uber. That's totally true. I think I'm just a hypocrite in that respect. But mostly I think the trade-off is between saving, investing, and giving. Beast of the money that I have saved up and past things. So this is all sort of a defense of why I don't have a recurring donation going on.SARAH: I'm not asking you to defend yourself because I do not do that either.AARON: I think if I was making enough money that I could give away $10,000 a year and plan on doing that indefinitely, I would be unlikely to set up a recurring donation. What I would really want to do is once or twice a year, really try to prioritize deciding on how to give it away rather than making it the default. This has a real cost for charities. If you set up a recurring donation, they have more certainty in some sense of their future cash flow. But that's only good to do if you're really confident that you're going to want to keep giving there in the future. I could learn new information that says something else is better. So I don't think I would do that.SARAH: Now I'm just thinking about how many shrimp did you say it was per dollar?AARON: Don't quote me. I didn't say an actual thing.SARAH: It was like some big number. Right. Because I just feel like that's such a brainworm. Imagine if you let that actually get in your head and then every time you spend some unnecessary amount of money on something you don't really need, you think about how many shrimp you just killed by getting an Uber or buying lunch out. That is so stressful. I think I'm going to try not to think about that.AARON: I don't mean to belittle this. This is like a core, I think you're new to EA type of thinking. It's super natural and also troubling when you first come upon it. Do you want me to talk about how I, or other people deal with that or take action?SARAH: Yeah, tell me how to get the shrimp off my conscience.AARON: Well, for one thing, you don't want to totally do that. But I think the main thing is that the salience of things like this just decreases over time. I would be very surprised if, even if you're still very engaged in the EA adjacent communities or EA itself in five years, that it would be as emotionally potent. Brains make things less important over time. But I think the thing to do is basically to compartmentalize in a sort of weird sense. Decide how much you're willing to donate. And it might be hard to do that, but that is sort of a process. Then you have that chunk of money and you try to give it away the best you can under whatever you think the best ethics are. But then on the daily, you have this other set pot of money. You just are a normal person. You spend it as you wish. You don't think about it unless you try not to. And maybe if you notice that you might even have leftover money, then you can donate the rest of it. But I really do think picking how much to give should sort of be its own project. And then you have a pile of money you can be a hardcore EA about.SARAH: So you pick a cut off point and then you don't agonize over anything over and above that.AARON: Yeah. And then people, I mean, the hard part is that if somebody says their cut off point is like 1% of their income and they're making like $200,000, I don't know. Maybe their cut off point should be higher. So there is a debate. It depends on that person's specific situation. Maybe if they have a kid or some super expensive disease, it's a different story. If you're just a random guy making $200,000, I think you should give more.SARAH: Maybe you should be giving away enough to feel the pinch. Well, not even that. I don't think I'm going to do that. This is something that I do actually want to do at some point, but I need to think about it more and maybe get a better job.AARON: Another thing is, if you're wanting to earn to give as a path to impact, you could think and strive pretty hard. Maybe talk to people and choose your education or professional development opportunities carefully to see if you can get a better paying job. That's just much more important than changing how much you give from 10% to 11% or something. You should have this macro level optimization. How can I have more money to spend? Let me spend, like, I don't know, depends what life stage you are, but if you had just graduated college or maybe say you're a junior in college or something. It could make sense to spend a good amount of time figuring out what that path might look like.AARON: I'm a huge hypocrite because I definitely haven't done all this nearly as much as I should, but I still endorse it.SARAH: Yeah, I think it's fine to say what you endorse doing in an ideal world, even if you're not doing that, that's fine.AARON: For anybody listening, I tweeted a while ago, asking if anyone has resources on how to think about giving away wealth. I'm not very wealthy but have some amount of savings. It's more than I really need. At the same time, maybe I should be investing it because EA orgs don't feel like, or they think they can't invest it because there's potentially a lot of blowback if they make poor investments, even though it would be higher expected value.There's also the question of, okay, having some amount of savings allows me to take higher, potentially somewhat higher risk, but higher value opportunities because I have a cushion. But I'm very confused about how to give away what I should do here. People should DM me on Twitter or anywhere they have ideas.SARAH: I think you should calculate how much you need to cover your very basic needs. Maybe you should work out, say, if you were working 40 hours a week in a minimum wage job, like how much would you make then? And then you should keep that for yourself. And then the rest should definitely all go to the shrimp. Every single penny. All of it.AARON: This is pretty plausible. Just to make it more complicated, there's also the thing that I feel like my estimates or my best guesses of the best charities to give to over time has changed. And so there's like two competing forces. One is that I might get wiser and more knowledgeable as time goes on. The other one is that in general, giving now is better than giving later. All else equal, because I think for a couple of reasons, the main one just being that the charities don't know that you're going to give later.AARON: So it's like they can plan for the future much better if they get money now. And also there's just higher leverage opportunities or higher value per dollar opportunities now in general than there will be later for a couple of reasons I don't really need to. This is what makes it really complicated. So I've donated in the past to places that I don't think, or I don't think even at the time were the best to. So then there's a question of like, okay, how long do I save this money? Do I sit on it for months until I'm pretty confident, like a year.AARON: I do think that probably over the course of zero to five years or something, becoming more confident or changing your mind is like the stronger effect than how much good you give to the, or how much better it is for the charities to give now instead of later. But also that's weird because you're never committing at all.Sometimes you might decide to give it away, and maybe you won't. Maybe at that time you're like, “Oh, that's what I want. A car, I have a house, whatever.” It's less salient or something. Maybe something bad happened with EA and you no longer identify that way. Yeah, there's a lot of really thorny considerations. Sorry, I'm talking way too much.SARAH: Long, are you factoring AI timelines into this?AARON: That makes it even more sketchy. But that could also go both ways. On one hand, you have the fact that if you don't give away your money now and you die with it, it's never going to do any good. The other thing is that it might be that especially high leverage opportunities come in the future or something potentially you need, I don't know, whatever I can imagine I could make something up about. OpenPhil needs as much money as it can get to do X, Y and Z. It's really important right now, but I won't know that until a few years down the line. So just like everything else, it doesn't neatly wash out.SARAH: What do you think the AGI is going to do to the shrimp? I reckon it's probably pretty neat, like one shrimp per paperclip. Maybe you could get more. I wonder what the sort of shrimp to paperclip conversion rate is.AARON: Has anyone looked into that morally? I think like one to zero. I don't think in terms of money. You could definitely price that. I have no idea.SARAH: I don't know. Maybe I'm not taking this as seriously as I should be because I'm.AARON: No, I mean, humor is good. When people are giving away money or deciding what to do, they should be serious. But joking and humor is good. Sorry, go ahead.SARAH: No, you go ahead.AARON: I had a half-baked idea. At EA Global, they should have a comedy show where people roast everybody, but it's a fundraiser. You have to pay to get 100 people to attend. They have a bidding contest to get into the comedy show. That was my original idea. Or they could just have a normal comedy show. I think that'd be cool.SARAH: Actually, I think that's a good idea because you guys are funny. There is a lot of wit on this side of Twitter. I'm impressed.AARON: I agree.SARAH: So I think that's a very good idea.AARON: Okay. Dear Events team: hire Aaron Bergman, professional comedian.SARAH: You can just give them your Twitter as a source for how funny you are, and that clearly qualifies you to set this up. I love it.AARON: This is not important or related to anything, but I used to be a good juggler for entertainment purposes. I have this video. Maybe I should make sure the world can see it. It's like a talent show. So maybe I can do that instead.SARAH: Juggling. You definitely should make sure the world has access to this footage.AARON: It had more views than I expected. It wasn't five views. It was 90 or something, which is still nothing.SARAH: I can tell you a secret right now if you want. That relates to Max asking in the chat about glee.AARON: Yes.SARAH: This bit will also have to edit out, but me having a public meltdown over AI was the second time that I've ever blown up on the Internet. The first time being. I can't believe I'm telling you this. I think I'm delirious right now. Were you ever in any fandoms, as a teenager?AARON: No.SARAH: Okay. Were you ever on Tumblr?AARON: No. I sort of know what the cultural vibes were. I sort of know what you're referring to. There are people who like Harry Potter stuff and bands, like Kpop stuff like that.SARAH: So people would make these fan videos where they'd take clips from TV shows and then they edit them together to music. Sometimes people would edit the clips to make it look like something had happened in the plot of the show that hadn't actually happened. For example, say, what if X character had died? And then you edit the clips together to try and make it look like they've died. And you put a sad song, how to save a life by the fray or something, over the top. And then you put it on YouTube.AARON: Sorry, tell me what…"Hat I should search or just send the link here. I'm sending my link.SARAH: Oh, no, this doesn't exist anymore. It does not exist anymore. Right? So, say if you're, like, eleven or twelve years old and you do this, and you don't even have a mechanism to download videos because you don't know how to do technology. Instead, you take your little iPod touch and you just play a YouTube video on your screen, and you literally just film the screen with your iPod touch, and that's how you're getting the clips. It's kind of shaky because you're holding the camera anyway.SARAH: Then you edit together on the iMovie app of your iPod touch, and then you put it on the Internet, and then you just forget about it. You forget about it. Two years later, you're like, oh, I wonder what happened to that YouTube account? And you log in and this little video that you've made with edited clips that you've filmed off the screen of your laptop to ‘How To Save Life' by The Fray with clips from Glee in it, has nearly half a million views.AARON: Nice. Love it.SARAH: Embarrassing because this is like, two years later. And then all the comments were like, oh, my God, this was so moving. This made me cry. And then obviously, some of them were hating and being like, do you not even know how to download video clips? Like, what? And then you're so embarrassed.AARON: I could totally seem it. Creative, but only a reasonable solution. Yeah.SARAH: So that's my story of how I went viral when I was like, twelve.AARON: It must have been kind of overwhelming.SARAH: Yeah, it was a bit. And you can tell that my time, it's like 20 to eleven at night, and now I'm starting to really go off on one and talk about weird things.AARON: Like an hour. So, yeah, we can wrap up. And I always say this, but it's actually true. Which is that low standard, like, low stakes or low threshold. Low bar for doing that in recording some of the time.SARAH: Yeah, probably. We'll have to get rid of the part about how I went viral on YouTube when I was twelve. I'll sleep on that.AARON: Don't worry. I'll send the transcription at some point soon.SARAH: Yeah, cool.AARON: Okay, lovely. Thank you for staying up late into the night for this.SARAH: It's not that late into the night. I'm just like, lame and go to bed early.AARON: Okay, cool. Yeah, I know. Yeah, for sure. All right, bye. Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

* Follow ⁠Max on Twitter⁠* And read his ⁠blog⁠* Listen here or on Spotify or Apple Podcasts * RIP Google Podcasts

* Listen on Spotify or Apple Podcasts* Be sure to check out and follow Holly's Substack and org Pause AI. Blurb and summary from ClongBlurbHolly and Aaron had a wide-ranging discussion touching on effective altruism, AI alignment, genetic conflict, wild animal welfare, and the importance of public advocacy in the AI safety space. Holly spoke about her background in evolutionary biology and how she became involved in effective altruism. She discussed her reservations around wild animal welfare and her perspective on the challenges of AI alignment. They talked about the value of public opinion polls, the psychology of AI researchers, and whether certain AI labs like OpenAI might be net positive actors. Holly argued for the strategic importance of public advocacy and pushing the Overton window within EA on AI safety issues.Detailed summary* Holly's background - PhD in evolutionary biology, got into EA through New Atheism and looking for community with positive values, did EA organizing at Harvard* Worked at Rethink Priorities on wild animal welfare but had reservations about imposing values on animals and whether we're at the right margin yet* Got inspired by FLI letter to focus more on AI safety advocacy and importance of public opinion* Discussed genetic conflict and challenges of alignment even with "closest" agents* Talked about the value of public opinion polls and influencing politicians* Discussed the psychology and motives of AI researchers* Disagreed a bit on whether certain labs like OpenAI might be net positive actors* Holly argued for importance of public advocacy in AI safety, thinks we have power to shift Overton window* Talked about the dynamics between different AI researchers and competition for status* Discussed how rationalists often dismiss advocacy and politics* Holly thinks advocacy is neglected and can push the Overton window even within EA* Also discussed Holly's evolutionary biology takes, memetic drive, gradient descent vs. natural selectionFull transcript (very imperfect)AARONYou're an AI pause, Advocate. Can you remind me of your shtick before that? Did you have an EA career or something?HOLLYYeah, before that I was an academic. I got into EA when I was doing my PhD in evolutionary biology, and I had been into New Atheism before that. I had done a lot of organizing for that in college. And while the enlightenment stuff and what I think is the truth about there not being a God was very important to me, but I didn't like the lack of positive values. Half the people there were sort of people like me who are looking for community after leaving their religion that they grew up in. And sometimes as many as half of the people there were just looking for a way for it to be okay for them to upset people and take away stuff that was important to them. And I didn't love that. I didn't love organizing a space for that. And when I got to my first year at Harvard, harvard Effective Altruism was advertising for its fellowship, which became the Elite Fellowship eventually. And I was like, wow, this is like, everything I want. And it has this positive organizing value around doing good. And so I was totally made for it. And pretty much immediately I did that fellowship, even though it was for undergrad. I did that fellowship, and I was immediately doing a lot of grad school organizing, and I did that for, like, six more years. And yeah, by the time I got to the end of grad school, I realized I was very sick in my fifth year, and I realized the stuff I kept doing was EA organizing, and I did not want to keep doing work. And that was pretty clear. I thought, oh, because I'm really into my academic area, I'll do that, but I'll also have a component of doing good. I took giving what we can in the middle of grad school, and I thought, I actually just enjoy doing this more, so why would I do anything else? Then after grad school, I started applying for EA jobs, and pretty soon I got a job at Rethink Priorities, and they suggested that I work on wild animal welfare. And I have to say, from the beginning, it was a little bit like I don't know, I'd always had very mixed feelings about wild animal welfare as a cause area. How much do they assume the audience knows about EA?AARONA lot, I guess. I think as of right now, it's a pretty hardcore dozen people. Also. Wait, what year is any of this approximately?HOLLYSo I graduated in 2020.AARONOkay.HOLLYYeah. And then I was like, really?AARONOkay, this is not extremely distant history. Sometimes people are like, oh, yeah, like the OG days, like four or something. I'm like, oh, my God.HOLLYOh, yeah, no, I wish I had been in these circles then, but no, it wasn't until like, 2014 that I really got inducted. Yeah, which now feels old because everybody's so young. But yeah, in 2020, I finished my PhD, and I got this awesome remote job at Rethink Priorities during the Pandemic, which was great, but I was working on wild animal welfare, which I'd always had some. So wild animal welfare, just for anyone who's not familiar, is like looking at the state of the natural world and seeing if there's a way that usually the hedonic so, like, feeling pleasure, not pain sort of welfare of animals can be maximized. So that's in contrast to a lot of other ways of looking at the natural world, like conservation, which are more about preserving a state of the world the way preserving, maybe ecosystem balance, something like that. Preserving species diversity. The priority with wild animal welfare is the effect of welfare, like how it feels to be the animals. So it is very understudied, but I had a lot of reservations about it because I'm nervous about maximizing our values too hard onto animals or imposing them on other species.AARONOkay, that's interesting, just because we're so far away from the margin of I'm like a very pro wild animal animal welfare pilled person.HOLLYI'm definitely pro in theory.AARONHow many other people it's like you and formerly you and six other people or whatever seems like we're quite far away from the margin at which we're over optimizing in terms of giving heroin to all the sheep or I don't know, the bugs and stuff.HOLLYBut it's true the field is moving in more my direction and I think it's just because they're hiring more biologists and we tend to think this way or have more of this perspective. But I'm a big fan of Brian domestics work. But stuff like finding out which species have the most capacity for welfare I think is already sort of the wrong scale. I think a lot will just depend on how much. What are the conditions for that species?AARONYeah, no, there's like seven from the.HOLLYCoarseness and the abstraction, but also there's a lot of you don't want anybody to actually do stuff like that and it would be more possible to do the more simple sounding stuff. My work there just was consisted of being a huge downer. I respect that. I did do some work that I'm proud of. I have a whole sequence on EA forum about how we could reduce the use of rodenticide, which I think was the single most promising intervention that we came up with in the time that I was there. I mean, I didn't come up with it, but that we narrowed down. And even that just doesn't affect that many animals directly. It's really more about the impact is from what you think you'll get with moral circle expansion or setting precedents for the treatment of non human animals or wild animals, or semi wild animals, maybe like being able to be expanded into wild animals. And so it all felt not quite up to EA standards of impact. And I felt kind of uncomfortable trying to make this thing happen in EA when I wasn't sure that my tentative conclusion on wild animal welfare, after working on it and thinking about it a lot for three years, was that we're sort of waiting for transformative technology that's not here yet in order to be able to do the kinds of interventions that we want. And there are going to be other issues with the transformative technology that we have to deal with first.AARONYeah, no, I've been thinking not that seriously or in any formal way, just like once in a while I just have a thought like oh, I wonder how the field of, like, I guess wild animal sorry, not wild animal. Just like animal welfare in general and including wild animal welfare might make use of AI above and beyond. I feel like there's like a simple take which is probably mostly true, which is like, oh, I mean the phrase that everybody loves to say is make AI go well or whatever that but that's basically true. Probably you make aligned AI. I know that's like a very oversimplification and then you can have a bunch of wealth or whatever to do whatever you want. I feel like that's kind of like the standard line, but do you have any takes on, I don't know, maybe in the next couple of years or anything more specifically beyond just general purpose AI alignment, for lack of a better term, how animal welfare might put to use transformative AI.HOLLYMy last work at Rethink Priorities was like looking a sort of zoomed out look at the field and where it should go. And so we're apparently going to do a public version, but I don't know if that's going to happen. It's been a while now since I was expecting to get a call about it. But yeah, I'm trying to think of what can I scrape from that?AARONAs much as you can, don't reveal any classified information. But what was the general thing that this was about?HOLLYThere are things that I think so I sort of broke it down into a couple of categories. There's like things that we could do in a world where we don't get AGI for a long time, but we get just transformative AI. Short of that, it's just able to do a lot of parallel tasks. And I think we could do a lot we could get a lot of what we want for wild animals by doing a ton of surveillance and having the ability to make incredibly precise changes to the ecosystem. Having surveillance so we know when something is like, and the capacity to do really intense simulation of the ecosystem and know what's going to happen as a result of little things. We could do that all without AGI. You could just do that with just a lot of computational power. I think our ability to simulate the environment right now is not the best, but it's not because it's impossible. It's just like we just need a lot more observations and a lot more ability to simulate a comparison is meteorology. Meteorology used to be much more of an art, but it became more of a science once they started just literally taking for every block of air and they're getting smaller and smaller, the blocks. They just do Bernoulli's Law on it and figure out what's going to happen in that block. And then you just sort of add it all together and you get actually pretty good.AARONDo you know how big the blocks are?HOLLYThey get smaller all the time. That's the resolution increase, but I don't know how big the blocks are okay right now. And shockingly, that just works. That gives you a lot of the picture of what's going to happen with weather. And I think that modeling ecosystem dynamics is very similar to weather. You could say more players than ecosystems, and I think we could, with enough surveillance, get a lot better at monitoring the ecosystem and then actually have more of a chance of implementing the kinds of sweeping interventions we want. But the price would be just like never ending surveillance and having to be the stewards of the environment if we weren't automating. Depending on how much you want to automate and depending on how much you can automate without AGI or without handing it over to another intelligence.AARONYeah, I've heard this. Maybe I haven't thought enough. And for some reason, I'm just, like, intuitively. I feel like I'm more skeptical of this kind of thing relative to the actual. There's a lot of things that I feel like a person might be skeptical about superhuman AI. And I'm less skeptical of that or less skeptical of things that sound as weird as this. Maybe because it's not. One thing I'm just concerned about is I feel like there's a larger scale I can imagine, just like the choice of how much, like, ecosystem is like yeah, how much ecosystem is available for wild animals is like a pretty macro level choice that might be not at all deterministic. So you could imagine spreading or terraforming other planets and things like that, or basically continuing to remove the amount of available ecosystem and also at a much more practical level, clean meat development. I have no idea what the technical bottlenecks on that are right now, but seems kind of possible that I don't know, AI can help it in some capacity.HOLLYOh, I thought you're going to say that it would increase the amount of space available for wild animals. Is this like a big controversy within, I don't know, this part of the EA animal movement? If you advocate diet change and if you get people to be vegetarians, does that just free up more land for wild animals to suffer on? I thought this was like, guys, we just will never do anything if we don't choose sort of like a zone of influence and accomplish something there. It seemed like this could go on forever. It was like, literally, I rethink actually. A lot of discussions would end in like, okay, so this seems like really good for all of our target populations, but what about wild animals? I could just reverse everything. I don't know. The thoughts I came to on that were that it is worthwhile to try to figure out what are all of the actual direct effects, but I don't think we should let that guide our decision making. Only you have to have some kind of theory of change, of what is the direct effect going to lead to? And I just think that it's so illegible what you're trying to do. If you're, like, you should eat this kind of fish to save animals. It doesn't lead society to adopt, to understand and adopt your values. It's so predicated on a moment in time that might be convenient. Maybe I'm not looking hard enough at that problem, but the conclusion I ended up coming to was just like, look, I just think we have to have some idea of not just the direct impacts, but something about the indirect impacts and what's likely to facilitate other direct impacts that we want in the future.AARONYeah. I also share your I don't know. I'm not sure if we share the same or I also feel conflicted about this kind of thing. Yeah. And I don't know, at the very least, I have a very high bar for saying, actually the worst of factory farming is like, we should just like, yeah, we should be okay with that, because some particular model says that at this moment in time, it has some net positive effect on animal welfare.HOLLYWhat morality is that really compatible with? I mean, I understand our morality, but maybe but pretty much anyone else who hears that conclusion is going to think that that means that the suffering doesn't matter or something.AARONYeah, I don't know. I think maybe more than you, I'm willing to bite the bullet if somebody really could convince me that, yeah, chicken farming is actually just, in fact, good, even though it's counterintuitive, I'll be like, all right, fine.HOLLYSurely there are other ways of occupying.AARONYeah.HOLLYSame with sometimes I would get from very classical wild animal suffering people, like, comments on my rodenticide work saying, like, well, what if it's good to have more rats? I don't know. There are surely other vehicles for utility other than ones that humans are bent on destroying.AARONYeah, it's kind of neither here nor there, but I don't actually know if this is causally important, but at least psychologically. I remember seeing a mouse in a glue trap was very had an impact on me from maybe turning me, like, animal welfare pills or something. That's like, neither here nor there. It's like a random anecdote, but yeah, seems bad. All right, what came after rethink for you?HOLLYYeah. Well, after the publication of the FLI Letter and Eliezer's article in Time, I was super inspired by pause. A number of emotional changes happened to me about AI safety. Nothing intellectual changed, but just I'd always been confused at and kind of taken it as a sign that people weren't really serious about AI risk when they would say things like, I don't know, the only option is alignment. The only option is for us to do cool, nerd stuff that we love doing nothing else would. I bought the arguments, but I just wasn't there emotionally. And seeing Eliezer advocate political change because he wants to save everyone's lives and he thinks that's something that we can do. Just kind of I'm sure I didn't want to face it before because it was upsetting. Not that I haven't faced a lot of upsetting and depressing things like I worked in wild animal welfare, for God's sake, but there was something that didn't quite add up for me, or I hadn't quite grocked about AI safety until seeing Eliezer really show that his concern is about everyone dying. And he's consistent with that. He's not caught on only one way of doing it, and it just kind of got in my head and I kept wanting to talk about it at work and it sort of became clear like they weren't going to pursue that sort of intervention. But I kept thinking of all these parallels between animal advocacy stuff that I knew and what could be done in AI safety. And these polls kept coming out showing that there was really high support for Paws and I just thought, this is such a huge opportunity, I really would love to help out. Originally I was looking around for who was going to be leading campaigns that I could volunteer in, and then eventually I thought, it just doesn't seem like somebody else is going to do this in the Bay Area. So I just ended up quitting rethink and being an independent organizer. And that has been really I mean, honestly, it's like a tough subject. It's like a lot to deal with, but honestly, compared to wild animal welfare, it's not that bad. And I think I'm pretty used to dealing with tough and depressing low tractability causes, but I actually think this is really tractable. I've been shocked how quickly things have moved and I sort of had this sense that, okay, people are reluctant in EA and AI safety in particular, they're not used to advocacy. They kind of vaguely think that that's bad politics is a mind killer and it's a little bit of a threat to the stuff they really love doing. Maybe that's not going to be so ascendant anymore and it's just stuff they're not familiar with. But I have the feeling that if somebody just keeps making this case that people will take to it, that I could push the Oberson window with NEA and that's gone really well.AARONYeah.HOLLYAnd then of course, the public is just like pretty down. It's great.AARONYeah. I feel like it's kind of weird because being in DC and I've always been, I feel like I actually used to be more into politics, to be clear. I understand or correct me if I'm wrong, but advocacy doesn't just mean in the political system or two politicians or whatever, but I assume that's like a part of what you're thinking about or not really.HOLLYYeah. Early on was considering working on more political process type advocacy and I think that's really important. I totally would have done it. I just thought that it was more neglected in our community to do advocacy to the public and a lot of people had entanglements that prevented them from doing so. They work sort of with AI labs or it's important to their work that they not declare against AI labs or something like that or be perceived that way. And so they didn't want to do public advocacy that could threaten what else they're doing. But I didn't have anything like that. I've been around for a long time in EA and I've been keeping up on AI safety, but I've never really worked. That's not true. I did a PiBBs fellowship, but.AARONI've.HOLLYNever worked for anybody in like I was just more free than a lot of other people to do the public messaging and so I kind of felt that I should. Yeah, I'm also more willing to get into conflict than other EA's and so that seems valuable, no?AARONYeah, I respect that. Respect that a lot. Yeah. So like one thing I feel like I've seen a lot of people on Twitter, for example. Well, not for example. That's really just it, I guess, talking about polls that come out saying like, oh yeah, the public is super enthusiastic about X, Y or Z, I feel like these are almost meaningless and maybe you can convince me otherwise. It's not exactly to be clear, I'm not saying that. I guess it could always be worse, right? All things considered, like a poll showing X thing is being supported is better than the opposite result, but you can really get people to say anything. Maybe I'm just wondering about the degree to which the public how do you imagine the public and I'm doing air quotes to playing into policies either of, I guess, industry actors or government actors?HOLLYWell, this is something actually that I also felt that a lot of EA's were unfamiliar with. But it does matter to our representatives, like what the constituents think it matters a mean if you talk to somebody who's ever interned in a congressperson's office, one person calling and writing letters for something can have actually depending on how contested a policy is, can have a largeish impact. My ex husband was an intern for Jim Cooper and they had this whole system for scoring when calls came in versus letters. Was it a handwritten letter, a typed letter? All of those things went into how many points it got and that was something they really cared about. Politicians do pay attention to opinion polls and they pay attention to what their vocal constituents want and they pay attention to not going against what is the norm opinion. Even if nobody in particular is pushing them on it or seems to feel strongly about it. They really are trying to calibrate themselves to what is the norm. So those are always also sometimes politicians just get directly convinced by arguments of what a policy should be. So yeah, public opinion is, I think, underappreciated by ya's because it doesn't feel like mechanistic. They're looking more for what's this weird policy hack that's going to solve what's? This super clever policy that's going to solve things rather than just like what's acceptable discourse, like how far out of his comfort zone does this politician have to go to advocate for this thing? How unpopular is it going to be to say stuff that's against this thing that now has a lot of public support?AARONYeah, I guess mainly I'm like I guess I'm also I definitely could be wrong with this, but I would expect that a lot of the yeah, like for like when politicians like, get or congresspeople like, get letters and emails or whatever on a particular especially when it's relevant to a particular bill. And it's like, okay, this bill has already been filtered for the fact that it's going to get some yes votes and some no votes and it's close to or something like that. Hearing from an interested constituency is really, I don't know, I guess interesting evidence. On the other hand, I don't know, you can kind of just get Americans to say a lot of different things that I think are basically not extremely unlikely to be enacted into laws. You know what I mean? I don't know. You can just look at opinion. Sorry. No great example comes to mind right now. But I don't know, if you ask the public, should we do more safety research into, I don't know, anything. If it sounds good, then people will say yes, or am I mistaken about this?HOLLYI mean, on these polls, usually they ask the other way around as well. Do you think AI is really promising for its benefits and should be accelerated? They answer consistently. It's not just like, well now that sounds positive. Okay. I mean, a well done poll will correct for these things. Yeah. I've encountered a lot of skepticism about the polls. Most of the polls on this have been done by YouGov, which is pretty reputable. And then the ones that were replicated by rethink priorities, they found very consistent results and I very much trust Rethink priorities on polls. Yeah. I've had people say, well, these framings are I don't know, they object and wonder if it's like getting at the person's true beliefs. And I kind of think like, I don't know, basically this is like the kind of advocacy message that I would give and people are really receptive to it. So to me that's really promising. Whether or not if you educated them a lot more about the topic, they would think the same is I don't think the question but that's sometimes an objection that I get. Yeah, I think they're indicative. And then I also think politicians just care directly about these things. If they're able to cite that most of the public agrees with this policy, that sort of gives them a lot of what they want, regardless of whether there's some qualification to does the public really think this or are they thinking hard enough about it? And then polls are always newsworthy. Weirdly. Just any poll can be a news story and journalists love them and so it's a great chance to get exposure for the whatever thing. And politicians do care what's in the news. Actually, I think we just have more influence over the political process than EA's and less wrongers tend to believe it's true. I think a lot of people got burned in AI safety, like in the previous 20 years because it would be dismissed. It just wasn't in the overton window. But I think we have a lot of power now. Weirdly. People care what effective altruists think. People see us as having real expertise. The AI safety community does know the most about this. It's pretty wild now that's being recognized publicly and journalists and the people who influence politicians, not directly the people, but the Fourth Estate type, people pay attention to this and they influence policy. And there's many levels of I wrote if people want a more detailed explanation of this, but still high level and accessible, I hope I wrote a thing on EA forum called The Case for AI Safety Advocacy. And that kind of goes over this concept of outside versus inside game. So inside game is like working within a system to change it. Outside game is like working outside the system to put pressure on that system to change it. And I think there's many small versions of this. I think that it's helpful within EA and AI safety to be pushing the overton window of what I think that people have a wrong understanding of how hard it is to communicate this topic and how hard it is to influence governments. I want it to be more acceptable. I want it to feel more possible in EA and AI safety to go this route. And then there's the public public level of trying to make them more familiar with the issue, frame it in the way that I want, which is know, with Sam Altman's tour, the issue kind of got framed as like, well, AI is going to get built, but how are we going to do it safely? And then I would like to take that a step back and be like, should AI be built or should AGI be just if we tried, we could just not do that, or we could at least reduce the speed. And so, yeah, I want people to be exposed to that frame. I want people to not be taken in by other frames that don't include the full gamut of options. I think that's very possible. And then there's a lot of this is more of the classic thing that's been going on in AI safety for the last ten years is trying to influence AI development to be more safety conscious. And that's like another kind of dynamic. There, like trying to change sort of the general flavor, like, what's acceptable? Do we have to care about safety? What is safety? That's also kind of a window pushing exercise.AARONYeah. Cool. Luckily, okay, this is not actually directly responding to anything you just said, which is luck. So I pulled up this post. So I should have read that. Luckily, I did read the case for slowing down. It was like some other popular post as part of the, like, governance fundamentals series. I think this is by somebody, Zach wait, what was it called? Wait.HOLLYIs it by Zach or.AARONKatya, I think yeah, let's think about slowing down AI. That one. So that is fresh in my mind, but yours is not yet. So what's the plan? Do you have a plan? You don't have to have a plan. I don't have plans very much.HOLLYWell, right now I'm hopeful about the UK AI summit. Pause AI and I have planned a multi city protest on the 21 October to encourage the UK AI Safety Summit to focus on safety first and to have as a topic arranging a pause or that of negotiation. There's a lot of a little bit upsetting advertising for that thing that's like, we need to keep up capabilities too. And I just think that's really a secondary objective. And that's how I wanted to be focused on safety. So I'm hopeful about the level of global coordination that we're already seeing. It's going so much faster than we thought. Already the UN Secretary General has been talking about this and there have been meetings about this. It's happened so much faster at the beginning of this year. Nobody thought we could talk about nobody was thinking we'd be talking about this as a mainstream topic. And then actually governments have been very receptive anyway. So right now I'm focused on other than just influencing opinion, the targets I'm focused on, or things like encouraging these international like, I have a protest on Friday, my first protest that I'm leading and kind of nervous that's against Meta. It's at the Meta building in San Francisco about their sharing of model weights. They call it open source. It's like not exactly open source, but I'm probably not going to repeat that message because it's pretty complicated to explain. I really love the pause message because it's just so hard to misinterpret and it conveys pretty clearly what we want very quickly. And you don't have a lot of bandwidth and advocacy. You write a lot of materials for a protest, but mostly what people see is the title.AARONThat's interesting because I sort of have the opposite sense. I agree that in terms of how many informational bits you're conveying in a particular phrase, pause AI is simpler, but in some sense it's not nearly as obvious. At least maybe I'm more of a tech brain person or whatever. But why that is good, as opposed to don't give extremely powerful thing to the worst people in the world. That's like a longer everyone.HOLLYMaybe I'm just weird. I've gotten the feedback from open source ML people is the number one thing is like, it's too late, there's already super powerful models. There's nothing you can do to stop us, which sounds so villainous, I don't know if that's what they mean. Well, actually the number one message is you're stupid, you're not an ML engineer. Which like, okay, number two is like, it's too late, there's nothing you can do. There's all of these other and Meta is not even the most powerful generator of models that it share of open source models. I was like, okay, fine. And I don't know, I don't think that protesting too much is really the best in these situations. I just mostly kind of let that lie. I could give my theory of change on this and why I'm focusing on Meta. Meta is a large company I'm hoping to have influence on. There is a Meta building in San Francisco near where yeah, Meta is the biggest company that is doing this and I think there should be a norm against model weight sharing. I was hoping it would be something that other employees of other labs would be comfortable attending and that is a policy that is not shared across the labs. Obviously the biggest labs don't do it. So OpenAI is called OpenAI but very quickly decided not to do that. Yeah, I kind of wanted to start in a way that made it more clear than pause AI. Does that anybody's welcome something? I thought a one off issue like this that a lot of people could agree and form a coalition around would be good. A lot of people think that this is like a lot of the open source ML people think know this is like a secret. What I'm saying is secretly an argument for tyranny. I just want centralization of power. I just think that there are elites that are better qualified to run everything. It was even suggested I didn't mention China. It even suggested that I was racist because I didn't think that foreign people could make better AIS than Meta.AARONI'm grimacing here. The intellectual disagreeableness, if that's an appropriate term or something like that. Good on you for standing up to some pretty bad arguments.HOLLYYeah, it's not like that worth it. I'm lucky that I truly am curious about what people think about stuff like that. I just find it really interesting. I spent way too much time understanding the alt. Right. For instance, I'm kind of like sure I'm on list somewhere because of the forums I was on just because I was interested and it is something that serves me well with my adversaries. I've enjoyed some conversations with people where I kind of like because my position on all this is that look, I need to be convinced and the public needs to be convinced that this is safe before we go ahead. So I kind of like not having to be the smart person making the arguments. I kind of like being like, can you explain like I'm five. I still don't get it. How does this work?AARONYeah, no, I was thinking actually not long ago about open source. Like the phrase has such a positive connotation and in a lot of contexts it really is good. I don't know. I'm glad that random tech I don't know, things from 2004 or whatever, like the reddit source code is like all right, seems cool that it's open source. I don't actually know if that was how that right. But yeah, I feel like maybe even just breaking down what the positive connotation comes from and why it's in people's self. This is really what I was thinking about, is like, why is it in people's self interest to open source things that they made and that might break apart the allure or sort of ethical halo that it has around it? And I was thinking it probably has something to do with, oh, this is like how if you're a tech person who makes some cool product, you could try to put a gate around it by keeping it closed source and maybe trying to get intellectual property or something. But probably you're extremely talented already, or pretty wealthy. Definitely can be hired in the future. And if you're not wealthy yet I don't mean to put things in just materialist terms, but basically it could easily be just like in a yeah, I think I'll probably take that bit out because I didn't mean to put it in strictly like monetary terms, but basically it just seems like pretty plausibly in an arbitrary tech person's self interest, broadly construed to, in fact, open source their thing, which is totally fine and normal.HOLLYI think that's like 99 it's like a way of showing magnanimity showing, but.AARONI don't make this sound so like, I think 99.9% of human behavior is like this. I'm not saying it's like, oh, it's some secret, terrible self interested thing, but just making it more mechanistic. Okay, it's like it's like a status thing. It's like an advertising thing. It's like, okay, you're not really in need of direct economic rewards, or sort of makes sense to play the long game in some sense, and this is totally normal and fine, but at the end of the day, there's reasons why it makes sense, why it's in people's self interest to open source.HOLLYLiterally, the culture of open source has been able to bully people into, like, oh, it's immoral to keep it for yourself. You have to release those. So it's just, like, set the norms in a lot of ways, I'm not the bully. Sounds bad, but I mean, it's just like there is a lot of pressure. It looks bad if something is closed source.AARONYeah, it's kind of weird that Meta I don't know, does Meta really think it's in their I don't know. Most economic take on this would be like, oh, they somehow think it's in their shareholders interest to open source.HOLLYThere are a lot of speculations on why they're doing this. One is that? Yeah, their models aren't as good as the top labs, but if it's open source, then open source quote, unquote then people will integrate it llama Two into their apps. Or People Will Use It And Become I don't know, it's a little weird because I don't know why using llama Two commits you to using llama Three or something, but it just ways for their models to get in in places where if you just had to pay for their models too, people would go for better ones. That's one thing. Another is, yeah, I guess these are too speculative. I don't want to be seen repeating them since I'm about to do this purchase. But there's speculation that it's in best interests in various ways to do this. I think it's possible also that just like so what happened with the release of Llama One is they were going to allow approved people to download the weights, but then within four days somebody had leaked Llama One on four chan and then they just were like, well, whatever, we'll just release the weights. And then they released Llama Two with the weights from the beginning. And it's not like 100% clear that they intended to do full open source or what they call Open source. And I keep saying it's not open source because this is like a little bit of a tricky point to make. So I'm not emphasizing it too much. So they say that they're open source, but they're not. The algorithms are not open source. There are open source ML models that have everything open sourced and I don't think that that's good. I think that's worse. So I don't want to criticize them for that. But they're saying it's open source because there's all this goodwill associated with open source. But actually what they're doing is releasing the product for free or like trade secrets even you could say like things that should be trade secrets. And yeah, they're telling people how to make it themselves. So it's like a little bit of a they're intentionally using this label that has a lot of positive connotations but probably according to Open Source Initiative, which makes the open Source license, it should be called something else or there should just be like a new category for LLMs being but I don't want things to be more open. It could easily sound like a rebuke that it should be more open to make that point. But I also don't want to call it Open source because I think Open source software should probably does deserve a lot of its positive connotation, but they're not releasing the part, that the software part because that would cut into their business. I think it would be much worse. I think they shouldn't do it. But I also am not clear on this because the Open Source ML critics say that everyone does have access to the same data set as Llama Two. But I don't know. Llama Two had 7 billion tokens and that's more than GPT Four. And I don't understand all of the details here. It's possible that the tokenization process was different or something and that's why there were more. But Meta didn't say what was in the longitude data set and usually there's some description given of what's in the data set that led some people to speculate that maybe they're using private data. They do have access to a lot of private data that shouldn't be. It's not just like the common crawl backup of the Internet. Everybody's basing their training on that and then maybe some works of literature they're not supposed to. There's like a data set there that is in question, but metas is bigger than bigger than I think well, sorry, I don't have a list in front of me. I'm not going to get stuff wrong, but it's bigger than kind of similar models and I thought that they have access to extra stuff that's not public. And it seems like people are asking if maybe that's part of the training set. But yeah, the ML people would have or the open source ML people that I've been talking to would have believed that anybody who's decent can just access all of the training sets that they've all used.AARONAside, I tried to download in case I'm guessing, I don't know, it depends how many people listen to this. But in one sense, for a competent ML engineer, I'm sure open source really does mean that. But then there's people like me. I don't know. I knew a little bit of R, I think. I feel like I caught on the very last boat where I could know just barely enough programming to try to learn more, I guess. Coming out of college, I don't know, a couple of months ago, I tried to do the thing where you download Llama too, but I tried it all and now I just have like it didn't work. I have like a bunch of empty folders and I forget got some error message or whatever. Then I tried to train my own tried to train my own model on my MacBook. It just printed. That's like the only thing that a language model would do because that was like the most common token in the training set. So anyway, I'm just like, sorry, this is not important whatsoever.HOLLYYeah, I feel like torn about this because I used to be a genomicist and I used to do computational biology and it was not machine learning, but I used a highly parallel GPU cluster. And so I know some stuff about it and part of me wants to mess around with it, but part of me feels like I shouldn't get seduced by this. I am kind of worried that this has happened in the AI safety community. It's always been people who are interested in from the beginning, it was people who are interested in singularity and then realized there was this problem. And so it's always been like people really interested in tech and wanting to be close to it. And I think we've been really influenced by our direction, has been really influenced by wanting to be where the action is with AI development. And I don't know that that was right.AARONNot personal, but I guess individual level I'm not super worried about people like you and me losing the plot by learning more about ML on their personal.HOLLYYou know what I mean? But it does just feel sort of like I guess, yeah, this is maybe more of like a confession than, like a point. But it does feel a little bit like it's hard for me to enjoy in good conscience, like, the cool stuff.AARONOkay. Yeah.HOLLYI just see people be so attached to this as their identity. They really don't want to go in a direction of not pursuing tech because this is kind of their whole thing. And what would they do if we weren't working toward AI? This is a big fear that people express to me with they don't say it in so many words usually, but they say things like, well, I don't want AI to never get built about a pause. Which, by the way, just to clear up, my assumption is that a pause would be unless society ends for some other reason, that a pause would eventually be lifted. It couldn't be forever. But some people are worried that if you stop the momentum now, people are just so luddite in their insides that we would just never pick it up again. Or something like that. And, yeah, there's some identity stuff that's been expressed. Again, not in so many words to me about who will we be if we're just sort of like activists instead of working on.AARONMaybe one thing that we might actually disagree on. It's kind of important is whether so I think we both agree that Aipause is better than the status quo, at least broadly, whatever. I know that can mean different things, but yeah, maybe I'm not super convinced, actually, that if I could just, like what am I trying to say? Maybe at least right now, if I could just imagine the world where open eye and Anthropic had a couple more years to do stuff and nobody else did, that would be better. I kind of think that they are reasonably responsible actors. And so I don't know. I don't think that actually that's not an actual possibility. But, like, maybe, like, we have a different idea about, like, the degree to which, like, a problem is just, like, a million different not even a million, but, say, like, a thousand different actors, like, having increasingly powerful models versus, like, the actual, like like the actual, like, state of the art right now, being plausibly near a dangerous threshold or something. Does this make any sense to you?HOLLYBoth those things are yeah, and this is one thing I really like about the pause position is that unlike a lot of proposals that try to allow for alignment, it's not really close to a bad choice. It's just more safe. I mean, it might be foregoing some value if there is a way to get an aligned AI faster. But, yeah, I like the pause position because it's kind of robust to this. I can't claim to know more about alignment than OpenAI or anthropic staff. I think they know much more about it. But I have fundamental doubts about the concept of alignment that make me think I'm concerned about even if things go right, like, what perverse consequences go nominally right, like, what perverse consequences could follow from that. I have, I don't know, like a theory of psychology that's, like, not super compatible with alignment. Like, I think, like yeah, like humans in living in society together are aligned with each other, but the society is a big part of that. The people you're closest to are also my background in evolutionary biology has a lot to do with genetic conflict.AARONWhat is that?HOLLYGenetic conflict is so interesting. Okay, this is like the most fascinating topic in biology, but it's like, essentially that in a sexual species, you're related to your close family, you're related to your ken, but you're not the same as them. You have different interests. And mothers and fathers of the same children have largely overlapping interests, but they have slightly different interests in what happens with those children. The payoff to mom is different than the payoff to dad per child. One of the classic genetic conflict arenas and one that my advisor worked on was my advisor was David Haig, was pregnancy. So mom and dad both want an offspring that's healthy. But mom is thinking about all of her offspring into the future. When she thinks about how much.AARONWhen.HOLLYMom is giving resources to one baby, that is in some sense depleting her ability to have future children. But for dad, unless the species is.AARONPerfect, might be another father in the future.HOLLYYeah, it's in his interest to take a little more. And it's really interesting. Like the tissues that the placenta is an androgenetic tissue. This is all kind of complicated. I'm trying to gloss over some details, but it's like guided more by genes that are active in when they come from the father, which there's this thing called genomic imprinting that first, and then there's this back and forth. There's like this evolution between it's going to serve alleles that came from dad imprinted, from dad to ask for more nutrients, even if that's not good for the mother and not what the mother wants. So the mother's going to respond. And you can see sometimes alleles are pretty mismatched and you get like, mom's alleles want a pretty big baby and a small placenta. So sometimes you'll see that and then dad's alleles want a big placenta and like, a smaller baby. These are so cool, but they're so hellishly complicated to talk about because it involves a bunch of genetic concepts that nobody talks about for any other reason.AARONI'm happy to talk about that. Maybe part of that dips below or into the weeds threshold, which I've kind of lost it, but I'm super interested in this stuff.HOLLYYeah, anyway, so the basic idea is just that even the people that you're closest with and cooperate with the most, they tend to be clearly this is predicated on our genetic system. There's other and even though ML sort of evolves similarly to natural selection through gradient descent, it doesn't have the same there's no recombination, there's not genes, so there's a lot of dis analogies there. But the idea that being aligned to our psychology would just be like one thing. Our psychology is pretty conditional. I would agree that it could be one thing if we had a VNM utility function and you could give it to AGI, I would think, yes, that captures it. But even then, that utility function, it covers when you're in conflict with someone, it covers different scenarios. And so I just am like not when people say alignment. I think what they're imagining is like an omniscient. God, who knows what would be best? And that is different than what I think could be meant by just aligning values.AARONNo, I broadly very much agree, although I do think at least this is my perception, is that based on the right 95 to 2010 Miri corpus or whatever, alignment was like alignment meant something that was kind of not actually possible in the way that you're saying. But now that we have it seems like actually humans have been able to get ML models to understand basically human language pretty shockingly. Well, and so actually, just the concern about maybe I'm sort of losing my train of thought a little bit, but I guess maybe alignment and misalignment aren't as binary as they were initially foreseen to be or something. You can still get a language model, for example, that tries to well, I guess there's different types of misleading but be deceptive or tamper with its reward function or whatever. Or you can get one that's sort of like earnestly trying to do the thing that its user wants. And that's not an incoherent concept anymore.HOLLYNo, it's not. Yeah, so yes, there is like, I guess the point of bringing up the VNM utility function was that there was sort of in the past a way that you could mathematically I don't know, of course utility functions are still real, but that's not what we're thinking anymore. We're thinking more like training and getting the gist of what and then getting corrections when you're not doing the right thing according to our values. But yeah, sorry. So the last piece I should have said originally was that I think with humans we're already substantially unaligned, but a lot of how we work together is that we have roughly similar capabilities. And if the idea of making AGI is to have much greater capabilities than we have, that's the whole point. I just think when you scale up like that, the divisions in your psyche or are just going to be magnified as well. And this is like an informal view that I've been developing for a long time, but just that it's actually the low capabilities that allows alignment or similar capabilities that makes alignment possible. And then there are, of course, mathematical structures that could be aligned at different capabilities. So I guess I have more hope if you could find the utility function that would describe this. But if it's just a matter of acting in distribution, when you increase your capabilities, you're going to go out of distribution or you're going to go in different contexts, and then the magnitude of mismatch is going to be huge. I wish I had a more formal way of describing this, but that's like my fundamental skepticism right now that makes me just not want anyone to build it. I think that you could have very sophisticated ideas about alignment, but then still just with not when you increase capabilities enough, any little chink is going to be magnified and it could be yeah.AARONSeems largely right, I guess. You clearly have a better mechanistic understanding of ML.HOLLYI don't know. My PiBBs project was to compare natural selection and gradient descent and then compare gradient hacking to miotic drive, which is the most analogous biological this is a very cool thing, too. Meatic drive. So Meiosis, I'll start with that for everyone.AARONThat's one of the cell things.HOLLYYes. Right. So Mitosis is the one where cells just divide in your body to make more skin. But Meiosis is the special one where you go through two divisions to make gametes. So you go from like we normally have two sets of chromosomes in each cell, but the gametes, they recombine between the chromosomes. You get different combinations with new chromosomes and then they divide again to bring them down to one copy each. And then like that, those are your gametes. And the gametes eggs come together with sperm to make a zygote and the cycle goes on. But during Meiosis, the point of it is to I mean, I'm going to just assert some things that are not universally accepted, but I think this is by far the best explanation. But the point of it is to take this like, you have this huge collection of genes that might have individually different interests, and you recombine them so that they don't know which genes they're going to be with in the next generation. They know which genes they're going to be with, but which allele of those genes. So I'm going to maybe simplify some terminology because otherwise, what's to stop a bunch of genes from getting together and saying, like, hey, if we just hack the Meiosis system or like the division system to get into the gametes, we can get into the gametes at a higher rate than 50%. And it doesn't matter. We don't have to contribute to making this body. We can just work on that.AARONWhat is to stop that?HOLLYYeah, well, Meiosis is to stop that. Meiosis is like a government system for the genes. It makes it so that they can't plan to be with a little cabal in the next generation because they have some chance of getting separated. And so their best chance is to just focus on making a good organism. But you do see lots of examples in nature of where that cooperation is breaking down. So some group of genes has found an exploit and it is fucking up the species. Species do go extinct because of this. It's hard to witness this happening. But there are several species. There's this species of cedar that has a form of this which is, I think, maternal genome. It's maternal genome elimination. So when the zygote comes together, the maternal chromosomes are just thrown away and it's like terrible because that affects the way that the thing works and grows, that it's put them in a death spiral and they're probably going to be extinct. And they're trees, so they live a long time, but they're probably going to be extinct in the next century. There's lots of ways to hack meiosis to get temporary benefit for genes. This, by the way, I just think is like nail in the coffin. Obviously, gene centered view is the best evolutionarily. What is the best the gene centered view of evolution.AARONAs opposed to sort of standard, I guess, high school college thing would just be like organisms.HOLLYYeah, would be individuals. Not that there's not an accurate way to talk in terms of individuals or even in terms of groups, but to me, conceptually.AARONThey'Re all legit in some sense. Yeah, you could talk about any of them. Did anybody take like a quirk level? Probably not. That whatever comes below the level of a gene, like an individual.HOLLYWell, there is argument about what is a gene because there's multiple concepts of genes. You could look at what's the part that makes a protein or you can look at what is the unit that tends to stay together in recombination or something like over time.AARONI'm sorry, I feel like I cut you off. It's something interesting. There was meiosis.HOLLYMeiotic drive is like the process of hacking meiosis so that a handful of genes can be more represented in the next generation. So otherwise the only way to get more represented in the next generation is to just make a better organism, like to be naturally selected. But you can just cheat and be like, well, if I'm in 90% of the sperm, I will be next in the next generation. And essentially meiosis has to work for natural selection to work in large organisms with a large genome and then yeah, ingredient descent. We thought the analogy was going to be with gradient hacking, that there would possibly be some analogy. But I think that the recombination thing is really the key in Meadic Drive. And then there's really nothing like that in.AARONThere'S. No selection per se. I don't know, maybe that doesn't. Make a whole lot of sense.HOLLYWell, I mean, in gradient, there's no.AARONG in analog, right?HOLLYThere's no gene analog. Yeah, but there is, like I mean, it's a hill climbing algorithm, like natural selection. So this is especially, I think, easy to see if you're familiar with adaptive landscapes, which looks very similar to I mean, if you look at a schematic or like a model of an illustration of gradient descent, it looks very similar to adaptive landscapes. They're both, like, in dimensional spaces, and you're looking at vectors at any given point. So the adaptive landscape concept that's usually taught for evolution is, like, on one axis you have fitness, and on the other axis you have well, you can have a lot of things, but you have and you have fitness of a population, and then you have fitness on the other axis. And what it tells you is the shape of the curve there tells you which direction evolution is going to push or natural selection is going to push each generation. And so with gradient descent, there's, like, finding the gradient to get to the lowest value of the cost function, to get to a local minimum at every step. And you follow that. And so that part is very similar to natural selection, but the Miosis hacking just has a different mechanism than gradient hacking would. Gradient hacking probably has to be more about I kind of thought that there was a way for this to work. If fine tuning creates a different compartment that doesn't there's not full backpropagation, so there's like kind of two different compartments in the layers or something. But I don't know if that's right. My collaborator doesn't seem to think that that's very interesting. I don't know if they don't even.AARONKnow what backup that's like a term I've heard like a billion times.HOLLYIt's updating all the weights and all the layers based on that iteration.AARONAll right. I mean, I can hear those words. I'll have to look it up later.HOLLYYou don't have to full I think there are probably things I'm not understanding about the ML process very well, but I had thought that it was something like yeah, like in yeah, sorry, it's probably too tenuous. But anyway, yeah, I've been working on this a little bit for the last year, but I'm not super sharp on my arguments about that.AARONWell, I wouldn't notice. You can kind of say whatever, and I'll nod along.HOLLYI got to guard my reputation off the cuff anymore.AARONWe'll edit it so you're correct no matter what.HOLLYHave you ever edited the Oohs and UMS out of a podcast and just been like, wow, I sound so smart? Like, even after you heard yourself the first time, you do the editing yourself, but then you listen to it and you're like, who is this person? Looks so smart.AARONI haven't, but actually, the 80,000 Hours After hours podcast, the first episode of theirs, I interviewed Rob and his producer Kieran Harris, and that they have actual professional sound editing. And so, yeah, I went from totally incoherent, not totally incoherent, but sarcastically totally incoherent to sounding like a normal person. Because of that.HOLLYI used to use it to take my laughter out of I did a podcast when I was an organizer at Harvard. Like, I did the Harvard Effective Alchruism podcast, and I laughed a lot more than I did now than I do now, which is kind of like and we even got comments about it. We got very few comments, but they were like, girl hosts laughs too much. But when I take my laughter out, I would do it myself. I was like, wow, this does sound suddenly, like, so much more serious.AARONYeah, I don't know. Yeah, I definitely say like and too much. So maybe I will try to actually.HOLLYRealistically, that sounds like so much effort, it's not really worth it. And nobody else really notices. But I go through periods where I say like, a lot, and when I hear myself back in interviews, that really bugs me.AARONYeah.HOLLYGod, it sounds so stupid.AARONNo. Well, I'm definitely worse. Yeah. I'm sure there'll be a way to automate this. Well, not sure, but probably not too distant.HOLLYFuture people were sending around, like, transcripts of Trump to underscore how incoherent he is. I'm like, I sound like that sometimes.AARONOh, yeah, same. I didn't actually realize that this is especially bad. When I get this transcribed, I don't know how people this is a good example. Like the last 10 seconds, if I get it transcribed, it'll make no sense whatsoever. But there's like a free service called AssemblyAI Playground where it does free drAARONased transcription and that makes sense. But if we just get this transcribed without identifying who's speaking, it'll be even worse than that. Yeah, actually this is like a totally random thought, but I actually spent not zero amount of effort trying to figure out how to combine the highest quality transcription, like whisper, with the slightly less goodAARONased transcriptions. You could get the speaker you could infer who's speaking based on the lower quality one, but then replace incorrect words with correct words. And I never I don't know, I'm.HOLLYSure somebody that'd be nice. I would do transcripts if it were that easy, but I just never have but it is annoying because I do like to give people the chance to veto certain segments and that can get tough because even if I talk you.AARONHave podcasts that I don't know about.HOLLYWell, I used to have the Harvard one, which is called the turning test. And then yeah, I do have I.AARONProbably listened to that and didn't know it was you.HOLLYOkay, maybe Alish was the other host.AARONI mean, it's been a little while since yeah.HOLLYAnd then on my I like, publish audio stuff sometimes, but it's called low effort. To underscore.AARONOh, yeah, I didn't actually. Okay. Great minds think alike. Low effort podcasts are the future. In fact, this is super intelligent.HOLLYI just have them as a way to catch up with friends and stuff and talk about their lives in a way that might recorded conversations are just better. You're more on and you get to talk about stuff that's interesting but feels too like, well, you already know this if you're not recording it.AARONOkay, well, I feel like there's a lot of people that I interact with casually that I don't actually they have these rich online profiles and somehow I don't know about it or something. I mean, I could know about it, but I just never clicked their substack link for some reason. So I will be listening to your casual.HOLLYActually, in the 15 minutes you gave us when we pushed back the podcast, I found something like a practice talk I had given and put it on it. So that's audio that I just cool. But that's for paid subscribers. I like to give them a little something.AARONNo, I saw that. I did two minutes of research or whatever. Cool.HOLLYYeah. It's a little weird. I've always had that blog as very low effort, just whenever I feel like it. And that's why it's lasted so long. But I did start doing paid and I do feel like more responsibility to the paid subscribers now.AARONYeah. Kind of the reason that I started this is because whenever I feel so much I don't know, it's very hard for me to write a low effort blog post. Even the lowest effort one still takes at the end of the day, it's like several hours. Oh, I'm going to bang it out in half an hour and no matter what, my brain doesn't let me do that.HOLLYThat usually takes 4 hours. Yeah, I have like a four hour and an eight hour.AARONWow. I feel like some people apparently Scott Alexander said that. Oh, yeah. He just writes as fast as he talks and he just clicks send or whatever. It's like, oh, if I could do.HOLLYThat, I would have written in those paragraphs. It's crazy. Yeah, you see that when you see him in person. I've never met him, I've never talked to him, but I've been to meetups where he was and I'm at this conference or not there right now this week that he's supposed to be at.AARONOh, manifest.HOLLYYeah.AARONNice. Okay.HOLLYCool Lighthaven. They're now calling. It looks amazing. Rose Garden. And no.AARONI like, vaguely noticed. Think I've been to Berkeley, I think twice. Right? Definitely. This is weird. Definitely once.HOLLYBerkeley is awesome. Yeah.AARONI feel like sort of decided consciously not to try to, or maybe not decided forever, but had a period of time where I was like, oh, I should move there, or we'll move there. But then I was like I think being around other EA's in high and rational high concentration activates my status brain or something. It is very less personally bad. And DC is kind of sus that I was born here and also went to college here and maybe is also a good place to live. But I feel like maybe it's actually just true.HOLLYI think it's true. I mean, I always like the DCAS. I think they're very sane.AARONI think both clusters should be more like the other one a little bit.HOLLYI think so. I love Berkeley and I think I'm really enjoying it because I'm older than you. I think if you have your own personality before coming to Berkeley, that's great, but you can easily get swept. It's like Disneyland for all the people I knew on the internet, there's a physical version of them here and you can just walk it's all in walking distance. That's all pretty cool. Especially during the pandemic. I was not around almost any friends and now I see friends every day and I get to do cool stuff. And the culture is som

Listen on: * Spotify* Apple Podcasts* Google PodcastsNote: the core discussion on ethics begins at 7:58 and moves into philosophy of language at ~1:12:19Daniel's stuff:* AI X-risk podcast * The Filan Cabined podcast* Personal website and blogBlurb and bulleted summary from ClongThis wide-ranging conversation between Daniel and Aaron touches on movies, business drama, philosophy of language, ethics and legal theory. The two debate major ethical concepts like utilitarianism and moral realism. Thought experiments around rational beings choosing to undergo suffering feature prominently. meandering tangents explore the semantics of names and references.* Aaron asserts that total utilitarianism does not imply that any amount of suffering can be morally justified by creating more happiness. His argument is that the affirmative case for this offsetting ability has not been clearly made.* He proposes a thought experiment - if offered to experience the suffering of all factory farmed animals in exchange for unlimited happiness, even a perfectly rational being would refuse. This indicates there are some levels of suffering not offsettable.* Aaron links this to experiences like hunger where you realize suffering can be worse than you appreciate normally. This causes his intuition some suffering can't be outweighed.* Daniel disagrees, believing with the right probabilities and magnitudes of suffering versus happiness, rational beings would take that gamble.* For example, Daniel thinks the atomic bombing of Japan could be offset by reducing more suffering. Aaron is less sure given the pain inflicted.* Daniel also proposes offsets for animal farming, but Aaron doesn't think factory farming harm is offsettable by any amount of enjoyment of meat.* They discuss definitions of rationality and whether evolution pressures against suicide impact the rationality of not killing oneself.* Aaron ties his argument to siding with what a perfectly rational being would choose to experience, not necessarily what they would prefer.* They debate whether hypothetical aliens pursuing "schmorality" could point to a concept truly analogous to human morality. Aaron believes not.Transcript(Very imperfect)AARONO'how's, it going it's going all right.DANIELYeah, I just so yesterday I saw Barbie and today I saw Oppenheimer, so it's good to oh, cool. That cultural.AARONNice, nice.DANIELDo you have takes? Yeah, I thought it was all right. It was a decent view of Oppenheimer as a person. It was like a how? I don't know. I feel like the public can tend to be taken in by this physicist figures you get this with quotes, right? Like, the guy was just very good at having fun with journalists, and now we get these amazing nuggets of wisdom from Einstein. I don't know. I think that guy was just having good I don't know. The thing that I'm coming away from is I thought I only watched Barbie because it was coming out on the same day as Oppenheimer, right? Like, otherwise it wouldn't have occurred to me to watch it. I was like, yeah, whatever. Barbie is, like, along for the ride, and Oppenheimer is going to be amazing, but in like, maybe Oppenheimer was a bit better than Barbie, but I'm not even sure of that, actually.AARONYeah, I've been seeing people say that on Twitter. I haven't seen either, but I've been seeing several people say that I'm following, say, like, Barbie was exceptional. And also that kind of makes sense because I'm following all these EA people who are probably care more about the subject matter for the latter one. So it's like, I kind of believe that Barbie is, like, aesthetically better or something. That's my take. Right.DANIELGuess. Well, if you haven't seen them, I guess I don't want to spoil them for you. They're trying to do different things aesthetically. Right. Like, I'm not quite sure I'd want to say one is aesthetically better. Probably in some ways, I think Barbie probably has more aesthetic blunders than Oppenheimer does. Okay. But yeah, I don't know if you haven't seen it, I feel like I don't want to spoil it for you.AARONOkay. No, that's fine. This isn't supposed to be like probably isn't the most important the most interesting thing we could be talking about is that the bar?DANIELOh, jeez.AARONOh, no, that's a terrible bar. That was like an overstatement. That would be a very high bar. It would also be, like, kind of paralyzing. I don't know. Actually know what that would be, honestly. Probably some social juicy gossip thing. Not that we necessarily have any.DANIELYeah, I think your interestingness. Yeah, I think I don't have the know, the closest to gossip thing I saw was like, do you see this bit of Carolyn Elson's diaries and letters to SBF that was leaked to the.AARONNo, I don't. Was this like today or recently? How recently?DANIELThis was like a few days ago.AARONI've been seeing her face on Twitter, but I don't actually think I know anything about this. And no, I would not have.DANIELBackground of who she is and stuff.AARONYeah, hold on. Let the audience know that I am on a beach family vacation against my will. Just kidding. Not against my will. And I have to text my sister back. Okay, there we go. I mean, I broadly know the FTX story. I know that she was wait, I'm like literally blanking on the Alameda.DANIELThat's the name of research.AARONOkay. Yeah. So she was CEO, right? Yeah. Or like some sort of like I think I know the basics.DANIELThe like, she was one of the OG Stanford EA people and was around.AARONYeah, that's like a generation. Not an actual generation, like an EA generation. Which is what, like six years or.DANIELLike the I don't know, I've noticed like, in the there's like I feel like there's this gap between pre COVID people and post COVID people. No one left their house. Partly people moved away, but also you were inside for a while and never saw anyone in person. So it felt like, oh, there's like this crop of new people or something. Whereas in previous years, there'd be some number of new people per year and they'd get gradually integrated in. Anyway, all that is to say that, I don't know, I think SBF's side of the legal battle leaked some documents to The New York Times, which were honestly just like her saying, like, oh, I feel very stressed and I don't like my job, and I'm sort of glad that the thing is blown up now. I don't know. It honestly wasn't that salacious. But I think that's, like, the way I get in the loop on gossip like some of the New York Times.AARONAnd I eventually I love how it's funny that this particular piece of gossip is, like, running through the most famous and prestigious news organization in the world. Or, like, one of them or something. Yeah. Instead of just being like, oh, yeah, these two people are dating, or whatever. Anyway, okay, I will maybe check that out.DANIELYeah, I mean, honestly, it's not even that interesting.AARONThe whole thing is pretty I am pretty. This is maybe bad, but I can't wait to watch the Michael Lewis documentary, pseudo documentary or whatever.DANIELYeah, it'll be good to read the book. Yeah, it's very surreal. I don't know. I was watching Oppenheimer. Right. And I have to admit, part of what I'm thinking is be if humanity survives, there's going to be this style movie about open AI, presumably, right? And I'm like, oh, man, it'll be amazing to see my friend group depicted on film. But that is going to happen. It's just going to be about FTX and about how they're all criminals. So that's not great.AARONYeah, actually, everybody dunks on crypto now, and it's like low status now or whatever. I still think it's really cool. I never had more than maybe $2,000 or whatever, which is not a trivial I mean, it's not a large amount of my money either, but it's not like, nothing. But I don't know, if it wasn't for all the cultural baggage, I feel like I would be a crypto bro or I would be predisposed to being a crypto bro or something.DANIELYeah. I should say I was like joking about the greedy crypto people who want their money to not be stolen. I currently have a Monero sticker on the back of my a big I don't know, I'm a fan of the crypto space. It seems cool. Yeah. I guess especially the bit that is less about running weird scams. The bit that's running weird scams I'm less of a fan of.AARONYeah. Yes. I'm also anti scam. Right, thank you. Okay, so I think that thing that we were talking about last time we talked, which is like the thing I think we actually both know stuff about instead of just like, repeating New York Times articles is my nuanced ethics takes and why you think about talk about that and then we can just also branch off from there.DANIELYeah, we can talk about that.AARONMaybe see where that did. I luckily I have a split screen up, so I can pull up things. Maybe this is kind of like egotistical or something to center my particular view, but you've definitely given me some of the better pushback or whatever that I haven't gotten that much feedback of any kind, I guess, but it's still interesting to hear your take. So basically my ethical position or the thing that I think is true is that which I think is not the default view. I think most people think this is wrong is that total utilitarianism does not imply that for some amount of suffering that could be created there exists some other extremely large arbitrarily, large amount of happiness that could also be created which would morally justify the former. Basically.DANIELSo you think that even under total utilitarianism there can be big amounts of suffering such that there's no way to morally tip the calculus. However much pleasure you can create, it's just not going to outweigh the fact that you inflicted that much suffering on some people.AARONYeah, and I'd highlight the word inflicted if something's already there and you can't do anything about it, that's kind of neither here nor there as it pertains to your actions or something. So it's really about you increasing, you creating suffering that wouldn't have otherwise been created. Yeah. It's also been a couple of months since I've thought about this in extreme detail, although I thought about it quite a bit. Yeah.DANIELMaybe I should say my contrary view, I guess, when you say that, I don't know, does total utilitarianism imply something or not? I'm like, well, presumably it depends on what we mean by total utilitarianism. Right. So setting that aside, I think that thesis is probably false. I think that yeah. You can offset great amounts of suffering with great amounts of pleasure, even for arbitrary amounts of suffering.AARONOkay. I do think that position is like the much more common and even, I'd say default view. Do you agree with that? It's sort of like the implicit position of people who are of self described total utilitarians who haven't thought a ton about this particular question.DANIELYeah, I think it's probably the implicit default. I think it's the implicit default in ethical theory or something. I think that in practice, when you're being a utilitarian, I don't know, normally, if you're trying to be a utilitarian and you see yourself inflicting a large amount of suffering, I don't know. I do think there's some instinct to be like, is there any way we can get around this?AARONYeah, for sure. And to be clear, I don't think this would look like a thought experiment. I think what it looks like in practice and also I will throw in caveats as I see necessary, but I think what it looks like in practice is like, spreading either wild animals or humans or even sentient digital life through the universe. That's in a non as risky way, but that's still just maybe like, say, making the earth, making multiple copies of humanity or something like that. That would be an example that's probably not like an example of what an example of creating suffering would be. For example, just creating another duplicate of earth. Okay.DANIELAnything that would be like so much suffering that we shouldn't even the pleasures of earth outweighs.AARONNot necessarily, which is kind of a cop out. But my inclination is that if you include wild animals, the answer is yes, that creating another earth especially. Yeah, but I'm much more committed to some amount. It's like some amount than this particular time and place in human industry is like that or whatever.DANIELOkay, can I get a feel of some other concrete cases to see?AARONYeah.DANIELSo one example that's on my mind is, like, the atomic bombing of Hiroshima and Nagasaki, right? So the standard case for this is, like, yeah, what? A hundred OD thousand people died? Like, quite terrible, quite awful. And a lot of them died, I guess a lot of them were sort of some people were sort of instantly vaporized, but a lot of people died in extremely painful ways. But the countercase is like, well, the alternative to that would have been like, an incredibly grueling land invasion of Japan, where many more people would have died or know regardless of what the actual alternatives were. If you think about the atomic bombings, do you think that's like the kind of infliction of suffering where there's just not an offsetting amount of pleasure that could make that okay?AARONMy intuition is no, that it is offsettable, but I would also emphasize that given the actual historical contingencies, the alternative, the implicit case for the bombing includes reducing suffering elsewhere rather than merely creating happiness. There can definitely be two bad choices that you have to make or something. And my claim doesn't really pertain to that, at least not directly.DANIELRight. Sorry. But when you said you thought your answer was no, you think you can't offset that with pleasure?AARONMy intuition is that you can, but I know very little about how painful those deaths were and how long they lasted.DANIELYeah, so the non offset so it's like, further out than atomic bombing.AARONThat's my guess, but I'm like.DANIELOkay, sure, that's your guess. You're not super confident. That's fine. I guess another thing would be, like, the animal farming system. So, as you're aware, tons of animals get kept in farms for humans to eat, by many count. Many of them live extremely horrible lives. Is there some amount that humans could enjoy meat such that that would be okay?AARONNo. So the only reason I'm hesitating is because, like, the question is, like, what the actual alternative is here, but, like, if it's like, if it's, like, people enjoy, like, a meat a normal amount and there's no basically the answer is no. Although, like, what I would actually endorse doing depends on what the alternative is.DANIELOkay, but you think that factory farming is so bad that it's not offsettable by pleasure.AARONYeah, that's right. I'm somewhat maybe more confident than the atomic bombing case, but again, I don't know what it's like to be a factory farm pig. I wouldn't say I'm, like, 99% sure. Probably more than 70% or something. Or 70%, like, conditional on me being right about this thesis, I guess something like that, which I'm like. Yeah, okay. I don't know. Some percent, maybe, not probably not 99% sure, but also more than 60. Probably more than 70% sure or something.DANIELAll right. Yeah. So I guess maybe can you tell us a little bit about why you would believe that there's some threshold that you like where you can no longer compensate by permitting pleasure?AARONYes. Let me run through my argument and sort of a motivation, and the motivation actually is sort of more a direct answer to what you just said. So the actual argument that I have and I have a blog post about this that I'll link, it was part of an EA forum post also that you'll also link in the show description is that the affirmative default case doesn't seem to actually be made anywhere. That's not the complete argument, but it's a core piece of it, which is that it seems to be, like, the default received view, which doesn't mean it's wrong, but does mean that we should be skeptical. If you accept that I'm right, that the affirmative case hasn't been made, we can talk about that. Then you should default to some other heuristic. And the heuristic that I assert and sort of argue, but kind of just assert is a good heuristic is. Okay. Is you do the following thought experiment. If I was a maximally or perfectly rational being, would I personally choose to undergo this amount of suffering in compensation or not compensation, exchange for later undergoing or earlier undergoing some arbitrarily large amount of happiness. And I personally have the intuition that there are events or things that certainly conceivable states and almost certainly possible states that I could be in such that even as a rational being, like as a maximum rational being, I would choose to just disappear and not exist rather than undergo both of these things.DANIELOkay.AARONYeah.DANIELWhy do you think that?AARONYeah, so good question. I think the answer comes at a couple of different levels. So there's a question of why I'm saying it and why I'm saying it is because I'm pretty sure this is the answer I would actually give if actually given if Credibly offered this option. But that just pushes the question back. Okay, why do I feel that.DANIELEven what option are we talking about here? There exists a thing such that for.AARONAll pleasures, basically, for example, let's just run with the fact, the assumption that a genie God descends. And I think it's credible, and he offers that I can live the life of every factory, farmed animal in exchange for whatever I want for any amount of time or something like that. Literally, I don't have to give the answer now. It can just be like an arbitrarily good state for an arbitrarily long period of time.DANIELOh, yeah.AARONAnd not only would I say the words no, I don't want to do that, I think that the words no, I don't want to do that, are selfishly in a non pejorative sense. Correct. And then there's a question of why do I have that intuition? And now I'm introspecting, which is maybe not super reliable. I think part of my intuition that I can kind of maybe sort of access via introspection just comes from basically, I'm very fortunate to not have had a mostly relatively comfortable life, like as a Westerner with access to painkillers, living in the 21st century. Even still, there have definitely been times when I've been suffered, at least not in a relative sense, but just like, in an absolute sense to me, in a pretty bad way. And one example I can give was just like, I was on a backpacking trip, and this is the example I give in another blog post I can link. I was on a backpacking trip, and we didn't have enough food, and I was basically very hungry for like five days. And I actually think that this is a good and I'm rambling on, but I'll finish up. I think it's illustrative. I think there's some level of suffering where you're still able to do at least for me, I'm still able to do something like reasoning and intentionally storing memories. One of the memories I tried to intentionally codify via language or something was like, yeah, this is really bad, this really sucks, or something like, that what.DANIELSucked about it, you were just like, really hungry yeah.AARONFor five days.DANIELOkay. And you codified the thought, like, feeling of this hunger I'm feeling, this really sucks.AARONSomething like that. Right. I could probably explicate it more, but that's basically okay. Actually, hold on. All right. Let me add so not just it really sucks, but it sucks in a way that I can't normally appreciate, so I don't normally have access to how bad it sucks. I don't want to forget about this later or something.DANIELYeah. The fact that there are pains that are really bad where you don't normally appreciate how bad they are, it's not clear how that implies non offset ability.AARONRight, I agree. It doesn't.DANIELOkay.AARONI do think that's causally responsible for my intuition that I lend link to a heuristic that I then argue does constitute an argument in the absence of other arguments for offset ability.DANIELYeah. Okay. So that causes this intuition, and then you give some arguments, and the argument is like, you think that if a genie offered you to live liable factory farmed animals in exchange for whatever you wanted, you wouldn't go for that.AARONYes. And furthermore, I also wouldn't go for it if I was much more rational.DANIELIf you were rational, yeah. Okay. Yeah. What do I think about this? One thing I think is that the I think the case of live experience this suffering and then experience this pleasure, to me, I think that this is kind of the wrong way to go about this. Because the thing about experiencing suffering is that it's not just we don't live in this totally dualistic world where suffering just affects only your immaterial mind or something in a way where afterwards you could just be the same. In the real world, suffering actually affects you. Right. Perhaps indelibly. I think instead, maybe the thing I'd want to say is suppose you're offered a gamble, right, where there's like a 1% chance that you're going to have to undergo excruciating suffering and a 99% chance that you get extremely awesome pleasures or something.AARONYeah.DANIELAnd this is meant to model a situation in which you do some action in which one person is going to undergo really bad suffering and 99 other people are going to undergo really great pleasure. And to me, I guess my intuition is that for any bad thing, you could make the probability small enough and you can make the rest of the probability mass good enough that I want to do that. I feel like that's worth it for me. And now it feels a little bit unsatisfying that we're just going that we're both drilling down to, like, well, this is the choice I would make, and then maybe you can disagree that it's the choice you would make. But yeah, I guess about the gambling case, what do you think about that? Let's say it's literally a one in a million chance that you would have to undergo, let's say, the life of one factory farmed animal.AARONYeah.DANIELOr is that not enough? Do you want it to be like, more?AARONWell, I guess it would have to be like one of the worst factory farmed animals. Life, I think would make that like.DANIELYeah, okay, let's say it's like, maybe literally one in a billion chance.AARONFirst of all, I do agree that these are basically isomorphic or morally equivalent, or if anything, time ordering in my example does mess things up a little bit, I'll be happy to reverse them or say that instead compare one person to 1000 people. So, yeah, you can make the probability small enough that my intuition changes. Yeah. So in fact, 1%, I'm very like, no, definitely not doing that. One in a million. I'm like, I don't know, kind of 50 50. I don't have a strong intuition either way. 100 trillion. I have the intuition. You know what? That's just not going to happen. That's my first order intuition. I do think that considering the case where you live, one being lives both lives, or you have, say, one being undergoing the suffering and then like 100 trillion undergoing the pleasure makes small probabilities more if you agree that they're sort of isomorphic makes them more complete or something like that, or complete more real in some. Not tangible is not the right word, but more right.DANIELYou're less tempted to round it to zero.AARONYeah. And so I tend to think that I trust my intuitions more about reasoning. Okay, there's one person undergoing suffering and like 100 trillion undergoing happiness as it pertains to the question of offset ability more than I trust my intuitions about small probabilities.DANIELI guess that's strange because that strikes me as strange because I feel like you're regularly in situations where you make choices that have some probability of causing you quite bad suffering, but a large probability of being fun. Like going to the beach. There could be a shark there. I guess this is maybe against your will, but you can go to a restaurant, maybe get food poisoning, but how often are you like, oh man, if I flip this switch, one person will be poisoned, but 99 people will?AARONWell, then you'd have to think that, okay, staying home would actually be safer for some reason, which I don't affirmatively think is true, but this actually does work out for the question of whether you should kill yourself. And there hopefully this doesn't get censored by Apple or whatever, so nobody do that. But there I just think that my lizard brain or there's enough evolutionary pressure to not trust that I would be rational when it comes to the question of whether to avoid a small chance of suffering by unaliving myself, as they say on TikTok.DANIELHang on, evolution is pressured. So there's some evolutionary pressure to make sure you really don't want to kill yourself, but you think that's like, irrational.AARONI haven't actually given this a ton of thought. It gets hard when you loop in altruism and yeah, the question also there's like some chance that of sentient's after death, there's not literally zero or something like that. Yeah, I guess those are kind of cop outs. So I don't know, I feel like it certainly could be. And I agree this is sort of like a strike against my argument or something. I can set up a situation you have no potential to improve the lives of others, and you can be absolutely sure that you're not going to experience any sentience after death. And then I feel like my argument does kind of imply that, yeah, that's like the rational thing to do. I wouldn't do it. Right. So I agree. This is like a strike against me.DANIELYeah. I guess I just want to make two points. So the first point I want to make is just methodologically. If we're talking about which are you likely to be more rational about gambles of small risks, small probabilities of risk versus large rewards as opposed to situations where you can do a thing that affects a large number of people one way and a small number of people another way? I think the gambles are more like decisions that you make a bunch and you should be rational about and then just the second thing in terms of like, I don't know, I took you to be making some sort of argument along the lines of there's evolutionary pressure to want to not kill yourself. Therefore, that's like a debunking explanation. The fact that there was evolutionary pressure to not kill ourselves means that our instinct that we shouldn't kill ourselves is irrational. Whereas I would tend to look at it and say the fact that there was very strong evolutionary pressure to not kill ourselves is an explanation of why I don't want to kill myself. And I see that as affirming the choice to not kill myself, actually.AARONWell, I just want to say I don't think it's an affirmative argument that it is irrational. I think it opens up the question. I think it means it's more plausible that for other I guess not even necessarily for other reasons, but it just makes it more plausible that it is irrational. Well.DANIELYeah, I take exactly the opposite view. Okay. I think that if I'm thinking about, like, oh, what do I really want? If I consider my true preferences, do I really want to kill myself or something? And then I learn that, oh, evolution has shaped me to not kill myself, I think the inference I should make is like, oh, I guess probably the way evolution did that is that it made it such that my true desires are to not kill myself.AARONYeah. So one thing is I just don't think preferences have any intrinsic value. So I don't know, we might just like I guess I should ask, do you agree with that or disagree with.DANIELThat do I think preferences have intrinsic value? No, but so no, but I think like, the whole game here is like, what do I prefer? Or like, what would I prefer if I understood things really clearly?AARONYes. And this is something I didn't really highlight or maybe I didn't say it at all, is that I forget if I really argue it or kind of just assert it, but I at least assert that the answer to hedonic utilitarian. What you should do under hedonic utilitarianism is maybe not identical to, but exactly the same as what a rational agent would do or what a rational agent would prefer if they were to experience everything that this agent would cause. Or something like that. And so these should give you the exact same answers is something I believe sure. Because I do think preferences are like we're built to understand or sort of intuit and reason about our own preferences.DANIELKind of, yeah. But broadly, I guess the point I'm making at a high level is just like if we're talking about what's ethical or what's good or whatever, I take this to ultimately be a question about what should I understand myself as preferring? Or to the extent that it's not a question of that, then it's like, I don't know, then I'm a bit less interested in the exercise.AARONYeah. It's not ideal that I appeal to this fake and that fake ideally rational being or something. But here's a reason you might think it's more worth thinking about this. Maybe you've heard about I think Tomasic makes an argument about yeah. At least in principle, you can have a pig that's in extreme pain but really doesn't want to be killed still or doesn't want to be taken out of its suffering or whatever, true ultimate preference or whatever. And so at least I think this is pretty convincing evidence that you can have where that's just like, wrong about what would be good for it, you know what I mean?DANIELYeah, sorry, I'm not talking about preference versus hedonic utilitarianism or anything. I'm talking about what do I want or what do I want for living things or something. That's what I'm talking about.AARONYeah. That language elicits preferences to me and I guess the analogous but the idea.DANIELIs that the answer to what I want for living things could be like hedonic utilitarianism, if you see what I mean.AARONOr it could be by that do you mean what hedonic utilitarianism prescribes?DANIELYeah, it could be that what I want is that just whatever maximizes beings pleasure no matter what they want.AARONYeah. Okay. Yeah, so I agree with that.DANIELYeah. So anyway, heading back just to the suicide case right. If I learn that evolution has shaped me to not want to kill myself, then that makes me think that I'm being rational in my choice to not kill myself.AARONWhy?DANIELBecause being rational is something like optimally achieving your goals. And I'm a little bit like I sort of roughly know the results of killing myself, right? There might be some question about like, but what are my goals? And if I learned that evolution has shaped my goals such that I would hate killing myself right, then I'm like, oh, I guess killing myself probably ranks really low on the list of states ordered by how much I like them.AARONYeah, I guess then it seems like you have two mutually incompatible goals. Like, one is staying alive and one is hedonic utilitarianism and then you have to choose which of these predominates or whatever.DANIELYeah, well, I think that to the extent that evolution is shaping me to not want to commit suicide, it looks like the not killing myself one is winning. I think it's evidence. I don't think it's conclusive. Right. Because there could be multiple things going on. But I take evolutionary explanations for why somebody would want X. I think that's evidence that they are rational in pursuing X rather than evidence that they are irrational in pursuing X.AARONSometimes that's true, but not always. Yeah, there's a lot in general it is. Yeah. But I feel like moral anti realistic, we can also get into that. Are going to not think this is like woo or Joe Carl Smith says when he's like making fun of moralists I don't know, in a tongue in cheek way. In one of his posts arguing for explicating his stance on antirealism basically says moral realists want to say that evolution is not sensitive to moral reasons and therefore evolutionary arguments. Actually, I don't want to quote him from memory. I'll just assert that evolution is sensitive to a lot of things, but one of them is not moral reasons and therefore evolutionary arguments are not a good evidence or are not good evidence when it comes to purely, maybe not even purely, but philosophical claims or object level moral claims, I guess, yeah, they can be evidenced by something, but not that.DANIELYeah, I think that's wrong because I think that evolution why do I think it's wrong? I think it's wrong because what are we talking about when we talk about morality? We're talking about some logical object that's like the completion of a bunch of intuitions we have. Right. And those I haven't thought about intuitions are the product of evolution. The reason we care about morality at all is because of evolution under the standard theory that evolution is the reason our brains are the way they are.AARONYeah, I think this is a very strange coincidence and I am kind of weirded out by this, but yes, I.DANIELDon'T think it's a coincidence or like not a coincidence.AARONSo it's not a coincidence like conditional honor, evolutionary history. It is like no extremely lucky or something that we like, of course we'd find it earthlings wound up with morality and stuff. Well, of course you would.DANIELWait. Have you read the metafic sequence by Elizar? Yudkowski.AARONI don't think so. And I respect Elias a ton, except I think he's really wrong about ethics and meta ethics in a lot of like I don't even know if I but I have not, so I'm not really giving them full time.DANIELOkay. I don't know. I basically take this from my understanding of the meta ethics sequence, which I recommend people read, but I don't think it's a coincidence. I don't think we got lucky. I think it's a coincidence. There are some species that get evolved, right, and they end up caring about schmorality, right?AARONYeah.DANIELAnd there are some species that get evolved, right? And they end up caring about the prime numbers or whatever, and we evolved and we ended up caring about morality. And it's not like a total so, okay, partly I'm just like, yeah, each one of them is really glad they didn't turn out to be the other things. The ones that care about two of.AARONThem are wrong, but two of them are wrong.DANIELWell, they're morally wrong. Two of them do morally wrong things all the time. Right?AARONI want to say that I hate when people say that. Sorry. So what I am saying is that you can call those by different names, but if I'm understanding this argument right, they all think that they're getting at the same core concept, which is like, no, what should we do in some okay, so does schmorality have any sort of normativity?DANIELNo, it has schmormativity.AARONOkay, well, I don't know what schmormativity is.DANIELYou know how normativity I feel like that's good. Schmormativity is about promoting the schmud.AARONOkay, so it sounds like that's just normativity, except it's normativity about different propositions. That's what it sounds like.DANIELWell, basically, I don't know, instead of these schmalians wait, no, they're aliens. They're not shmalians. They're aliens. They just do a bunch of schmud things, right? They engage in projects, they try and figure out what the schmud is. They pursue a schmud and then they look at humans, they're like, oh, these humans are doing morally good things. That's horrible. I'm so glad that we pursue the schmood instead.AARONYeah, I don't know if it's incoherent. I don't think they're being incoherent. Your description of a hypothetical let's just take for granted whatever in the thought experiment is in fact happening. I think your description is not correct. And the reason it's not correct is because there is like, what's a good analogy? So when it comes to abstract concepts in general, it is very possible for okay, I feel like it's hard to explain directly, but here an analogy, is you can have two different people who have very different conceptions of justice, but fundamentally are earnestly trying to get at the same thing. Maybe justice isn't well defined or isn't like, actually, I should probably have come up with a good example here. But you know what? I'm happy to change the word for what I use as morality or whatever, but it has the same core meaning, which is like, okay, really, what should you do at the end of the day?DANIELYeah.AARONWhat should you do?DANIELWhereas they care about morality, which is what they should do, which is a different thing. They have strong desires to do what they should do.AARONI don't think it is coherent to say that there are multiple meanings of the word should or multiple kinds. Yeah.DANIELNo, there aren't.AARONSorry. There aren't multiple meanings of the word should. Fine.DANIELThere's just a different word, which is schmood, which means something different, and that's what their desires are pegged to.AARONI don't think it's coherent, given what you've already the entire picture, I think, is incoherent. Given everything else besides the word schmoud, it is incoherent to assert that there is something broadly not analogous, like maybe isomorphic to normativity or, like, the word should. Yeah. There is only what's yeah. I feel like I'm not gonna I'm not gonna be able to verbalize it super well. I do. Yeah. Can you take something can you pick.DANIELA sentence that I said that was wrong or that was incoherent?AARONWell, it's all wrong because these aliens don't exist.DANIELThe aliens existed.AARONOkay, well, then we're debating, like, I actually don't know. It depends. You're asserting something about their culture and psychology, and then the question is, like, are you right or wrong about that? If we just take for granted that you're right, then you're right. All right. I'm saying no, you can't be sure. So conditional on being right, you're right. Then there's a question of, like, okay, what is the probability? So, like, conditional on aliens with something broad, are you willing to accept this phrase, like, something broadly analogous to morality? Is that okay?DANIELYeah, sure.AARONOkay. So if we accept that there's aliens with something broadly analogous to morality, then you want to say that they can have not only a different word, but truly a pointer to a different concept. And I think that's false.DANIELSo you think that in conceptual space, there's morality and that there's, like, nothing near it for miles.AARONThe study, like yeah, basically. At least when we're talking about, like, the like, at the at the pre conclusion stage. So, like, before you get to the point where you're like, oh, yeah, I'm certain that, like, the answer is just that we need, like, we need to make as many tennis balls as possible or whatever the general thing of, like, okay, broadly, what is the right thing to do? What should I do? Would it be good for me to do this cluster of things yeah. Is, like, miles from everything else.DANIELOkay. I think there's something true to that. I think I agree with that in some ways and on others, my other response is I think it's not a total coincidence that humans ended up caring about morality. I think if you look at these evolutionary arguments for why humans would be motivated to pursue morality. They rely on very high level facts. Like, there are a bunch of humans around. There's not one human who's, like, a billion times more powerful than everyone else. We have language. We talk through things. We reason. We need to make decisions. We need to cooperate in certain ways to produce stuff. And it's not about the fact that we're bipedal or something. So in that sense, I think it's not a total coincidence that we ended up caring about morality. And so in some sense, I think because that's true, you could maybe say you couldn't slightly tweak our species that it cared about something other than morality, which is kind of like saying that there's nothing that close to morality in concept space.AARONBut I think I misspoke earlier what I should have said is that it's very weird that we care about that most people at least partially care about suffering and happiness. I think that's just a true statement. Sorry, that is the weird thing. Why is it weird? The weird thing is that it happens to be correct, even though I only.DANIELHave what do you mean it's correct?AARONNow we have to get okay, so this is going into moral realism. I think moral realism is true, at least.DANIELSorry, what do you mean by moral realism? Wait, different by moral realism?AARONYes. So I actually have sort of a weak version of moral realism, which is, like, not that normative statements are true, but that there is, like, an objective. So if you can rank hypothetical states of the world in an ordinal way such that one is objectively better than another.DANIELYes. Okay. I agree with that, by the way. I think that's true. Okay.AARONIt sounds like you're a moral realist.DANIELYeah, I am.AARONOkay. Oh, really? Okay. I don't know. I thought you weren't. Okay, cool.DANIELLots of people in my reference class aren't. I think most Bay Area rationalists are not moral realists, but I am.AARONOkay. Maybe I was confused. Okay, that's weird. Okay. Sorry about that. Wait, so what do I mean by it happens to be true? It's like it happens to coincide with yeah, sorry, go ahead.DANIELYou said it happens to be correct that we care about morality or that we care about suffering and pleasure and something and stuff.AARONMaybe that wasn't the ideal terminology it happens to so, like, it's not morally correct? The caring about it isn't the morally correct thing. It seems sort of like the caring is instrumentally useful in promoting what happens to be legitimately good or something. Or, like legitimately good or something like that.DANIELBut but I think, like so the aliens could say a similar thing, right? They could say, like, oh, hey, we've noticed that we all care about schmurality. We all really care about promoting Schmeasure and avoiding Schmuffering. Right? And they'd say, like, they'd say, like, yeah, what's? What's wrong?AARONI feel like it's not maybe I'm just missing something, but at least to me, it's like, only adding to the confusion to talk about two different concepts of morality rather than just like, okay, this alien thinks that you should tile the universe paperclips, or something like that, or even that more reasonably, more plausibly. Justice is like that. Yeah. I guess this gets back to there's only one concept anywhere near that vicinity in concept space or something. Maybe we disagree about that. Yeah.DANIELOkay. If I said paperclips instead of schmorality, would you be happy?AARONYes.DANIELI mean, cool, okay, for doing the.AARONMorally correct thing and making me happy.DANIELI strive to. But take the paperclipper species, right? What they do is they notice, like, hey, we really care about making paperclips, right? And, hey, the fact that we care about making paperclips, that's instrumentally useful in making sure that we end up making a bunch of paperclips, right? Isn't that an amazing coincidence that we ended up caring our desires were structured in this correct way that ends up with us making a bunch of paperclips. Is that like, oh, no, total coincidence. That's just what you cared about.AARONYou left at the part where they assert that they're correct about this. That's the weird thing.DANIELWhat proposition are they correct about?AARONOr sorry, I don't think they're correct implicitly.DANIELWhat proposition do they claim they're correct about?AARONThey claim that the world in which there is many paperclips is better than the world in which there is fewer paperclips.DANIELOh, no, they just think it's more paperclipy. They don't think it's better. They don't care about goodness. They care about paperclips.AARONSo it sounds like we're not talking about anything remotely like morality, then, because I could say, yeah, morality, morality. It's pretty airy. It's a lot of air in here. I don't know, maybe I'm just confused.DANIELNo, what I'm saying is, so you're like, oh, it's like this total coincidence that humans we got so lucky. It's so weird that humans ended up caring about morality, and it's like, well, we had to care about something, right? Like anything we don't care about.AARONOh, wow, sorry, I misspoke earlier. And I think that's generating some confusion. I think it's a weird coincidence that we care about happiness and suffering.DANIELHappiness and suffering, sorry. Yeah, but mutatus mutantus, I think you want to say that's like a weird coincidence. And I'm like, well, we had to care about something.AARONYeah, but it could have been like, I don't know, could it have been otherwise, right? At least conceivably it could have been otherwise.DANIELYeah, the paperclip guys, they're like, conceivably, we could have ended up caring about pleasure and suffering. I'm so glad we avoided that.AARONYeah, but they're wrong and we're right.DANIELRight about what?AARONAnd then maybe I don't agree. Maybe this isn't the point you're making. I'm sort of saying that in a blunt way to emphasize it. I feel like people should be skeptical when I say, like okay, I have good reason to think that even though we're in a very similar epistemic position, I have reason to believe that we're right and not the aliens. Right. That's like a hard case to make, but I do think it's true.DANIELThere's no proposition that the aliens and us disagree on yes.AARONThe intrinsic value of pleasure and happiness.DANIELYeah, no, they don't care about value. They care about schmalu, which is just.AARONLike, how much paperclips there is. I don't think that's coherent. I don't think they can care about value.DANIELOkay.AARONThey can, but only insofar as it's a pointer to the exact same not exact, but like, basically the same concept as our value.DANIELSo do you reject the orthogonality thesis?AARONNo.DANIELOkay. I think that is super intelligent.AARONYeah.DANIELSo I take the orthogonality thesis to mean that really smart agents can be motivated by approximately any desires. Does that sound right to you?AARONYeah.DANIELSo what if the desire is like, produce a ton of paperclips?AARONYeah, it can do that descriptively. It's not morally good.DANIELOh, no, it's not morally good at all. They're not trying to be morally good. They're just trying to produce a bunch of paperclips.AARONOkay, in that case, we don't disagree. Yeah, I agree. This is like a conceivable state of the world.DANIELYeah. But what I'm trying to say is when you say it's weird that we got lucky the reason you think it's weird is that you're one of the humans who cares about pleasure and suffering. Whereas if you were one of the aliens who cared about paperclips. The analogous shmarin instead of Aaron would be saying, like, oh, it's crazy that we care about paperclips, because that actually causes us to make a ton of paperclips.AARONDo they intrinsically care about paperclips or is it a means of cement?DANIELIntrinsically, like, same as in the Orphogonality thesis.AARONDo they experience happiness because of the paperclips or is it more of a functional intrinsic value?DANIELI think they probably experience happiness when they create paperclips, but they're not motivated by the happiness. They're motivated by like, they're happy because they succeeded at their goal of making tons of paperclips. If they can make tons of paperclips but not be happy about it, they'd be like, yeah, we should do that. Sorry. No, they wouldn't. They'd say, like, we should do that and then they would do it.AARONWould your case still work if we just pretended that they're not sentient?DANIELYeah, sure.AARONOkay. I think this makes it cleaner for both sides. Yeah, in that case, yes. So I think the thing that I reject is that there's an analog term that's anything like morality in their universe. They can use a different word, but it's pointing to the same concept.DANIELWhen you say anything like morality. So the shared concepts sorry, the shared properties between morality and paperclip promotion is just that you have a species that is dedicated to promoting it.AARONI disagree. I think morality is about goodness and badness.DANIELYes, that's right.AARONOkay. And I think it is totally conceivable. Not even conceivable. So humans wait, what's a good example? In some sense I intrinsically seem to value about regular. I don't know if this is a good example. Let's run with it intrinsically value like regulating my heartbeat. It happens to be true that this is conducive to my happiness and at least local non suffering. But even if it weren't, my brain stem would still try really hard to keep my heart beating or something like that. I reject that there's any way in which promoting heart beatingness is an intrinsic moral or schmoral value or even that could be it could be hypothesized as one but it is not in fact one or something like that.DANIELOkay.AARONLikewise, these aliens could claim that making paperclips is intrinsically good. They could also just make them and not make that claim. And those are two very different things.DANIELThey don't claim it's good. They don't think it's good.AARONThey think it's claim it schmud.DANIELWhich they prefer. Yeah, they prefer.AARONDon't. I think that is also incoherent. I think there is like one concept in that space because wait, I feel like also this is just like at some point it has to cash out in the real world. Right? Unless we're talking about really speculative not even physics.DANIELWhat I mean is they just spend all of their time promoting paperclips and then you send them a copy of Jeremy Bentham's collected writings, they read it and they're like all right, cool. And then they just keep on making paperclips because that's what they want to do.AARONYeah. So descriptively.DANIELSure.AARONBut they never claim that. It's like we haven't even introduced objectivity to this example. So did they ever claim that it's objectively the right thing to do?DANIELNo, they claim that it's objectively the paperclipy thing to do.AARONI agree with that. It is the paperclippy thing to do.DANIELYeah, they're right about stuff. Yeah.AARONSo they're right about that. They're just not a right. So I do think this all comes back down to the question of whether there's analogous concepts in near ish morality that an alien species might point at. Because if there's not, then the paperclippiness is just like a totally radically different type of thing.DANIELBut why does it like when did I say that they were closely analogous? This is what I don't understand.AARONSo it seems to be insinuated by the closeness of the word semantic.DANIELOh yeah, whatever. When I was making it a similar sounding word, all I meant to say is that they talk about it plays a similar role in their culture as morality plays in our culture. Sorry. In terms of their motivations, I should say. Oh, yeah.AARONI think there's plenty of human cultures that are getting at morality. Yeah. So I think especially historically, plenty of human cultures that are getting at the same core concept of morality but just are wrong about it.DANIELYeah, I think that's right.AARONFundamentalist religious communities or whatever, you can't just appeal to like, oh, we're like they have some sort of weird it's kind of similar but very different thing called morality.DANIELAlthough, I don't know, I actually think that okay, backing up. All I'm saying is that beings have to care about something, and we ended up caring about morality. And I don't think, like I don't know, I don't think that's super surprising or coincidental or whatever. A side point I want to make is that I think if you get super into being religious, you might actually start referring to a different concept by morality. How familiar are you with classical theism?AARONThat's not a term that I recognize, although I took a couple of theology classes, so maybe more of them if I hadn't done that.DANIELYeah, so classical theism, it's a view about the nature of God, which is that I'm going to do a bad job of describing it. Yeah, I'm not a classical theist, so you shouldn't take classical theist doctrine from me. But it's basically that God is like sort of God's the being whose attributes are like his existence or something like that. It's weird. But anyway, there's like some school of philosophical where they're like, yeah, there's this transcendent thing called God. We can know God exists from first principles and in particular their account of goodness. So how do you get around the Euphyro dilemma, right? Instead of something like divine command theory, what they say is that when we talk about things being good, good just refers to the nature of God. And if you really internalize that, then I think you might end up referring to something different than actual goodness. Although I think it's probably there's no such being as God in the article. Theist sense.AARONYeah. So they argue what we mean by good is this other.DANIELConcept. They would say that when everyone talks about good, what they actually mean is pertaining to the divine nature, but we just didn't really know that we meant that the same way that when we talked about water, we always meant H 20, but we didn't used to know that.AARONI'm actually not sure if this is I'm very unconfident, but I kind of want to bite the bullet and say, like, okay, fine, in that case, yeah, I'm talking about the divine nature, but we just have radically different understandings of what the divine nature is.DANIELYou think you're talking about the divine nature.AARONRight?DANIELWhy do you think that?AARONSorry, I think I very slightly was not quite pedantic enough. Sorry, bad cell phone or whatever. Once again, not very confident at all.DANIELBut.AARONThink think that I'm willing to I'm so I think that I'm referring to the divine nature, but what I mean by the divine nature is that which these fundamentalist people are referring to. So I want to get around the term and say like, okay, whatever these fundamentalists are referring to, I am also referring to them.DANIELYeah, I should say classical theism is not slightly a different when people say fundamentalists, they often mean like a different corner of Christian space than classical theists. Classical. Theists think like Ed Fesser esoteric Catholics or something. Yeah, they're super into it.AARONOkay, anyway yes, just to put it all together, I think that when I say morality, I am referring to the same thing that these people are referring to by the divine nature. That's what it took me like five minutes to actually say.DANIELOh yeah, so I don't think you are. So when they refer to the divine nature, what they at least think they mean is they think that the divine is sort of defined by the fact that its existence is logically necessary. Its existence is in some sense attributes it couldn't conceivably not have its various attributes. The fact that it is like the primary cause of the world and sustainer of all things. And I just really doubt that the nature of that thing is what you mean by morality.AARONNo, those are properties that they assert, but I feel like tell me if I'm wrong. But my guess is that if one such person were to just suddenly come to believe that actually all of that's right. Except it's not actually logically necessary that the divine nature exists. It happens to be true, but it's not logically necessary. They would still be sort of pointing to the same concept. And I just think, yeah, it's like that, except all those lists of properties are wrong.DANIELI think if that were true, then classical theism would be false.AARONOkay.DANIELSo maybe in fact you're referring to the same thing that they actually mean by the divine nature, but what they think they mean is this classical theistic thing. Right. And it seems plausible to me that some people get into it enough that what they actually are trying to get at when they say good is different than what normal people are trying to get at when they say good.AARONYeah, I don't think that's true. Okay, let's set aside the word morality because especially I feel like in circles that we're in, it has a strong connotation with a sort of like modern ish analytics philosophy, maybe like some other things that are in that category.DANIELYour video is worsen, but your sound is back.AARONOkay, well, okay, I'll just keep talking. All right, so you have the divine nature and morality and maybe other things that are like those two things but still apart from them. So in that class of things and then there's the question of like, okay, maybe everybody necessarily anybody who thinks that there's any true statements about something broadly in their vicinity of goodness in the idea space is pointing to the meta level of that or whichever one of those is truly correct or something. This is pretty speculative. I have not thought about this. I'm not super confident.DANIELYeah, I think I broadly believe this, but I think this is right about most people when they talk. But you could imagine even with utilitarianism, right? Imagine somebody getting super into the weeds of utilitarianism. They lived utilitarianism twenty four, seven. And then maybe at some point they just substitute in utilitarianism for morality. Now when they say morality, they actually just mean utilitarianism and they're just discarding the latter of the broad concepts and intuitions behind them. Such a person might just I don't know, I think that's the kind of thing that can happen. And then you might just want a.AARONDifferent thing by the word. I don't know if it's a bad thing, but I feel like I do this when I say, oh, x is moral to do or morally good to do. It's like, what's the real semantic relationship between that and it's correct on utilitarianism to do? I feel like they're not defined as the same, but they happen to be the same or something. Now we're just talking about how people use words.DANIELYeah, they're definitely going to happen to be the same in the case that utilitarianism is like the right theory of morality. But you could imagine that. You could imagine even in the case where utilitarianism was the wrong theory, you might still just mean utilitarianism by the word good because you just forgot the intuitions from which you were building theory of morality and you're just like, okay, look, I'm just going to talk about utilitarianism now.AARONYeah, I think this is like, yeah, this could happen. I feel like this is a cop out and like a non answer, but I feel like getting into the weeds of the philosophy of language and what people mean by concepts and words and true the true nature of concepts. It's just not actually that useful. Or maybe it's just not as interesting to me as I'm glad that somebody thought about that ever.DANIELI think this can happen, though. I think this is actually a practical concern. Right. Okay. Utilitarianism might be wrong, right? Does that strike you as right? Yeah, I think it's possible for you to use language in such a way that if utilitarianism were wrong, what that would mean is that in ordinary language, goodness, the good thing to do is not always the utilitarian thing to do, right? Yes, but I think it's possible to get down an ideological rabbit hole. This is not specific to utilitarianism. Right. I think this can happen to tons of things where when you say goodness, you just mean utilitarianism and you don't have a word for what everyone else meant by goodness, then I think that's really hard to recover from. And I think that's the kind of thing that can conceivably happen and maybe sometimes actually happens.AARONYeah, I guess as an empirical matter and like an empirical psychological matter and yes. Do people's brains ever operate this way? Yes. I don't really know where that leaves that leaves us. Maybe we should move on to a different topic or whatever.DANIELCan I just say one more thing?AARONYeah, totally.DANIELFirst, I should just give this broad disclaimer that I'm not a philosopher and I don't really know what I'm talking about. But the second thing is that particular final point. I was sort of inspired by a paper I read. I think it's called, like, do Christians and Muslims worship the same god? Which is actually a paper about the philosophy of naming and what it means for proper names to refer to the same thing. And it's pretty interesting, and it has a footnote about why you would want to discourage blasphemy, which is sort of about this. Anyway.AARONNo, I personally don't find this super interesting. I can sort of see how somebody would and I also think it's potentially important, but I think it's maybe yeah.DANIELActually it's actually kind of funny. Can I tell you a thing that I'm a little bit confused about?AARONYeah, sure.DANIELSo philosophers just there's this branch of philosophy that's the philosophy of language, and in particular the philosophy of right. Like, what does it mean when we say a word refers to something in the real world? And some subsection of this is the philosophy of proper names. Right. So when I say Aaron is going to the like, what do I mean by know who is like, if it turned out that these interactions that I'd been having with an online like, all of them were faked, but there was a real human named Bergman, would that count as making that send is true or whatever? Anyway, there's some philosophy on this topic, and apparently we didn't need it to build a really smart AI. No AI person has studied this. Essentially, these theories are not really baked into the way we do AI these days.AARONWhat do you think that implies or suggests?DANIELI think it's a bit confusing. I think naively, you might have thought that AIS would have to refer to things, and naively, you might have thought that in order for us to make that happen, we would have had to understand the philosophy of reference or of naming, at least on some sort of basic level. But apparently we just didn't have to. Apparently we could just like I don't have that.AARONIn fact, just hearing your description, my initial intuition is like, man, this does not matter for anything.DANIELOkay. Can I try and convince you that it should matter? Yeah, tell me how I fail to convince you.AARONYeah, all right.DANIELHumans are pretty smart, right? We're like the prototypical smart thing. How are humans smart? I think one of the main ingredients of that is that we have language. Right?AARONYes. Oh, and by the way, this gets to the unpublished episode with Nathan Barnard.DANIELComing out an UN I think I've seen an episode with him.AARONOh, yeah. This is the second one because he's.DANIELBeen very oh, exciting. All right, well well, maybe all this will be superseded by this unpublished episode.AARONI don't think so. We'll see.DANIELBut okay, we have language, right. Why is language useful? Well, I think it's probably useful in part because it refers to stuff. When I say stuff, I'm talking about the real world, right?AARONYes.DANIELNow, you might think that in order to build a machine that was smart and wielded the language usefully, it would also have to have language. We would have to build it such that its language referred to the real world. Right. And you might further think that in order to build something that use languages that actually succeeds at doing reference, we would have to understand what reference was.AARONYes. I don't think that's right. Because insofar as we can get what we call useful is language in, language out without any direct interaction, without the AIS directly manipulating the world, or maybe not directly, but without using language understanders or beings that do have this reference property, that's what their language means to them, then this would be right. But because we have Chat GPT, what the use comes from is like giving language to humans, and the humans have reference to the real world. But if the humans you need some connection to your reference, but it doesn't have to be at every level or something like that.DANIELOkay, so do you think that suppose we had something that was like Chat GPT, but we gave it access to some robot limbs and it could pick up mice. Maybe it could pick up apples and throw the apples into the power furnace powering its data center. We give it these limbs and these actuators sort of analogous to how humans interact with the world. Do you think in order to make a thing like that that worked, we would need to understand the philosophy of reference?AARONNo. I'm not sure why.DANIELI also don't know why.AARONOkay, well, evolution didn't understand the philosophy of reference. I don't know what that tells us.DANIELI actually think this is, like, my lead answer of, like, we're just making AIS by just randomly tweaking them until they work. That's my rough summary of Scastic gradient descent. In some sense, this does not require you to have a strong sense of how to implement your AIS. Maybe that's why we don't need to.AARONUnderstand philosophy or the SDD process is doing the philosophy. In some sense, that's kind of how I think about it or how I think about it now. I guess during the SDD process, you're, like, tweaking basically the algorithm, and at the end of the day, probably in order to, say, pick up marbles or something, reference to a particular marble or the concept of marble, not only the concept, but both the concept

Very imperfect transcript: bit.ly/3QhFgEJSummary from Clong: The discussion centers around the concept of a unitary general intelligence or cognitive ability. Whether this exists as a real and distinct thing. Nathan argues against it, citing evidence from cognitive science about highly specialized and localized brain functions that can be damaged independently. Losing linguistic ability does not harm spatial reasoning ability. He also cites evidence from AI, like systems excelling at specific tasks without general competency, and tasks easy for AI but hard for humans. This suggests human cognition isn't defined by some unitary general ability. Aaron is more open to the idea, appealing to an intuitive sense of a qualitative difference between human and animal cognition - using symbolic reasoning in new domains. But he acknowledges the concept is fuzzy. They discuss whether language necessitates this general ability in humans, or is just associated. Nathan leans toward specialized language modules in the brain. They debate whether strong future AI systems could learn complex motor skills just from textual descriptions, without analogous motor control data. Nathan is highly skeptical. Aaron makes an analogy to the universe arising from simple physical laws. Nathan finds this irrelevant to the debate. Overall, Nathan seems to push Aaron towards a more skeptical view of a unitary general cognitive ability as a scientifically coherent concept. But Aaron retains some sympathy for related intuitions about human vs animal cognition. Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

Summary (by Claude.ai)This informal podcast covers a wide-ranging conversation between two speakers aligned in the effective altruism (EA) community. They have a similar background coming to EA from interests in philosophy, rationality, and reducing suffering. The main topic explored is reducing s-risks, or risks of extreme suffering in the future.Winston works for the Center for Reducing Suffering (CRS), focused on spreading concern for suffering, prioritizing interventions, and specifically reducing s-risks. He outlines CRS's focus on research and writing to build a moral philosophy foundation for reducing suffering. Aaron is skeptical s-risk reduction is tractable currently, seeing the research as abstract without a clear theory of change.They discuss how CRS and a similar group CLR are trying to influence AI alignment and digital sentience to reduce potential future s-risks. But Aaron worries about identifying and affecting the "digital neural correlates of suffering." Winston responds these efforts aim to have a positive impact even if unlikely to succeed, and there are potential lock-in scenarios that could be influenced.Aaron explains his hesitancy to donate based on tractability concerns. He outlines his EA independent research, which includes an archive project around nuclear war. More broadly, the two find they largely ethically agree, including on a suffering-focused ethics and "lexical negative utilitarianism within total utilitarianism.Some disagreements arise around the nature of consciousness, with Aaron arguing rejecting qualia implies nihilism while Winston disagrees. They also diverge on moral realism, with Aaron defending it and Winston leaning anti-realist.As they wrap up the wide-ranging conversation, they joke about convincing each other and make predictions on podcast listens. They thank each other for the thought-provoking discussion, aligned in ethics but with some disagreements on consciousness and metaethics. The conversation provides an insider perspective on efforts to reduce s-risks through research and outreach.EA Archive: https://forum.effectivealtruism.org/posts/DndmvDGStD3gTfhXk Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

Summary/specific topics:- Stress Tests and AI Regulation: Nathan elaborates on the concept of stress tests conducted by central banks. These tests assess the resilience of banks to severe economic downturns and the potential for a domino effect if one bank fails. They believe that lessons from this process can be applied to AI regulation. Aaron agrees, but also highlights the need for a proactive approach to AI regulation, as opposed to the reactive measures often seen in banking regulation.- The Role of Central Banks in AI Regulation: Nathan suggests that institutions structured like central banks, staffed with technical experts and independent from government, could be beneficial for AI regulation. They believe such institutions could respond quickly and effectively to crises. However, they acknowledge that this approach may not be effective if AI development leads to rapid, uncontrollable self-improvement.- Compute Governance: The conversation then shifts to compute governance, which Nathan sees as a promising area for AI regulation due to the obviousness of someone using large amounts of compute. They believe that this could provide governments with a control lever over cutting-edge AI labs, similar to how central banks control banking loans and affairs.- AI Regulation and the Role of Public Actors: Nathan acknowledges that the leaders of major AI labs seem sensible and aligned with AI safety principles. However, they argue that regulation and public actors can play a crucial role in creating common knowledge between labs and preventing a race to the bottom. They also discuss the potential benefits and drawbacks of different regulatory approaches.- Financial Regulation as a Model for AI Regulation: Nathan believes that post-crisis financial regulation, such as the Dodd-Frank Act, has generally been effective. They suggest that AI regulation could follow a similar path, especially if AI becomes a significant part of the economy. However, Aaron expresses skepticism about the ability of political processes to produce effective AI regulation.- Regulation Before and After Crises: The speakers agree that pre-crisis regulation has generally been less effective than post-crisis regulation. They discuss the potential for AI regulation to follow a similar pattern, with effective regulation emerging in response to a crisis.- Regulatory Arbitrage: The conversation concludes with a discussion on regulatory arbitrage, where banks shift activities to where it's cheapest to do business. Despite evidence of this behavior, Nathan notes that there was no race to the bottom in terms of regulation during the financial crisis. Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

- Follow Arjun on Twitter: https://twitter.com/panickssery- Read and subscribe to his blog: https://arjunpanickssery.substack.com- A mediocre transcription can be found at https://www.assemblyai.com/playground/transcript/6x5h1mcemt-bff1-40fc-a676-9b59c66985f0

A transcript can be found at assemblyai.com/playground/transcript/6y7e7wz28c-30aa-4e83-ba4f-1bddf2e23dad Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

For All Good's inaugural episode, we talked to Rob Wiblin and Keiran Harris of 80,000 Hours about how and why they produce their show. This episode first appeared on their new feed "80,000 Hours: After Hours" here: https://80000hours.org/podcast/episodes/80k-after-hours-philosophy-of-the-80000-hours-podcast/ Hope you find it interesting! Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

Hear EA Georgetown member Aaron Bergman's recent interview as a guest on the Narratives Podcast! During the show, host Will Jarvis talks to Aaron about a key way he thinks people go wrong when choosing a career, how society treats children, how bureaucracy works, whether the FDA should have to approve medications, his interest in psychopharmacology, and a whole lot more. Check out Narratives at narrativespodcast.com or search for it wherever you're listening to this episode! Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

Welcome to All Good, a show by Georgetown Effective Altruism. Get full access to Aaron's Blog at www.aaronbergman.net/subscribe

Update Feb 12, 2023: Automated audio experiment with the Automator app no one usesIntroBack in 2016, I got my first job as a summer camp counselor. It was an outdoor adventure day camp, to which the six figure-making lawyers and consultants of Washington, D.C. sent their straight-A getting, lacrosse-playing children for a taste of the great outdoors. The campers ranged from four years of age to 15, with those nine and up able to choose which among the activities—kayaking, rock climbing, horseback riding, and more—to pursue. I, as a typical sheltered suburbanite (though a competitive rock climber rather than a lacrosse player), had attended the camp myself throughout my childhood. We Junior Counselors, or “JCs,” could request which age-activity combinations to supervise. Though I had to do my fair share of toddler-wrangling, I was granted a few weeks in the Big Leagues: the rock climbing program for kids aged 9-15. Each cohort of about 12 campers had 3 staff members: a JC, a head counselor or “HC” at least 18 years old, and an “instructor” to provide technical expertise. As a 16 year old mere months older than some of my campers, it was a little bizarre. I was an Adult, getting paid (barely) to ensure that the youth in my care remained happy and healthy in the rugged Maryland wilderness. By and large, it was a good camp. What I'm about to say doesn't negate the overall-positive experience of most campers.LyingStaff lied to campers constantly. We were expected to, and generally obliged. Not about important things, either. In fact, the lies generally concerned utterly trivial matters. Child protection is goodNow, I'm not entirely naive. I understand that placing children of all genders in the care of three young adults literally in the woods, including on one overnight campout per week, is a situation tailor-made for abuse. And the camp, to its credit, took this very seriously. We had training after training and rules and guidelines galore designed to ensure that no staff member would ever be tempted to misuse his or her power and status.This was a good thing. Child abuse is bad, and I'm glad we erred on the side of “careful.” I understood that my relationship with the campers was not to be intimate. Staff shouldn't, and didn't, reveal the skeletons in their closet to middle schoolers at rock climbing camp. No playing “truth or dare” around the campfire, either. Understood.Casual dishonestyBut did we really have to lie so much? Take my age. Though I don't recall being explicitly told “you may not reveal your age to campers,” doing just this was an obvious norm among staff. Like any normal human being, the campers wanted to know some basic biographical information about the people they were spending their summers with. This was particularly true for me, a 16 year old who looked about 14 on a good day, whose size-small “counselor” tank top was held up with safety pins to prevent the neck hole from drooping down to my nipples. My fellow counselors, when inevitably asked this question, would say something like “you have to guess” or “six hundred” or “maybe if we win the song competition I'll tell you at the end of the week” (they wouldn't). Nobody said “17” or “31” or “sorry, I can't tell you that.”Though human memory is fallible and it's been five years, I'm pretty sure I got tired of telling stupid, pointless lies. When campers asked, I made the completely banal decision to answer their question. Our conversations generally went like this:Camper: How old are you?Me: sixteen.Camper: You look like you're [insert number between 11 and 15].Me: I know.Camper: Ok.And that was that. My authority remained intact. No one stopped listening to me. Maybe I'm indulging myself, but I think the campers might have been pleasantly surprised at, you know, not being lied to for once. I didn't only do this for the sake of my campers' dignity. I really did look younger than some of the campers, and it was in my own best interest that they be aware I was not. My fellow counselors didn't get mad at me. They didn't bring it up. They just kept on lying to the kids, and I did not. I can't say for sure I was the only one who told the truth, but I sure never heard anyone else do the same.It wasn't just my age. They'd ask which school I went to, and (if my memory serves correctly) I told them. I'm sure they asked some things I really couldn't or shouldn't have answered, and then I wouldn't tell them. Ageism Back in my gung-ho veganism days, I remember thinking the term speciesism was really dumb. Damn right, we should treat different species differently. Unlike with, say, racial identity, there really are important differences—moral and otherwise—between dandelions, lobsters, chickens, and humans. I still think the term is ripe for misinterpretation, but I now see the underlying concept as sound. Speciesism is arbitrary differential treatment, rather than differential treatment justified by real differences in underlying traits. Ageism is the same way. We really should treat young kids differently than adults, but only because they have different underlying traits—not because of their age per se. If we don't think that kids' have the cognitive capacity or confidence or something to make big life decisions or competently assess risk, fine.I'm not saying we should let infants sign contracts, and I'm not even saying we should never lie to children. Whatever Kant would have you think, dishonesty is sometimes the lesser of two evils and therefore the right thing to do, but the chronological age of the person you'd be lying to isn't a good enough justification. The socially acceptable prejudiceI guess great minds think alike, because after writing this subheading I came across the AARP's article “Workplace Age Discrimination Still Flourishes in America.” Under its own subheading, “Ageism: An accepted bias,” it argues thatageism in the workplace occurs every day across America, and it is tolerated or — even worse — unrecognized for what it truly is: discrimination, plain and simple.“Age discrimination is so pervasive that people don't even recognize it's illegal,” asserts Kristin Alden, an attorney specializing in employee rights at the Alden Law Group in Washington, D.C. Ageism is most commonly discussed in the context of employment discrimination, but its social acceptability rings just as true in the context of other, non-workplace incarnations. Reasons whyAnyway, I think there are two fundamental reasons why ageism is so common and acceptable:* Age really does correlate with differences in underlying characteristics like mental aptitude.* All adults were once children, and most adults expect to eventually get old.The first reason seems pretty intuitive; it really can be hard to tell what is justified differential treatment (say, not letting toddlers sign legally-binding agreements) and what is straight-up, arbitrary bias.The second point is more subtle. It's the “I can't be racist—I have a black friend!” of ageism. “I can't be biased against children,” we think, “I was a child myself once!” I think there's a bit of retributive justice going on, too: if I had to endure a childhood of casual, well-meaning disrespect, why shouldn't I get to enjoy the status-privileges of adulthood?These narratives are rarely made explicit. I'm sure I've been disrespectful to children before without saying such things out loud, or even really thinking about them directly. Nor are they malicious. When my fellow counselors and I were lying to the campers, we thought of it as a big joke. We were teasing them, playing a game the campers never agreed to join.This is why ageism is so pernicious. It's easy to moralize racism and sexism; there are the oppressors and the oppressed. What happens when we're all on both sides of the power structure, albeit at different times in our lives? This bias, well-intentioned and hidden in plain sight, doesn't fit well into a compelling moral narrative. Candy-coated contempt If you only read one link in this post, make it the Atlantic's “There's a Better Way to Parent: Less Yelling, Less Praise” (warning: uses up one of your monthly free articles), an interview with the author of Hunt, Gather, Parent: What Ancient Cultures Can Teach Us About the Lost Art of Raising Happy, Helpful Little Humans. Part of it reads:Joe Pinsker: Many American parenting strategies, you estimate, are only about 100 years old, and some of them arose more recently than that. What about American parenting sticks out to you as distinctive and particularly strange?Michaeleen Doucleff: One of the craziest things we do is praise children constantly. When I was first working on the book, I recorded myself to see how frequently I praised my little girl, Rosy, and I noticed that I would exaggeratedly react to even her smallest accomplishments, like drawing a flower or writing a letter, with a comment like “Good job!” or “Wow! What a beautiful flower!”This is insane if you look around the world and throughout human history. Everywhere I went, I don't know if I ever heard a parent praise a child. Yet these kids are incredibly self-sufficient, confident, and respectful—everything we want praise to do, these kids already have it, without the praise.It's hard to cut back on praise, because it's so baked in, but later on, I decided to try. It's not that there's no feedback, but it's much gentler feedback—parents will smile or nod if a child is doing something they want. I started doing that, and Rosy's behavior really improved. A lot of the attention-seeking behavior went away.Doucleff emphasizes how excessive praise induces attention-seeking behavior, but I'd like to draw attention to its direct, first-order effect: children feel disrespected. BaseballI started playing baseball in first grade, and didn't stop for nearly a decade. During elementary school, I distinctly remember loathing the hollow, meaningless compliments that coach after coach fed to us players (which my father just confirmed as a regular complaint of mine, so I'm not making this up). Practices and games were a constant stream of “Way to go down swinging, Jackson!” and “Great effort, Mike!” If you had asked me at 8 why I hated these “compliments,” I'm not sure what I would have said. In retrospect, though, I think I (correctly) understood them as a subtle reminder that we elementary schoolers were unworthy of being treated like normal human beings. After all, I knew the coaches were being disingenuous most of the time; it just wasn't plausible that everything we did was so commendable. In other words, I knew that adults were lying to me. Perhaps the word “lying” is a little strong, but I'll stand by it. Saying “great job” when you know damn well that the job wasn't great, with intent to deceive, seems like a lie to me—however well-intentioned. (Literally) feels bad, manThere's another subtle objection that I'd like to preempt. In the annals of culture-war adjacent academia, there is plenty of discussion of things like “dehumanization,” “stereotype threat,” and structural racism or sexism, which some folks are skeptical of. Whatever you think about these things, I'd like to distinguish them from what I've been calling “casual disrespect” of kids. Some social theorists purport these institutions to be important because of their subtle, nefarious, perhaps even subconscious impact on the structure of society. Even if nobody is overtly offended by bits of casual sexism thrown around, the thinking goes, such behavior helps to perpetuate a structural imbalance between the power and autonomy of men and women.I'm not disputing this, but once again let me emphasize the direct, first-order effects of “casual disrespect.” It is not merely that kids are disempowered or whatever. It literally makes them feel bad! Ok, I can't say for sure what's going on in other people's heads, but it sure as hell made me feel bad, and it sure seemed like my campers felt bad when they were being lied to. We can speculate about the second-order, structural effects of ageism all we want, but let's not forget that people feel bad when they sense they're being disrespected, and kids are people too. Not just dishonestySo far, I've focused on dishonesty because it's the thing I have salient personal experience with (as a camp counselor and baseball player). That said, there are countless other ways that we adults treat children inappropriately without giving it a second thought.Mannerism and tone of voiceYou know how everyone talks to dogs and babies in weird, high-pitched voices? Yeah, we do a slightly toned-down version to kids as old as, say, 13, and I don't think it's cool. This video (starts at 0:52, watch for about 30 seconds) is a fascinating example. The speaker is explicitly demonstrating how to treat children respectfully and yet still talks in a distinctly “baby talk” tone of voice. It goes like this:The first thing to remember is, children are humans, just like you.If you're not used to being around kids, I know it can be tempting to speak down to them or talk to them in baby talk, but honestly, you really don't have to do that. You can speak to a child the same way you speak to anyone else. In fact, that's the best way to speak to them…For example, it's better to say, even to a five month old infant, “Would you like your bottle?” versus “Baby want a baba?”…For instance, you could say, “Can you point to that crayon?”…The bolded lines are spoken in ‘baby talk-lite,' and I don't blame her. When I imagine speaking to an eight year old exactly the way I would speak to a peer (in tone and manner, not content), it feels weird. Instead, I instinctively make the pitch of my voice a bit higher, add more inflection to my voice, and otherwise subtly adjust my mannerisms. It's not full-on “baby-talking,” but it is a step in that (wrong) direction.Content censorshipI'm perfectly willing to accept that children are sometimes genuinely not mature enough to handle certain content. I, for example, had nightmares for weeks after watching the movie Contact when I was about seven. I'm pretty sure that wouldn't happen to me now. But I think we sometimes deem things “inappropriate” without even considering whether the content could plausibly cause anything bad to happen. If you think there's a reasonable chance your kid might learn that violence is ok or have nightmares after watching a violent movie, fine, don't show it to him. But if you're inclined not to let your kid watch Zero Dark Thirty or something because of some vague notion that it's “inappropriate,” consider interrogating this intuition.Compared to violence, it's even harder to justify our sex prohibition. I'm far from the first person to note America's prudish tendencies, but even liberal, sex-positive people often concede that kids shouldn't see naked bodies or watch people have sex on screen. Again, there are plausible justifications for this. If a sex scene is borderline (or not so borderline) abusive, you might worry that a child would learn the wrong lessons about power and consent and such. Likewise, I think everyone should be wary of regularly consuming superstimuli like porn, children included. That said, literally what bad thing is going to happen if your kid sees Rose naked in Titanic? I managed to survive the movie as a kid without becoming a sex addict. Curse words are very similar to content censorship. I get it, we don't want third graders running around yelling “FUCK” just to get everyone's attention at Walmart. But what, exactly, is so problematic with children using the same words that adults use in the appropriate setting? Path dependence and coordinationThere is one very real barrier to treating children more like normal human beings: coordination. If your child is the first of her peers who knows how babies are made, other parents might get upset if little Julia relates this information to her friends. Likewise, even if you're ok with Johnny cursing after he stubs his toe, others might look down on you both. Breaking social norms ain't easy. Even still, there are some things for which being the first mover isn't too difficult. If you're the only baseball coach who doesn't ceaselessly offer empty praise to the players and all the kids like you just as much, you're really not risking your reputation. Likewise, other adults might be a little surprised when you speak to a toddler in a completely normal tone of voice, but they're not going to exile you from polite society. Perhaps, gradually, the norms can creep in the right direction. ConclusionAccording to my parents, my second grade teacher told them that I “acted like a 40 year old man.” I still don't know what she meant by this—I cried all the time, so it definitely wasn't emotional stability. The point, though, is that perhaps my childhood experience was not representative of others'. If so, maybe kids really aren't harmed or offended by being told untruths and being prevented from watching violent movies. Maybe my campers really didn't mind when we refused to reveal our ages, and maybe other children genuinely prefer when adults speak in a modified tone of voice. But, to be frank, I don't think so. I may have been more interested in physics than the median first grader (an interest I have since lost), but I highly doubt that my entire experience was sui generis. After all, I was playing baseball and basketball and doing Cub Scout stuff and playing trumpet and going to summer camp just like so many of my peers. And, just like them, I endured the same needless censorship and well-intentioned disrespect so deeply ingrained in our culture. Get full access to Aaron's Blog at www.aaronbergman.net/subscribe