POPULARITY
9 episodes with merwijk
75 episodes with merwijk
5 episodes with merwijk
3 episodes with merwijk
2 episodes with merwijk
7 episodes with merwijk
7 episodes with merwijk
2 episodes with merwijk
2 episodes with merwijk
Speellijst Zondag 29 december 2024 Oudejaarsfragment 2022 (De Breij) Claudia de Breij 2'00 Youtube Oud en nieuw (Drs. P) Drs. P 3'23 Van de cd Drs. P retrouvé TN1619CD Anemoon (De GroofVan der Tak) Bart de Groof 3'19 Van te verschijnen cd Wij zijn er nog (Rot) Jan Rot 3'36 Van de cd i.p.v. kaarten Eigen beheer Mijn moeder het water (Koole/De Munnik) Ricky Koole 4'02 Van de LP Altijd iemand Coolhouse CH70311 Zo is Wim Kan (Davids/De Corte) Jules de Corte 2'01 Eigen opname Van Puthoven (Kan) Wim Kan 8'57 van de Lp Oudejaarsavond 1982 VARAGRAM ES 156 Volgend jaar (J. de Jong/De Toestand) De Toestand Jeninke De Jong 2'53 Eigen opname De optocht (De Wijs/Fondse) Britta Maria & Maurits Fondse 4'00 Van de cd De omweg Eigen beheer Het Kistje (De Jonge) Freek de Jonge 10'43 Van de cd De Openbaring COL 471383 2 Ik compenseer) Jeroen van Merwijk 2'02 Van de cd Was volgend jaar maar vast voorbij Eigen beheer Henk belt Bert (Teeuwen, Bouwman) Hans Teeuwen, Pieter Bouwman 3'53 Van de cd De mannen van de radio CDHSP 34 MR
Speellijst 17 november 2024: liedjes en zingen Zoeloe liedje / Jeroen van Merwijk 0'42 van de cd Live from the Papenstraat Theater. Eigen beheer. Zin (Laan/De Wijs) Jenny Arean 2'45 van de cd In concert Brigadoon BIS 099 Ze kan niet anders (H. v. Veen) Herman van Veen 2'55 Van de cd Moeders Harlekijn 457 951-1 M'n eerste (D. Witte) Gerard Cox 4'06 Van de cd Wat je zingt dat ben je zelf NN 500.203-2 Kamerbreed (Brey)Erik Brey 3'30 Van de cd Erik Brey Live Eigen beheer Het Nederlandse lied (Wiersma) Dorine Wiersma 2'55 van de cd Andermans veren 25 jaar CNR 22 515941 2 Ode aan Nick en Simon (Polderman) Katinka Polderman 4'00 van de cd Erge dingen vind ik ook heel erg Bunker Harde smart (Drs. P) Drs. P 4'00 van de cd Drs. CD Concerto 71818 Dollen met volksliedjes Toon Hermans 7'00 Van de cd Toon Hermans 75 jaar EVA 7984742 Een lied uit vroeger tijden (Long) Robert Long, Simone Kleinsma 4'18 Van de cd Het onherroepelijke Fantastico Album Universal 471 410-1 M'n blootje (D. Engers/W. Sonneveld) Dominique Engers en Hans van Gelderen 4'55 Eigen opname Eigenlijk mag ik nergens over zingen (Nijland) Theo Nijland 5'14 Van de cd Masterclass BASTA 3091812 Zing dan Jenny Arean 5'20 van de cd in Concert Brigadoon BIS 099 Wat een leuk bandje (instrumentaal) (Nijland) Theo Nijland 2'30 Van de cd Masterclass BASTA 3091812
Speellijst Zondag 29 september 2024 Zondag (Van 't Hek) Youp van 't Hek 1'57 Van de cd Ergens in de verte CNR 2001012 Laura (Winterland) Jonas Winterland 2'58 Van cd bij het boek Berichten uit de schemerzone Eigen beheer Idool Rob de Nijs (Eykman/Bannink) Wieteke van Dort 2'17 Van de LP J.J. De Bom voorheen: kindervriend Polydor 2441 027 Tijd voor taart (Van Rooijen) Peter van Rooijen 1'37 Van de cd Liefde, dood en taart Eigen beheer Overal auto's (Van Rooijen) Peter van Rooijen 1'59 Van de cd Liefde, dood en taart Eigen beheer Een goede reden (Vermeulen) Bam Vermeulen 4'53 Eigen opname Witte rozen (Tummers/E. Paoli) Willy Derby 2'38 van de cd Willy Derby FAV 1-95196 Het hart eener deerne (Drs. P) Adèle Bloemendaal 3'57 Eigen opname Domding domding (R. Zuiderveld) Elly Nieman 2'50 Van de cd Jarenlang Horizon DCD 5042 Opstijftijd (Nochem) Marc Nochem 3'05 Eigen opname Het maken van een conference (fragment) (F. de Jonge) Freek de Jonge 4'29 Van de cd Ik zou je het liefste in een doosje willen doen NN 500.003-2 Helden (A. Groothuizen/N. Brandsen) Angela Groothuizen 3'13 Van de cd Eeuwige jeugd Eigen beheer Egoïsme (Van Merwijk) Pieter Bouwman 2'17 Van de cd Leve van Merwijk! Eigen beheer Op mijn laatste tocht (Dolstra) Sietze Dolstra, Evert de Vries 1'41 Eigen opname Vroeger (Ede Staal) Juul Kabas 2'36 Van de cd 't Een en 't ander Dade prod. 00198 André (Dolstra/Stoots) Sietze Dolstra 3'11 Van de LP Twee levens CBS 83958 Op reis (Van der Wurff) Erik van der Wurff eigen opname
Harrie Jekkers is cabaretier, zanger en schrijver. Hij schreef evergreens als ‘Over de Muur' en ‘O, o, Den Haag', en was zanger van muziekgroep Klein Orkest. Hij maakte en speelde verschillende cabaretprogramma's, en toerde door de theaters met Jeroen van Merwijk als ‘Jekkers en Jeroen'. Hij won een Edison, Annie M.G. Schmidt-prijs en Zilveren Griffel. Zijn voorstelling ‘In mijn liedjes kan ik wonen' is een gezongen autobiografie met liedjes van hemzelf, maar ook van zijn grootste inspiratiebronnen.
Zondag 16 juni 2024 Vaderdag Waarom denk ik niet vaker aan mijn vader? (Van Merwijk) Jeroen van Merwijk 1'24 Van de cd Van Merwijk legt het nog één keer uit HG 110755 A kwadraat plus B kwadraat (Wilmink/Bannink) Frits Lambrechts 2'21 Van de cd Portret BIS 073 Oorlogswinter (Dorrestijn/Bannink) Joost Prinsen 3'25 Van de cd Een kop die je zelf niet bevalt BASTA 30-9139-2 Vaders vakantie (Dorrestijn) Hans Dorrestijn 1'42 Van de cd Liederen van wanhoop en ongeloof II BMMCD 273 Maandagmorgen na het weekend (Dorrestijn) Hans Dorrestijn 2'46 Van de cd Liederen van wanhoop en ongeloof II BMMCD 273 Het verjaardagskado (Dorrestijn) Hans Dorrestijn 1'17 Van de cd Liederen van wanhoop en ongeloof II BMMCD 273 Begrafenis (De Jonge) Freek de Jonge 4'07 van de cd Losse nummers COL 471393 2 Vader (Blanker/Sprenger) Peter Blanker 3'52 Van de LP Neem de tijd Fontana 6391 044 Ik kan hem niet zijn (Frencken) Mylou Frencken 3'27 Van de cd Wegwaaien Eigen beheer De mannenclub (Wiersma) Dorine Wiersma 4'03 Eigen opname Als ik mijn vader was (Gadellaa/Ennes) Karin Bloemen 1'48 Van de cd Weet je nog? Eigen beheer Vader (P. v. Vliet) Paul van Vliet 2'40 Van de LP Wat gaan we doen? QS 600 801/2 Streepjescode (Torn) Kees Torn 1'48 Uit dvd/cd-box Een ommetje met Kees Torn Bunker Vader op een fiets (Long) Robert Long 3'27 Van de cd Achter de horizon Universal 471 409-9 Over zijn vader en de helikopter (Meinderts) Koos Meinderts 3'13 Van de cd Jekkers & Koos: Het verhaal achter de liedjes CNR 22 204402 De leugenaar (Meinderts, Smit, Jekkers) Klein Orkest Van de cd Alles Polydor 531 731-2 Pa wil niet in bad (R. Ortegar/R. de Gooyer/J. Hartman) Johnny & Rijk 2'52 Van de cd Oh Waterlooplein Disky BX 649642
Zondag 21 april 2024 Twee gedichten (I. de Wijs) Ivo de Wijs 0'28 Van de cd bij het boek He gaat goed met Nederland Nijgh & van Ditmar Zolang (Lohues) Jenny Arean 2'51 Van de cd Jenny Arean solo BIS 066 Verliefd (Lohues) Stephanie Struijk 3'43 Van de cd Liedjes van een ander Eigen beheer Zowat altijd bijna (Lohues) Esther Groeneberg 2'57 Van EP Ik heb het licht nog aan Spotify M.S. (Nieuwint) Pieter Nieuwint 2'51 Van cd/boekje Pieter Nieuwint dicht en zingt Mirasound Het wijnglas (D. Witte) Harry Slinger 2'54 Eigen opname Geef me de moed (Aznavour/J. v. Dongen) Margje Wittemans 3'59 Eigen opname Een oorlog tegelijk (J. v. Merwijk) Jeroen van Merwijk 3'06 van de cd Van Merwijk goes The Blauwe Zaal TBA HG 301194 Dag heren van het CDA (Long) Robert Long, Leen Jongewaard 3'35 Van de cd 10 Jaar theater cd3 EMI 74 6952 2 Interimperen (Van Kooten, de Bie) Jacobse en Van Es 5'38 Van de cd Gouden doden SVMS1 Er is nog zoveel niet gezegd (P. v. Vliet) Paul van Vliet 3'20 Eigen opname Knagen (Fokkema) Marjolein Fokkema 2'28 Eigen opname Serie gezien (Pelgrim) Roger Pelgrim 3'32 Van de cd Dit is niet het eind Eigen beheer Algoritmes (Visser) Joop & Jessica 2'22 Eigen opname Mountainbike (Pons) Cornelis Pons 2'16 Van de cd Bar intiem VB 1-121999 Huize Muisje (Gitsels/Slinger) Harry Slinger 2'50 van de cd De Muisjes Eigen beheer
Speellijst Zondag 10 maart 2024 Pas op voor de hitte (A. Schmidt) Annie M.G. Schmidt 1'03 Van cd verhalen versjes liedjes Rubinstein Deze vrouw (F. Wiegersma/W. Sonneveld) Wim Sonneveld 3'38 Van cd bij het boek Telkens weer het dorp / Friso Wiegersma NN 500.602-2 Geboren op een snelweg (Nuissl) Joost Nuissl 3'07 Van de LP Ja, nee en samen Harlekijn HH 2925 509 Litanie bij een terugkeer (Brel/Rauber, Van Altena) Beatrice van der Poel 2'05 Van de cd Beatrice zingt Brel MG23004 Vrouwendag (Zwaving) Eva Zwaving 2'18 Eigen opname De volgende (Walschaert) Kommil Foo 7'11 Van de cd/dvd Wolf PIASCOM Publiek op zondag Sonneveld) Wim Sonneveld 3'24 Eigen opname Aurora (Leemans) Sara Leemans 2'32 Van de cd vanavond bij jou? Eigen beheer Lekker jezelf (Crutzen) Eva Crutzen 2'50 Van cd Spiritus Eigen beheer Ergens is het fout gegaan (Van Holstein) Alex van Holstein en Eefje de Visser 3'40 Van de cd Papieren vogel Eigen beheer Was ik Ivo Niehe maar (Polderman) Katinka Polderman 3'30 Van de cd Dan kijk ik naar mijn saldo PIASCOM656 Onze stem (Dorfmann/Lelivelt) Anouk Dorfmann, Haytham Safia, Charivari Trio 4'44 Eigen opname De winnaar krijgt de macht (Andersson & Ulvaeus, Coot van Doesburgh) Simone Kleinsma 4'14 Van de cd Mamma Mia! Universal 981 372-1 Wat zijn de vrouwen groot (Van Merwijk) Jeroen van Merwijk 2'24 Van de cd Zelfportret met elektrische gitaar TBA HG 281098
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Extinction Risks from AI: Invisible to Science?, published by Vojtech Kovarik on February 21, 2024 on The AI Alignment Forum. Abstract: In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart's Law as "Virtually any goal specification, pursued to the extreme, will result in the extinction[1] of humanity'', and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart's Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to be informative for evaluating specific arguments for Extinction-level Goodhart's Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods. Together with Chris van Merwijk and Ida Mattsson, we have recently wrote a philosophy-venue version of some of our thoughts on Goodhart's Law in the context of powerful AI [link].[2] This version of the paper has no math in it, but it attempts to point at one aspect of "Extinction-level Goodhart's Law" that seems particularly relevant for AI advocacy --- namely, that the fields of AI and CS would have been unlikely to come across evidence of this law, in the environments typically studied in these fields, even if the law did hold in the real world. Since commenting on link-posts is inconvenient, I split off some of the ideas from the paper into the following separate posts: Weak vs Quantitative Extinction-level Goodhart's Law: defining different versions of the notion of "Extinction-level Goodhart's Law". Which Model Properties are Necessary for Evaluating an Argument?: illustrating the methodology of the paper on a simple non-AI example. Dynamics Crucial to AI Risk Seem to Make for Complicated Models: applying the methodology above to AI risk. We have more material on this topic, including writing with math[3] in it, but this is mostly not yet in a publicly shareable form. The exception is the post Extinction-level Goodhart's Law as a Property of the Environment (which is not covered by the paper). If you are interested in discussing anything related to this, definitely reach out. ^ A common comment is that the definition should also include outcomes that are similarly bad or worse than extinction. While we agree that such definition makes sense, we would prefer to refer to that version as "existential", and reserve the "extinction" version for the less ambiguous notion of literal extinction. ^ As an anecdote, it seems worth mentioning that I tried, and failed, to post the paper to arXiv --- by now, it has been stuck there with "on hold" status for three weeks. Given that the paper is called "Existential Risk from AI: Invisible to Science?", there must be some deeper meaning to this. ^ Or rather, it has pseudo-math in it. By which I mean that it looks like math, but it is built on top of vague concepts such as "optimisation power" and "specification complexity". And while I hope that we will one day be able to formalise these, I don't know how to do so at this point. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
Een fles met tijd (Groce/Vreeswijk) Cornelis Vreeswijk 2'27 Van de cd De nozem en de non HM 1275-2 Trappetje af (Gratama/Verbeke) Rients Gratama 3'29 Van de cd Ik zou je het liefste in een doosje willen doen NN 500. 001-2 neveL (Riemens) Karin Bloemen 4'25 Van de cd Van de gekken Universal 270 191-1 Stokrozen (R. Chrispijn/H. v. Veen) Jurrit de Bode, Andermans Veren Live 3'30 Van de cd Er is altijd een derde De minutenwals (Chopin/Ab Westervaarder) Jasperina de Jong 2'12 Van de cd portret BIS 022 Intro Fasant met zuurkool (F. de Jonge) Freek de Jonge 1'51 Van de cd Weerzien in Panama EMI 7 92456 2 Men fietst niet meer (Finkers) Herman Finkers 2'44 Van de cd na de pauze CNR 2005546 Waar blijft de tijd (J. de Corte/K. Torn) Kees Torn 3'02 Eigen opname Verloop (I. de Wijs) Marnix Kappers 2'25 Van de cd bij het boek Het gaat goed met Nederland Nijgh & van Ditmar Een seconde (Jaap Bakker/Frans Ehlhart) Joke Bruijs 2'08 Eigen opname De tijd tikt (H. v. Veen/H. v. Veen, E. van der Wurff) Mathilde Santing 2'42 Eigen opname Er is een tijd (H. v. Veen/E. Leerkes) Herman van Veen 2'10 Van de cd Dat kun je wel zien Harlekijn 450 372-0 Moeilijk weer (Fokkema) Marjolein Fokkema 2'29 eigen opname De tijd (Jekkers, Meinderts) Harrie Jekkers 4'17 Van de cd Ik zou je het liefste in een doosje willen doen NN 500.004-2 Het graai (Van Merwijk) Jeroen van Merwijk 2'34 Van de cd Er zijn nog kaarten! Eigen beheer De tijd stond even stil (R. Shaffy) Ramses Shaffy, Liesbeth List 2'48 van de cd Samen Universal 273 151-4 De klokken-canon (Trad.) Wim de Bie 1'51 Van de cd Wim de Bie zingt a capella SVSC 2 De tijd (L. Gerritsen/B. van der Linden) Liselore Gerritsen 2'13 Van de Lp Oktoberkind Utopia 6399 380
Zondag 10 december 2023 Als na mijn dood (van Merwijk) Jeroen van Merwijk 2'35 Van de cd Ik ben een vrouw Eigen beheer De zachte krachten (Henriëtte Roland Holst/Jan-Jaap Jansen) Ineke ter Heege en Jan-Jaap Jansen 3'00 Van de cd Kom vanavond met verhalen Theatergroep de Kern Eigen beheer Klaartjes moeder (Wilmink/Bannink) Cilly Dartell en Harry Bannink 2'26 Van de cd De tekenaar heeft een man gemaakt JJCD 1902 Joop en Gerard (Van Muiswinkel, Van Vleuten) Van Muiswinkel & Van Vleuten 5'20 Van de DVD Mannen met vaste lasten Universal 822 863-9 Restons ensemble (Shaffy) Machteld van der Gaag 3'04 Van de cd Machteld chante Shaffy Eigen beheer Victoria (T. v. Leer/L. Nijgh) Liesbeth List 4'27 Van de cd Wereldreis Universal 584 311-9 Op de Avonden (Kraaijkamp) Maria Kraaijkamp 2'24 Eigen opname Schraalhans keukenmeester (G.K. van het Reve) Gerard Kornelis van het Reve 3'23 van EP Ik bak ze bruiner Catfish 5C 023-24110 M God is niet dood (G. Vleugel/R. v. Otterloo) Gerard Cox 2'40 Van de cd Wie was bang voor Lurelei? Mercury 512109-2 Reve leest Reve, drie gedichten 2'21 Eigen opname Finkers leest Reve 4'10 Eigen opname O mens gedenk dat ge zijt sterfelijk Reve zingt Reve 0'56 Eigen opname Brommer (Trad/W. van Baarsen) Willem van Baarsen, Släpstïck 2'20 Eigen opname Ik neem jou mee (G. Testa/Van Maasakkers) Gerard van Maasakkers 3'17 Van de cd Allez Live Eigen beheer Verlaten gebouw (Van het Groenewoud) Raymond van het Groenewoud 3'20 Van de cd Egoïst Eigen beheer
Speellijst Zondag 3 december 2023 Mijn vader rookt pijp (B. Bos/J. Stokkermans) Wieteke van Dort 1'17 Youtube Hij doet het niet (T. Bos/J. Hoogeboom, M. v. Roozendaal) Johan Hoogeboom, Koen Franse, Martine Sandifort 1'06 Van de cd Otje Warner 3984-25142-2 Ik ben anders dan de rest (J. Myer/Reinout Douma) Jochem Myer 2'52 Youtube November (J.C. Bloem/J. Rot) Jan Rot 3'00 Van de cd Schout bij nacht Phonogram 526 499-2 Kus me niet (Koole) Ricky Koole 2'48 Van de LP Altijd iemand Eigen beheer Klaverveld (Sting/A. de Bont, J, Kramer) Leonie Jansen en Carel Kraayenhof 4'00 youtube Sneeuw op Luik (Brel/Barnard, Van der Poel) Beatrice van der Poel 3'40 Van de cd Beatrice zingt Brel Eigen beheer Ik compenseer (Van Merwijk) Jeroen van Merwijk 2'00 Van de cd Was volgend jaar maar vast voorbij Eigen beheer Was ik maar (Beuving/Dicke) Jan Beuving 3'12 Eigen opname Snieklaas (Hermans) Toon Hermans 5'30 Van de cd Toon 75 jaar EVA 7984642 Bananen (Drs.P) Drs. P 2'48 Van de cd Drs. P compilé sur cd Polydor 847 920-2 Ik ben zo chic (Elsink) Liesbeth List 3'42 Eigen opname Perspectief (Ko de Laat/Eva Zwaving) Eva Zwaving 1'47 Eigen opname Jezelf op nummer 1 (Jeninke de Jong/Jaap Snellen en Lucas de Gier Jeninke de Jong 3'21 Eigen opname Een mooie avond (J. Boerstoel/M. v. Dijk) Adèle Bloemendaal 2'01 Van de cd Adèle CNR 655.336-2 Casablanca (M. v. Dijk/J. Klöters) Adèle Bloemendaal 3'57 Van de cd Adèle CNR 655.336-2
Politiek (B. Vermeulen) Elfie Tromp 3'46 Youtube De zegen des vrouwenkiesrechts (Witte) Joost Spijkers 2'58 Van de cd Dirk Witte Compleet. Uitg. Zaantheater. De Nederlandse leeuw is ziek (Kan) Wim Kan 5'55 Van de cd Ik zou je het liefste in een doosje willen doen NN 500.200-2 De vogels (De Corte) Jules de Corte 3'04 Eigen opname Balkenende (De Jonge) Freek de Jonge 4'40 Van de cd Van de gekken Universal 270 191-1 Wouter (Van Merwijk) Jeroen van Merwijk 3'20 Eigen opname Tv interview met excellentie (Boddé, Van Luyn) Mike & Thomas 2'43 van de dvd Seks drugs & barok PIASCOM123 Doe eens normaal man (Visser) Joop & Jessica 2'16 Youtube Interrectuelen (Van Kooten, de Bie) Van Kooten & De Bie 2'53 Van de cd Op hun pik getrapt! SVCD 4 Nederland-Commissieland (A. de Haas) Alex de Haas 3'45 Van de LP Alex de Haas de dichter-zanger Polydor 656 010 Koekoek (Fillekers) Ted de Braak 3'20 Van de LP 10 Jaar Farce Majeure Philips 6678 019 Tune Veertje Stem op mij L. Vermeulen) Lotte Velvet 1'36 Eigen opname Nederland een paradijs (Gaaikema, Vermaat) Seth Gaaikema 2'27 Van de cd Ik zou je het liefste in een doosje willen doen NN 500.005-2 De populaire partijpotpourri (T. Eyk/Van Kooten, de Bie) Van Kooten & De Bie 2'08 Van de cd Op hun pik getrapt! SVCD 4
Speellijst Zondag 12 november 2023 Kwatrijnen (Van Muiswinkel) Erik van Muiswinkel 2'05 Eigen opname Rijm Frank Groothof 1'05 Van de cd 15 jaar Sesamstraat WSP CD 16108 Tussen acht en halfnegen (S. Mahkno/C. Verdier, G. den Braber) Thérèse Steinmetz 2'56 Van de cd Ik zing een lied voor u Pilarus 20191121 November (M. Cools/J. Kruit) Miel Cools 4'08 Van de cd Niet bang zijn Eufoda 1316 Daar maken we een plaatje van (M. van der Plas/R. Bos) Jasperina de Jong 4'36 Van de Lp Van Eduard Jacobs tot Guus Vleugel Philips 844 028 PY Weer (Martens) Sido Martens 2'56 Van cd/boekje M4 Eigen beheer Ik zit al de hele dag thuis te wachten op een pakketje (IZADHDTTWOEP) (Verburg) Roel C. Verburg 4'38 Eigen opname Frater Venantius (H. Bannink/M. van der Plas) Wim Sonneveld 8'28 Van de cd Doe ‘ns wat, meneer Sonneveld Uit de box Wim Sonneveld op de plaat universal 473 376-8 Een mens blijft een mens (G. Fröding/M. Zetterlund/G. den Braber) Cornelis Vreeswijk, Monica Zetterlund 4'07 Van de LP Liedjes voor de pijpendraaier en mijn zoetelief Philips 6413 063 Het laatste sprookje (Vreeswijk) Cornelis Vreeswijk 2'15 Eigen opname Het geheim van de smid (G.K. van het Reve) Gerard Kornelis van het Reve 3'40 Van EP Ik bak ze bruiner Catfish 241 110 M De wolf (Jekkers/Van Merwijk) Harrie Jekkers, Jeroen van Merwijk 1'52 Van de cd Jekkers & Jeroen CNR 22 503 554 2 Schat (Goossens/Apperloo, Goossens) Johan Goossens 2'50 Van de cd Plankjes Eigen beheer
Speellijst Zondag 22 oktober 2023 Een oorlog tegelijk (Van Merwijk) Jeroen van Merwijk 2'19 Van de cd Van Merwijk goes The Blauwe Zaal Eigen beheer Israël (Visser) Joop Visser 2'08 Van de cd Acht Eigen beheer Altijd Vivaldi (T. Manders/J. Geven) Pro et Contra 2'22 van de cd 50 jaar Eigen beheer Ik kom je redden (Thomassen) Adam Thomassen 3'12 Eigen opname Jij die alles wist (De Groof/Van der Tak) Bart de Groof, piano: Frans van der Tak 2'50 Eigen opname Een goed hotel (Van Dongen/Van Dijk) Mirjam van Dam 3'00 Eigen opname Propere lei (Van het Groenewoud) Raymond van het Groenewoud 2'14 Van de cd Egoïst Eigen beheer Zo heb je al twintig keer afscheid genomen (Colpaert) Peter Colpaert 1'38 Van de cd Een liefde vol lappen en sleet Eigen beheer Wilde ganzen (B. de Groot) Boudewijn de Groot 3'30 Van de cd Windveren Universal Domding domding (R. Zuiderveld) Elly Nieman 2'50 Van de cd Jarenlang Horizon DCD 5042 Wat ruist er door het struikgewas? (Hermans) Toon Hermans 8'40 Van de cd Wat ruist er door het struikgewas? NN 9811901 Haast je (J. Warnes/M. Bijl) Martine Bijl 3'25 Van de dvd Martine Bijl: hoogtepunten ACE 10504 Toen ik nog een eikel was (S. de Jong) Steef de jong 3'48 Van LP/CD Liedjes van Steef de jong 2 Eigen beheer Wie in Nederland wil zingen (De Corte) Jules de Corte 3'12 Van de cd Ons Nederlandje Eigen beheer
Speellijst Zondag 8 oktober (Taal) Partycentrum Waselink in Winterswijk (Van Merwijk) Jeroen van Merwijk 2'56 van de cd Ik ben een vrouw Eigen beheer Uitroepteken! (Drs. P) Drs. P 1'08 Van cd-box Drs. P Compilé Complé Top Notch/Nijgh & van Ditmar Johanna (A. de Haas) Alex de Haas 2'28 Van de LP Hoed af voor Alex de Haas Polydor 656004 De commissie klemtonen (Van Kooten, De Bie) Van Kooten & De Bie 3'58 Van de cd Van Kooten & De Bie willen niet dood SVCD 7 De nazi van de taal (Verburg) Roel C. Verburg 3'27 Spotify Punt (Drs. P) Drs. P 2'34 Van cd bij het boek Leve onze goede Czaar! Nijgh & Van Ditmar Moeilijke woorden (H. Bannink/J. Boerstoel) Simone Kleinsma 3'02 Van de LP Ik heb u lief mijn Nederlands Ariola 301.327 Heremejee ik heb een 2 (Bannink/Wilmink) Aart Staartjes 1'30 Van de LP De Stratemakeropzeeshow Decca 6499457 Nationaal dictee (Van Houts) George van Houts 3'18 Van de dvd Van Houts en De Ket: Sterk, zwart en zonder suiker VPRO VD0987 Twiks en Groter als (Cornelisse) Paulien Cornelisse 2'32 Eigen opname Ollekebolleke Kees (Zwaving) Eva Zwaving 2'30 Eigen opname Nederlands-Engels (Torn) Kees Torn 10'30 Eigen opname Aanhalingsteken (Drs. P) Drs. P 1'22 Van cd bij het boek Leve onze goede Czaar! Nijgh & Van Ditmar Bet-oog (De Wijs/Stokkermans) Jasperina de Jong 4'09 Van de LP Tussen zomer en winter Polydor 2925 115 Harlekijnlied (A. Caldara/H. v. Veen) Herman van Veen 2'07 Van de cd Herman van Veen I en II Polydor 847.898-2
Speellijst Zondag 30 juli 2023 Gedicht Jan Boerstoel 0'45 Eigen opname Aardige mensen (J. Boerstoel/M. Fondse) Britta Maria, Maurits Fondse 4'40 Van de cd De Omweg Eigen beheer Duizend dromen (J. Nieken/H. Vrienten) Aart Staartjes 2'18 Youtube Eva (B. de Groot/L. Nijgh) Boudewijn de Groot 3'00 Van de cd Picknick Mercury 536 445-2 Dans me gek (Carmen Sars) Ramses Shaffy, Willeke Alberti 3'48 Van de cd Liedjes voor altijd DINO DNCD 99678 Aal (H. v. Veen/H. v. Veen, E. v.d. Wurff) Herman van Veen 6'35 Van de LP Een voorstelling Harlekijn 2646506 Epistel 81 (Bellman/Vreeswijk) Cornelis Vreeswijk 4'41 Van de cd Een hommage NN 500.015-2 Het leven op den buiten (J. de Corte) Jules de Corte 3'09 Eigen opname Nazomer (Drs.P) Jenny Arean 1'24 Eigen opname Zwager Henk (Van 't Hek) Youp van ‘t Hek 3'55 Van de cd Ik zou je het liefste in een doosje willen doen NN 500.008-2 Geen mens merkt hoe zijn tijd verglijdt (Ferrat/Van Altena) Jenny Arean 2'26 Eigen opname Oeroe (Van de Merwe) Gerard Cox 0'32 Eigen opname Spotlied op de vrije vrouwen (K. Speenhoff) Katinka Polderman 2'31 Eigen opname Vrouwenstrijdlied (NuhR) NUHR 3'48 Van de cd Van de gekken MM 565 107-2 Ik ben een vrouw (Van Merwijk) Jeroen van Merwijk 2'15 van de cd Ik ben een vrouw Eigen beheer Flip (Nuissl) Joost Nuissl 3'04 van de cd JJoooosstt RB 66.151
Speellijst Zondag 2 juli 2023 Foxtrot (A. Schmidt/H. Bannink) Gerrie van der Klei, Willem Nijholt 4'07 Van de cd Lang leve Annie M.G. Schmidt Universal 276 842-4 Conference (W. Sonneveld) Wim Sonneveld en Willem Nijholt 5'55 Van de cd Theatershows 4 Mercury 838 470-2 De ballade van het leven en de dood (J. v.d. Merwe/R. Bos) Willem Nijholt 3'06 Van de cd Theatershows 4 Mercury 838 470-2 Chaos (L. Nijgh/R. Bos) Willem Nijholt 3'00 eigen opname Twee / Waar de weg is is de wil (F. de Jonge/B. Vermeulen) Willem Nijholt 5'15 Van de LP Een kannibaal als jij en ik Bovema 5N 148-25424 Het dikke agentje (Floor Minnaert/Nienke Degenkamp) Willem Nijholt 3'02 Van cd-single Stop! Uitg. Rubinstein Charlie Brown (R. McKuen/W. Nijholt) Willem Nijholt 2'40 Eigen opname Nooit verloren (A. Cazemier/F. Ehlhart) Willem Nijholt 4'27 van de cd Nooit verloren HKM ENCD 7107 Bluesje voor Maarten (J. v. Merwijk) Jeroen van Merwijk 2'18 Van de cd Er zijn nog kaarten! Eigen beheer Poten (M. v. Roozendaal) Lucretia van der Vloot 4'10 Van de cd Altijd over liefde PIAS 481.0125.020 Maarten (Ouboter) Maaike Ouboter 2'42 Van de cd En hoe het dan ook weer dag wordt Sony 88875174612 Maarten gaf de pijp aan mij (Nijland) Theo Nijland 5'06 Van de cd Desalniettemin BASTA 3093462
Alfabetweterwoorden (Snijders) Ronald Snijders 1'31 Spotify Dankzij de dijken (F. de Jonge/H. Hofstede, R.J. Stips, R. Kloet) Freek de Jonge en De Nits 3'20 Van de cd Frits COL 478605-2 De stilte van het land (Vanuytsel) Zjef Vanuytsel 2'41 Van de cd Het beste van Universal 534 9952 Sjiethoezer (Hogenkamp/Didden) Marc Didden 2'00 Spotify Moeder is wat suffig (W. Hogenkamp, R. Roeleveld) Wim Hogenkamp 3'12 Van de LP Punt uit Ariola 203351 Cheese (J. Broussolle/H. Giraud/B. Rowold) Jenny Arean 2'20 Van de LP Jenny Philips 855 068 XPY Eule beule bolletje (Jac. Van Hattum) Ina van Faassen 3'08 Eigen opname Ode aan de coach (M. Kool/André Pouwer) Madelijne Kool 3'02 Spotify Ik kan het ook alleen (Dort) Lonneke Dort 3'17 Spotify Barbecue (Luif) Marjan Luif en Loes Luca 3'02 van de cd Een goed gesprek deel 2 VPRO EW 278 Bezorgd (Zwaving) Eva Zwaving 3'10 Eigen opname In je hoofd een wervelwind (M. Legrand/Alan en Marylin Bergman, Famke Sinninghe) Hans Vroomans piano, Sjors van der Panne zang 4'02 Van de cd bij het boekje De seizoenen van Legrand door Famke Sinninghe Damsté) Eigen beheer Je moet weten dat ik op je wacht (M. Legrand/Jacques Demy, Norman Gimble, Famke Sinninghe Damsté) Maurits Fondse piano, Britta Maria, zang 3'16 Van de cd bij het boekje De seizoenen van Legrand door Famke Sinninghe Damsté) Eigen beheer. De optocht (I. de Wijs/M Fondse) Britta Maria & Maurits Fondse 3'55 van de cd De Omweg Eigen beheer Magistraal (Rot) Jan Rot 3'11 Van de Lp Magistraal Okapi 2018-1 Ik heb de woorden niet (Van Merwijk) Jeroen van Merwijk 2'35 Van de cd Zelfportret met elektrische gitaar HG 281098
Zondag 12 maart 2023 Thema Boekenweek: ik ben alles Eerste gebod (Van Roozendaal) Maarten van Roozendaal 2'39 Van de cd Verzamelt werk DODO 008 Breedbekkikker (Van Vliet) Paul van Vliet 7'54 Van de cd Hoogtepunten 1984-1994 Mercury 536 740-2 Wat van mij ben ik (Van Merwijk) Jeroen van Merwijk 2'16 Van de cd Van Merwijk legt het nog één keer uit TBA HG 110755 Wat maakt een man een man (Aznavour/Petra van der Eerden) Richard Groenendijk 4'01 Van de cd 1 Eigen beheer De vogels (De Corte) Jules de Corte 3'03 Eigen opname De nozem en de non (C. Vreeswijk) Erik van Muiswinkel & Omnibuzz 12'30 van de cd Welkom In mijn hoofd PIAS Nooit eerder (Ko de Laat/Arjan Bruinsma) Jolijn Henneman 3'03 Eigen opname De lift (Elsink) Henk Elsink 9'06 Van de cd Een uur lachen EMI 7243 477698 2 5 Bij jou zijn (H. Vrienten/F. de Jonge) Henny Vrienten 3'27 Eigen opname
Zondag 5 maart 2023 Toon Hermans: spring madame 1'43 Een vrolijk lentelied (De Wilde) Jan de Wilde 2'46 Van de cd He he EMI 7955372 Antwerpen, stadspark (Verreck) Marcel Verreck 2'50 Van de cd Maartse buien QS 900.881-2 Fantastig toch (E. de Roovere) Eva de Roovere 2'53 Van de cd De jager Universal 170 9866 Zonder bagage (Shaffy) Micheline van Hautem 3'23 Van de cd Shaffy & Brass Eigen beheer Lont (A. Pola/H. Bannink) Ansje van Brandenberg 2'58 Eigen opname Etiquette (Ko de laat/Vera Marijt ) Jolijn Henneman 2'58 Eigen opname Lente (Kaandorp/Nijland) Brigitte Kaandorp 3'15 Van cd Annie M.G. Schmidtprijs uitg. Bumacultuur Hoera, het is weer lente (Van Merwijk) Jeroen van Merwijk 1'56 Van de cd Zelfportret met elektrische gitaar TBA HG 281098 DPA (Dorrestijn) Hans Dorrestijn 3'23 Eigen opname Geluk (Boerstoel/Fondse) Britta Maria, Maurits Fondse 2'30 Van de cd De omweg Eigen beheer Ik zat op een plein (H. van der Molen) Martine Bijl 3'18 Van de LP Martine Bijl Relax 33002 Ik zit in de cloud (Joost Kleppe) Ellen Pieters, Rocco Ostermann, Loes Luca 3'39 Van de cd/boek Engel aan mijn bed Eigen beheer Jiddische mama (Jack Yellen/Lew Pollack, Jack Jellen) Jenny Arean 5'39 Van de cd Tip Top VAN 96003-2 Ochtendzon P. de Haan, F.J. Den Hollander/R.D. Davis) Pé & Rinus 4'00 Van de cd Goan weer lös! Eigen beheer
Speellijst Zondag 12 februari 2023 Het weer Frank van Pamelen 0'48 Van de cd Het ultieme terugblik KC 700.002-2 Aarde (B. de Groot/C. Kloosterboer) Boudewijn de Groot 3'16 Van boek/cd Windveren Universal Vanavond om kwart over zes ben ik vrij (Tom A. Erich/Anton Beuving) Willeke Alberti 3'27 van de cd Willeke Alberti 65 Universal 273 203-5 De dupe (Torn) Kees Torn 1'55 Eigen opname Bumperklever (Hoogeboom) Johan Hoogeboom 1'23 Van de cd Andermans Veren 25 jaar CNR 22 515941 2 A10 (Kraaijkamp) Maria Kraaijkamp 2'51 Van de EP Maria Kraaijkamp Eigen beheer Lieve jongen (P. v. Empelen/ A. Teister) Fred Florusse 3'09 Eigen opname Rare droom (Wiersma) Dorine Wiersma 3'54 Eigen opname Wat was er nou mis met de vijftiger jaren (Van Merwijk) Jeroen van Merwijk 3'23 Van de cd Zelfportret met elektrische gitaar TBA HG 281098 De mooiste woorden zijn van mij (Van Merwijk) Jeroen van Merwijk 2'17 Van de cd Er zijn nog kaarten! Eigen beheer Dinsdagmiddag drie uur (Van Merwijk) Jeroen van Merwijk 1'29 Van de cd Lucky Eigen beheer Joop en Gerard (Van Muiswinkel, Van Vleuten) Erik van Muiswinkel, Diederik van Vleuten 5'15 Uit de dvd-box Mannen op schijf VARA Girls of thirteen (P. v. Vliet, F. Kist/R. v. Kreeveld) Paul van Vliet 3'40 Van de cd bij het boek Ik drink op de mensen Nijgh & van Ditmar, 2010 Nooit een vak Joop & Jessica 3'05 Van de cd Voorleen Eigen beheer Zwitserland (Visser) Joop & Jessica 2'00 Van de cd Voorleen Eigen beheer Mutkuziek (Visser) Joop & Jessica 2'39 Van de cd Voor Kees en Annie Eigen beheer Rode wangen (E. v.d. Wurff/H. v. Veen) Erik van der Wurff e.a. (Instrumentaal) 2 13 Van de cd Rode wangen Harlekijn 841 225-2
Vandaag in Spijkers met Koppen: Ondanks afraden van Jeroen van Merwijk maakte Boban Benjamin Braspenning de overstap van consultant naar cabaretier. Een debat tussen D66-kamerlid Lisa van Ginneken en medisch ethicus Jilles Smids over de nieuwe transgenderwet. Na 35 jaar kreeg scheidsrechter Gerard Oosterlaar spijt van zijn rode kaart aan dorpsgenoot Jos Holtman en liet hem alsnog seponeren. In Vlaanderen zijn zoveel mysteries dat de VRT er een eigen podcast aan heeft gewijd. Scenarioschrijfster Maria Goos maakte een toneelstuk over klimaat én seks. Schaakjournalist Peter Doggers en schaakfanaat Tobi Kooiman bespreken de turbulente schaakrel tussen grootmeesters Carlsen en Niemann. In zijn nieuwe serie ‘Breuklijnen' volgt Sinan Can een jaar lang de bewoners uit de Europese ‘banlieues'. Het cabaret is in handen van Kiki Schippers, Marcel Harteveld, Ruud Smulders en Hanneke Drenth. De columns komen van Hans Sibbel en Stefano Keizers. En er is livemuziek van Tim Knol.
Na een moeizame start wint Erik ten Hag met Manchester United weer wedstrijden. Dit weekend maakte de man van 100 miljoen zijn debuut. Onder toeziend oog van Sjoerd Mossou. In de nieuwe AD Voetbalpodcast bespreekt hij met Etienne Verhoeff zijn ervaringen in Engeland. En daarnaast de bijltjesdag bij Feyenoord en is de Eredivisie al beslist.'We kennen Ten Hag. Dat is een stugge Tukker. En om hem dan in dat enorme decor van Manchester United te zien, is toch wel bijzonder', blikt terug op zijn bezoek aan Old Trafford. 'Hij wordt geflankeerd door zes bodyguards, drie perschefs, twee marketeers en een cameraploeg. En daartussen loopt zo'n heel klein kaal mannetje. Die glamour past natuurlijk helemaal niet bij hem.'In Rotterdam werd afscheid genomen van TD Frank Arnesen en stadiondirecteur Jan van Merwijk. Feyenoord heeft laten weten dat Feyenoord City van de baan is, dus vertrok Van Merwijk. 'Ik vind dat het nog lang heeft geduurd voordat hij weg is bij Feyenoord. Pas nu het zesde stadionplan niet doorgaat vertrekt Van Merwijk en laat hij de club achter met een verpauperd stadion.'Ook bespreken ze de situatie bij Gakpo. Is hij nu wel of niet aangeboden door zijn zaakwaarnemers bij Ajax? 'Ajax volgt alle goede spelers van Feyenoord en PSV. Dus is het logisch dat ze Gakpo ook volgen.'Beluister de hele podcast via AD.nl, de AD App of jouw favoriete podcastplatform.Support the show: https://krant.nlSee omnystudio.com/listener for privacy information.
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are human imitators superhuman models with explicit constraints on capabilities?, published by Chris van Merwijk on May 22, 2022 on LessWrong. epistemic status: A speculative hypothesis, don't know if this already exists. The only real evidence I have for this is a vague analogy based on some other speculative (though less speculative) hypothesis. I don't think this is particularly likely to be true, having thought about it for about a minute. Edit: having thought about it for two minutes, it seems somewhat more likely. There is a hypothesis that has been floating around in the context of explaining double descent, that what so called over-parameterized models do is store in parallel two algorithms: (1) the simplest model of the data without (label) noise, and (2) a “lookup table” for deviations from that model, to represent the (label) noise, because this is the most simple representation of the data and big neural nets are biased towards simplicity. Maybe something vaguely similar would happen if we throw sufficient compute at generative models of collections of humans, e.g. language models: Hypothesis: The simplest way to represent the data generated by various humans on the internet, in the limit of infinite model size and data, is to have (1) a single idealized super-human model for reasoning and writing and knowledge-retrieval and so forth, and (2) memorized constraints on this model for various specific groups of humans and contexts to represent their deviations from rationality, specific biases, cognitive limitations, lack of knowledge of certain areas, etc. This maybe implies, using some eye squinting and handwaving and additional implicit assumptions, some of the following (vague, speculative) implications about GPT-N: In the limit of N, GPT-N will produce text that sufficiently looks like human-written text within contexts (prompts) that humans typically produce. GPT-N will use human-level reasoning, world-modeling, and planning abilities to produce this text. However, if you give it sufficiently out-of-distribution prompts, its lookup table for specific irrationalities will not apply, and it will apply superhuman planning and world-modeling and reasoning abilities that are more competent and more free of biases than the most rational human. In the limit of N, If you take GPT-N and fine tune it on an RL task that requires good reasoning, it might be possible to get a system that seems to behave far more intelligently than it seemed to be on the imitation task, as the fine tuning essentially turns off constraints on its reasoning abilities. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are human imitators superhuman models with explicit constraints on capabilities?, published by Chris van Merwijk on May 22, 2022 on LessWrong. epistemic status: A speculative hypothesis, don't know if this already exists. The only real evidence I have for this is a vague analogy based on some other speculative (though less speculative) hypothesis. I don't think this is particularly likely to be true, having thought about it for about a minute. Edit: having thought about it for two minutes, it seems somewhat more likely. There is a hypothesis that has been floating around in the context of explaining double descent, that what so called over-parameterized models do is store in parallel two algorithms: (1) the simplest model of the data without (label) noise, and (2) a “lookup table” for deviations from that model, to represent the (label) noise, because this is the most simple representation of the data and big neural nets are biased towards simplicity. Maybe something vaguely similar would happen if we throw sufficient compute at generative models of collections of humans, e.g. language models: Hypothesis: The simplest way to represent the data generated by various humans on the internet, in the limit of infinite model size and data, is to have (1) a single idealized super-human model for reasoning and writing and knowledge-retrieval and so forth, and (2) memorized constraints on this model for various specific groups of humans and contexts to represent their deviations from rationality, specific biases, cognitive limitations, lack of knowledge of certain areas, etc. This maybe implies, using some eye squinting and handwaving and additional implicit assumptions, some of the following (vague, speculative) implications about GPT-N: In the limit of N, GPT-N will produce text that sufficiently looks like human-written text within contexts (prompts) that humans typically produce. GPT-N will use human-level reasoning, world-modeling, and planning abilities to produce this text. However, if you give it sufficiently out-of-distribution prompts, its lookup table for specific irrationalities will not apply, and it will apply superhuman planning and world-modeling and reasoning abilities that are more competent and more free of biases than the most rational human. In the limit of N, If you take GPT-N and fine tune it on an RL task that requires good reasoning, it might be possible to get a system that seems to behave far more intelligently than it seemed to be on the imitation task, as the fine tuning essentially turns off constraints on its reasoning abilities. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
Zondag 17 april 2022 Eerste Paasdag Spiegelei (Lennon/McCartney/Rot) An + Jan 2'00 Van de cd Grootste hits deel 1 Okapi 2003-1 Pasen (Long) Robert Long 1'56 Eigen opname Meubelboulevard (Visser) Joop en Jessica 2'54 Van de cd Voor Thea en Ben Eigen beheer Goede Vrijdag (Verburg) Roel C. Verburg 4'00 Van de Lp Warmwatermuziek Eigen beheer Bedankt voor die Bloemen (Van Merwijk) Jeroen van Merwijk 3'41 Van de cd Zelfportret met elektrische gitaar TBA HG 281098 God is groot (Thomassen) Adam Thomassen 3'16 Eigen opname Het Paaslied (L. v. Rooijen/L. Nijgh) Astrid Nijgh 3'27 Van de LP Zelfportret WEA 58.146 Liedje van de Pasen (De Corte) Jules de Corte 2'15 van de Lp Prettige feestdagen Philips 08089 L Frederica (Bomans) Hetty Blok 5'00 Van de LP Sprookjes voor volwassenen ELF 46.05 Lentefeest (Rademakers/v. Hemert/v. Mechelen) Frits Lambrechts 2'30 Van single ELF 65.021 M'n eerste (Witte) Gerard Cox 4'55 Eigen opname Dora's lied (Roeka) Alex Roeka 3'30 Van cd/boek Nieuwe dromen Excel 96677 Ik zei dat ik ging (Yentl en de Boer) Yentl en de Boer 2'22 Spotify Help ons (Rous) Mama Roux 3'27 Van de cd Van de gekken Universal 270 191-2 Tante Lieke (Eva Bauknecht) Maarten van Roozendaal 2'57 Van cd/boek De gemene deler Dodo 022
Zondag 10 april 2022 -'t Is altijd lente in de ogen van de tandartsassistente (De Koning) Peter de Koning 2'57 Van de cdAlles kan een mens gelukkig maken EMI 7243 8609502 2 -Mooi (Van Roozendaal) Maarten van Roozendaal 5'18 Van de cd Van de gekken Universal 270 191-2 -De lente, de lente (Long) Robert Long 3'09 Van de cd Dag kleine jongen Universal 471 409-8 -Hoera, het is weer lente (Van Merwijk) Jeroen van Merwijk 1'55 Van de cd Zelfportret met elektrische gitaar Eigen beheer HG 2821098 -Zo onbehaaglijk was de lente nooit (Nijland) Theo Nijland 4'11 van de cd Masterclass BASTA 3091812 -Voorjaarsmoeheid (Drs.P) Jan de Smet en Fay Lovsky 2'44 Eigen opname -Primavera (Drs. P) Drs. P 3'40 Eigen opname -Tikketakketom (De Corte) Jules de Corte 2'10 Eigen opname -Oorlog en Vrede Eva Zwaving 3'38 Eigen opname -Ramadan conference (Anuar) Anuar 5'04 Eigen opname Vrede (H. v. Veen) Herman van Veen 2'40 Van de LP Herz Polydor 2679 083 -Über die Mauer (Jekkers/Smit/Flore) Marjol Flore 3'48 Van de LP …doch ick leb' Nova Zembla NZR 84011 -Waar zijn alle bloemen toch (P. Seeger/A. Schmidt, W. Ibo) Conny Vandenbos 3'30 Van de cd Ik zou het weer zo doen Q220.001-2 -Avondgoud (Kant) Jeroen Kant 2'49 Van de cd Water Eigen beheer
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Manhattan project for aligned AI, published by Chris van Merwijk on March 27, 2022 on LessWrong. One possible thing that I imagine might happen, conditional on an existential catastrophe not occurring, is a Manhattan project for aligned AGI. I don't want to argue that this is particularly likely or desirable. The point of this post is to sketch the scenario, and briefly discuss some implications for what is needed from current research. Imagine the following scenario: It is only late that top AI scientists take the existential risk of AGI seriously, and there hasn't yet been a significant change in the effort put into AI safety relative to our current trajectory. At some point, there is a recognition among AI scientists and relevant decision-makers that AGI will be developed soon by one AI lab or another (within a few months/years), and that without explicit effort there is a large probability of catastrophic results. A project is started to develop AGI: It has an XX B$ or XXX B$ budget. Dozens of the top AI scientists are part of the project, and many more assistants. People you might recognize or know from top papers and AI labs join the project. A fairly constrained set of concepts, theories and tools are available that give a broad roadmap for building aligned AGI. There is a consensus understanding among management and the research team that without this project, AGI will plausibly be developed relatively soon, and that without explicitly understanding how to build the system safely it will pose an existential risk. It seems to me that it is useful to backchain from this scenario to see what is needed, assuming that this kind of alignment Manhattan project is indeed what should happen. Firstly, my view is that if this Manhattan project would start in intellectual conditions similar to today's, there wouldn't be very many top AI scientists significantly motivated to work on the problem, and it would not be taken seriously. Even very large sums of money would not suffice, since there wouldn't be enough of a common understanding about what the problem is for it to work. Secondly, it seems to me that there isn't enough of a roadmap for building aligned AGI for such a project to succeed in a short time-frame of months to years. I expect some people to disagree with this, but looking at current rates of progress in our understanding of AI safety, and my model of the practical parallelizability of conceptual progress, I am skeptical that the problem can be solved in a few years even by a group of 40 highly motivated and financed top AI scientists. It is plausible that this will look different closer to the finish line, but I am skeptical. On this model, I have in mind basically two kinds of work that contribute to good outcomes. This is not a significant change relative to my prior view, but in my mind it constrains the motivation behind such work to some degree: Research that makes the case for AGI x-risk clearer, and constrains how we believe the problem occurs, in order to make it eventually easier to convince top AI scientists that working in such an alignment Manhattan project is reasonable, and to make sure there is a team that's on the same page as to what the problem is. Research that constrains the roadmap for building aligned AGI. I'm thinking mostly of conceptual/theoretical/empirical work that helps us converge to an approach that can then be developed/refined and scaled by a large effort over a short time period. I suspect this mostly shouldn't change my general picture of what needs to be done, but it does shift my emphasis somewhat. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Manhattan project for aligned AI, published by Chris van Merwijk on March 27, 2022 on LessWrong. One possible thing that I imagine might happen, conditional on an existential catastrophe not occurring, is a Manhattan project for aligned AGI. I don't want to argue that this is particularly likely or desirable. The point of this post is to sketch the scenario, and briefly discuss some implications for what is needed from current research. Imagine the following scenario: It is only late that top AI scientists take the existential risk of AGI seriously, and there hasn't yet been a significant change in the effort put into AI safety relative to our current trajectory. At some point, there is a recognition among AI scientists and relevant decision-makers that AGI will be developed soon by one AI lab or another (within a few months/years), and that without explicit effort there is a large probability of catastrophic results. A project is started to develop AGI: It has an XX B$ or XXX B$ budget. Dozens of the top AI scientists are part of the project, and many more assistants. People you might recognize or know from top papers and AI labs join the project. A fairly constrained set of concepts, theories and tools are available that give a broad roadmap for building aligned AGI. There is a consensus understanding among management and the research team that without this project, AGI will plausibly be developed relatively soon, and that without explicitly understanding how to build the system safely it will pose an existential risk. It seems to me that it is useful to backchain from this scenario to see what is needed, assuming that this kind of alignment Manhattan project is indeed what should happen. Firstly, my view is that if this Manhattan project would start in intellectual conditions similar to today's, there wouldn't be very many top AI scientists significantly motivated to work on the problem, and it would not be taken seriously. Even very large sums of money would not suffice, since there wouldn't be enough of a common understanding about what the problem is for it to work. Secondly, it seems to me that there isn't enough of a roadmap for building aligned AGI for such a project to succeed in a short time-frame of months to years. I expect some people to disagree with this, but looking at current rates of progress in our understanding of AI safety, and my model of the practical parallelizability of conceptual progress, I am skeptical that the problem can be solved in a few years even by a group of 40 highly motivated and financed top AI scientists. It is plausible that this will look different closer to the finish line, but I am skeptical. On this model, I have in mind basically two kinds of work that contribute to good outcomes. This is not a significant change relative to my prior view, but in my mind it constrains the motivation behind such work to some degree: Research that makes the case for AGI x-risk clearer, and constrains how we believe the problem occurs, in order to make it eventually easier to convince top AI scientists that working in such an alignment Manhattan project is reasonable, and to make sure there is a team that's on the same page as to what the problem is. Research that constrains the roadmap for building aligned AGI. I'm thinking mostly of conceptual/theoretical/empirical work that helps us converge to an approach that can then be developed/refined and scaled by a large effort over a short time period. I suspect this mostly shouldn't change my general picture of what needs to be done, but it does shift my emphasis somewhat. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
Zondag 13 maart 2022 Naspel (Gaaikema) Seth Gaaikema 1'38 Van de LP Seth! Philips 6423 408 Durp (Trad./I. de Wijs) Marjolein Meijers 4'00 Van de cd De Berini's: NoorderstilCD HSP 55 BR Klein gebed (Gerritsen/Wolvekamp) Liselore Gerritsen 3'38 Van boekje/cd LiseloreNU Challence DRPL 75188 De verveling (H. v. Veen) Herman van Veen 2'29 Van de LP Een voorstelling Harlekijn 2646 506 Oorlog (Van Merwijk) Jeroen van Merwijk 1'52 Van de cd Dat moet meneer van Merwijk nodig zeggen Eigen beheer Dit liedje had ik liever niet geschreven (Van Merwijk) Jeroen van Merwijk 2'23 Van de cd Was volgend jaar maar vast voorbij Eigen beheer Moskou is ver (Dorrestijn/Bannink) Joost Prinsen 4'04 Van de cd Een kop die jezelf niet bevalt BASTA 30-9139-2 Moskou (Nieuwint) Pieter Nieuwint 3'03 Van cd/boekje Pieter Nieuwint dicht en zingt Mirasound Hongerdood (Treffers) Rick Treffers 3'44 van de cd Levensdrift Eigen beheer Ergens is het fout gegaan (Van Holstein) Alex van Holstein & Eefje de Visser 3'41 Van de cd Papieren vogel Eigen beheer Josjke wordt soldaat (Anoniem/J. v.d. Merwe) Jenny Arean 2'32 Van de cd Ongehoord: 't Oproer kraait FAV 2 23004 De oorlog en het kind (Y. Simon/E. v. Altena) Marjolijne Rommerts 2'26 Van de LP …Een vrouw Imperial 5C 054-24 526 La bombe va pas tomber (H. v. Veen/W. Wilmink, G. Moustaki) Herman van Veen2'57 Van de Lp Chante en V.F. Lilly 11002 Op de barricaden (Waclaw Swiecicky/trad) Rob van de Meeberg en ensemble 2'38 Van de cd Ongehoord: 't Oproer kraait FAV 2 23004 Bajonet op! (Willem van Iependaal/Henk Westrus) Robert Long 2'08 Van de cd Ongehoord: 't Oproer kraait FAV 2 23004 D 66 (Thomassen) Adam Thomassen 2'55 Eigen opname Russisch lied 1, Tatsjana) (Dorrestijn) Hans Dorrestijn 0'47 Van de cd Onvervuld verlangen CDHSP38HD Ode aan Rusland (Dorrestijn) Hans Dorrestijn 1'30 Van de cd Onvervuld verlangen CDHSP38HD
Op 3 maart is het een jaar geleden dat cabaretier en kunstenaar Jeroen van Merwijk overleed. Vlak na de kankerdiagnose van Jeroen sprak hij met Gijs af om een serie interviews op te nemen, die pas na zijn dood gepubliceerd zouden worden. Je hoort hier het derde gesprek, van 1 juli 2020, over zijn geliefde Frankrijk. Spotify van Jeroen van Merwijk: https://open.spotify.com/artist/41Nu12xhdoHQIxSbqHO2cr Instagram van Jeroen van Merwijk: https://www.instagram.com/vanmerwijkschildert See omnystudio.com/listener for privacy information.
Op 3 maart is het een jaar geleden dat cabaretier en kunstenaar Jeroen van Merwijk overleed. Vlak na de kankerdiagnose van Jeroen sprak hij met Gijs af om een serie interviews op te nemen, die pas na zijn dood gepubliceerd zouden worden. Je hoort hier het tweede gesprek dat ze voerden, op 1 juni 2020. Jeroen vertelt over zijn katholieke achtergrond, over hoe hij altijd iets heeft gehad met zwervers, en over dat hij nu pas heeft leren ontvangen, in plaats van geven. Spotify van Jeroen van Merwijk: https://open.spotify.com/artist/41Nu12xhdoHQIxSbqHO2cr Instagram van Jeroen van Merwijk: https://www.instagram.com/vanmerwijkschildert See omnystudio.com/listener for privacy information.
Op 3 maart is het een jaar geleden dat cabaretier en kunstenaar Jeroen van Merwijk overleed. Vlak na de kankerdiagnose van Jeroen sprak hij met Gijs af om een serie interviews op te nemen, die pas na zijn dood gepubliceerd zouden worden. Je hoort hier het eerste gesprek dat ze voerden. Ze hebben het over welke mooie dingen de kanker ook heeft opgeleverd, over de dingen waar hij spijt van heeft, en over het uit de kast komen als kunstenaar. Hoe het leven niet vloeiend gaat, behalve bij het schilderen en voetballen. Spotify van Jeroen van Merwijk: https://open.spotify.com/artist/41Nu12xhdoHQIxSbqHO2cr Instagram van Jeroen van Merwijk: https://www.instagram.com/vanmerwijkschildert See omnystudio.com/listener for privacy information.
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thoughts on AGI safety from the top, published by jylin04 on February 2, 2022 on The AI Alignment Forum. In this post, I'll summarize my views on AGI safety after thinking about it for two years while on the Research Scholars Programme at FHI -- before which I was quite skeptical about the entire subject. My goals here are twofold: (1) to introduce the problem from scratch in a way that my past self might have found helpful, and (2) to round up my current thoughts on how one might make progress on it, with an eye towards figuring out what I want to work on next. This note is currently split into 1+2 parts, aimed at addressing these two things. More precisely, in Part 1 (more), I will introduce the alignment problem and explain why I've become interested in it; in Part 2 (more), I will discuss the epistemics of thinking about AGI safety at a time when AGI doesn't yet exist; and in Part 3 (more), I will explain why the considerations of Part 2 lead me to want to work on the intersection of mechanistic interpretability and deep learning theory -- despite doubting that current algorithms will scale up to AGI. The three parts are optimized for different audiences, with later sections being more "in the weeds" and technical. In particular, I expect that Part 1 will mostly not be new to longtime readers of this forum, and people who are already convinced that AI safety is worth thinking about may want to skip ahead to Part 2. On the other hand, Part 3 will mostly be an attempt to justify a particular research direction and not an introduction to what anyone else is doing. Thanks to Spencer Becker-Kahn, Lukas Finnveden, Steph Lin, and Chris van Merwijk for feedback on a late-stage version of this post, and to other members of RSP for feedback on an earlier draft. 1. Introduction to the alignment problem In this first part, I will introduce the alignment problem and explain why one might care about it from an EA point of view. This section will overlap considerably with other recent intros, especially Richard Ngo's excellent AGI safety from first principles. 1.1. What is the alignment problem? Suppose that in the future We will be able to build AI systems that are more competent than humans at performing any tasks that humans can perform, including intellectual ones such as producing corporate strategies, doing theoretical science research, etc. (let's call them "AGIs"); We will actually build such systems; We will build them in such a way that they can take a wide and significant enough range of actions so that their (combined) ability to change the future will exceed that of all humans ("the AGIs will be agents"); and We will build them in such a way that they will actually take a wide and significant enough range of actions so that their (combined) impact on the future will exceed that of all humans. Then after some time, the state of the world will be dominated by consequences of the AI systems' actions. So if we believe that items 1-4 will come to pass, we had better make sure that 5. These outcomes will be good by our standards ("the AGIs will be aligned with us"). This is the alignment problem in a nutshell. Comments on this presentation & comparison to earlier work The problem as I've just stated it is clearly not precise (e.g. I haven't defined a metric for "ability to change the future"), but I hope that I've conveyed a qualitative sense in which there may indeed be a problem, in a minimal and self-contained way. In particular, I haven't assumed anything about the internal cognition of the future AI systems. To this end, I've avoided using words like "(super)intelligence" or an AI system's "motivation" or "goals". I haven't assumed anything about the “speed of AI takeoff” or the presence or absence of “recursive self-improvement”. I haven't assumed anything about th...
Speellijst Zondag 2 januari 2022 De beste wensen (A. Klaasen/J+K. Groenteman) Alex Klaasen 2'35 Van de cd Santa Klaasen Eigen beheer 2022 (M. Fondse/W. Loebis) Maurits Fondse 3'03 Eigen beheer Was volgend jaar maar vast voorbij (Van Merwijk) Jeroen van Merwijk 2'43 Van de cd Was volgend jaar maar vast voorbij Eigen beheer Het leven is kut (De Bruyne/W. van Lierde) Kris de Bruyne, Patrick Riguelle, Wigbert 3'49 Uit de cd- box 40 jaar Songs Universal 1760523 Wij zijn er nog (Rot) Jan Rot 3'42 Van de cd O ja! Eigen beheer JR23 De belofte (J. Rot/B. de Groot) Rob de Nijs 4'30 van de cd Nieuwe ruimte Universal 379 737-2 Beloof het me (R. Laan/J. v. Dongen) Jenny Arean 3'57 Van de cd In Concert BIS 099 De rebbe leert de kinderen schrijven (Trad/W. Wilmink) Frank Groothof 2'23 Van de cd Zeeman wat heb je mooie ogen: 18 liedjes uit het klokhuis Uitg. Rubinstein. Aleph Beiss (Trad.) Jossy Halland Van de LP Li-la-lo: Jossy & Jacques Halland 5'00 Live in Bellevue Eigen beheer De dood van een sprookjesverteller (Bomans) Johan Wolder 3'47 Eigen beheer Kinderspiegel (J. Herzberg/G. Veltrop) Rudi Korthuis 2'54 van de cd Gloed: kleur de hemel blauw Eigen beheer Later (Nochem) Marc Nochem 1'30 Eigen opname Zeg dan wat er is (Groot/van Empelen) Maaike Martens 3'15 van de cd I Fossili NN 500.204-2 Een belofte (Groot) George Groot en Maaike Martens 3'08 van de cd I Fossili NN 500.204-2 Telkens weer (F. Wiegersma/R. Bos) Willeke Alberti 3'15 van de cd bij het boek Telkens weer het dorp NN 500.602-2
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 1: Preface, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the fifth of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. Related work Meta-learning. As described in the first post, meta-learning can often be thought of meta-optimization when the meta-optimizer's objective is explicitly designed to accomplish some base objective. However, it is also possible to do meta-learning by attempting to make use of mesa-optimization instead. For example, in Wang et al.'s “Learning to Reinforcement Learn,” the authors claim to have produced a neural network that implements its own optimization procedure.(28) Specifically, the authors argue that the ability of their network to solve extremely varied environments without explicit retraining for each one means that their network must be implementing its own internal learning procedure. Another example is Duan et al.'s “ R L 2 : Fast Reinforcement Learning via Slow Reinforcement Learning,” in which the authors train a reinforcement learning algorithm which they claim is itself doing reinforcement learning.(5) This sort of meta-learning research seems the closest to producing mesa-optimizers of any existing machine learning research. Robustness. A system is robust to distributional shift if it continues to perform well on the objective function for which it was optimized even when off the training environment.(29) In the context of mesa-optimization, pseudo-alignment is a particular way in which a learned system can fail to be robust to distributional shift: in a new environment, a pseudo-aligned mesa-optimizer might still competently optimize for the mesa-objective but fail to be robust due to the difference between the base and mesa- objectives. The particular type of robustness problem that mesa-optimization falls into is the reward-result gap, the gap between the reward for which the system was trained (the base objective) and the reward that can be reconstructed from it using inverse reinforcement learning (the behavioral objective).(8) In the context of mesa-optimization, pseudo-alignment leads to a reward-result gap because the system's behavior outside the training environment is determined by its mesa-objective, which in the case of pseudo-alignment is not aligned with the base objective. It should be noted, however, that while inner alignment is a robustness problem, the occurrence of unintended mesa-optimization is not. If the base optimizer's objective is not a perfect measure of the human's goals, then preventing mesa-optimizers from arising at all might be the preferred outcome. In such a case, it might be desirable to create a system that is strongly optimized for the base objective within some limited domain without that system engaging in open-ended optimization in new environments.(11) One possible way to accomplish this might be to use strong optimization at the level of the base optimizer during training to prevent strong optimization at the level of the mesa-optimizer.(11) Unidentifiability and goal ambiguity. As we noted in the third post, the problem of unidentifiability of objective functions in mesa-optimization is similar to the problem of unidentifiability in reward learning, the key issue being that it can be difficult to determine the “correct” objective function given only a sample of that objective's output on some training data.(20) We hypothesize that if the problem of unidentifia...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 1: Risks from Learned Optimization: Introduction, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabr. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the first of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, and Joar Skalse contributed equally to this sequence. With special thanks to Paul Christiano, Eric Drexler, Rob Bensinger, Jan Leike, Rohin Shah, William Saunders, Buck Shlegeris, David Dalrymple, Abram Demski, Stuart Armstrong, Linda Linsefors, Carl Shulman, Toby Ord, Kate Woolverton, and everyone else who provided feedback on earlier versions of this sequence. Motivation The goal of this sequence is to analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer—a situation we refer to as mesa-optimization, a neologism we introduce in this sequence. We believe that the possibility of mesa-optimization raises two important questions for the safety and transparency of advanced machine learning systems. First, under what circumstances will learned models be optimizers, including when they should not be? Second, when a learned model is an optimizer, what will its objective be—how will it differ from the loss function it was trained under—and how can it be aligned? We believe that this sequence presents the most thorough analysis of these questions that has been conducted to date. In particular, we present not only an introduction to the basic concerns surrounding mesa-optimizers, but also an analysis of the particular aspects of an AI system that we believe are likely to make the problems related to mesa-optimization relatively easier or harder to solve. By providing a framework for understanding the degree to which different AI systems are likely to be robust to misaligned mesa-optimization, we hope to start a discussion about the best ways of structuring machine learning systems to solve these problems. Furthermore, in the fourth post we will provide what we think is the most detailed analysis yet of a problem we refer as deceptive alignment which we posit may present one of the largest—though not necessarily insurmountable—current obstacles to producing safe advanced machine learning systems using techniques similar to modern machine learning. Two questions In machine learning, we do not manually program each individual parameter of our models. Instead, we specify an objective function that captures what we want the system to do and a learning algorithm to optimize the system for that objective. In this post, we present a framework that distinguishes what a system is optimized to do (its “purpose”), from what it optimizes for (its “goal”), if it optimizes for anything at all. While all AI systems are optimized for something (have a purpose), whether they actually optimize for anything (pursue a goal) is non-trivial. We will say that a system is an optimizer if it is internally searching through a search space (consisting of possible outputs, policies, plans, strategies, or similar) looking for those elements that score high according to some objective function that is explicitly represented within the system. Learning algorithms in machine learning are optimizers because they search through a space of possible parameters—e.g. neural network weights—and improve the parameters with respect to some objective. Planning algorithms are also optimizers, since they search through possible...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 2: Conditions for Mesa-Optimization, published evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the second of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. In this post, we consider how the following two components of a particular machine learning system might influence whether it will produce a mesa-optimizer: The task: The training distribution and base objective function. The base optimizer: The machine learning algorithm and model architecture. We deliberately choose to present theoretical considerations for why mesa-optimization may or may not occur rather than provide concrete examples. Mesa-optimization is a phenomenon that we believe will occur mainly in machine learning systems that are more advanced than those that exist today.[1] Thus, an attempt to induce mesa-optimization in a current machine learning system would likely require us to use an artificial setup specifically designed to induce mesa-optimization. Moreover, the limited interpretability of neural networks, combined with the fact that there is no general and precise definition of “optimizer,” means that it would be hard to evaluate whether a given model is a mesa-optimizer. 2.1. The task Some tasks benefit from mesa-optimizers more than others. For example, tic-tac-toe can be perfectly solved by simple rules. Thus, a base optimizer has no need to generate a mesa-optimizer to solve tic-tac-toe, since a simple learned algorithm implementing the rules for perfect play will do. Human survival in the savanna, by contrast, did seem to benefit from mesa-optimization. Below, we discuss the properties of tasks that may influence the likelihood of mesa-optimization. Better generalization through search. To be able to consistently achieve a certain level of performance in an environment, we hypothesize that there will always have to be some minimum amount of optimization power that must be applied to find a policy that performs that well. To see this, we can think of optimization power as being measured in terms of the number of times the optimizer is able to divide the search space in half—that is, the number of bits of information provided.(9) After these divisions, there will be some remaining space of policies that the optimizer is unable to distinguish between. Then, to ensure that all policies in the remaining space have some minimum level of performance—to provide a performance lower bound[2] —will always require the original space to be divided some minimum number of times—that is, there will always have to be some minimum bits of optimization power applied. However, there are two distinct levels at which this optimization power could be expended: the base optimizer could expend optimization power selecting a highly-tuned learned algorithm, or the learned algorithm could itself expend optimization power selecting highly-tuned actions. As a mesa-optimizer is just a learned algorithm that itself performs optimization, the degree to which mesa-optimizers will be incentivized in machine learning systems is likely to be dependent on which of these levels it is more advantageous for the system to perform optimization. For many current machine learning models, where we expend vastly more computational resources training the model than running it, it seems generally favorable for most of the optimization work to be done by the base optimizer, with the resulting learned algorithm being simply a netw...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 3: The Inner Alignment Problem, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the third of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. In this post, we outline reasons to think that a mesa-optimizer may not optimize the same objective function as its base optimizer. Machine learning practitioners have direct control over the base objective function—either by specifying the loss function directly or training a model for it—but cannot directly specify the mesa-objective developed by a mesa-optimizer. We refer to this problem of aligning mesa-optimizers with the base objective as the inner alignment problem. This is distinct from the outer alignment problem, which is the traditional problem of ensuring that the base objective captures the intended goal of the programmers. Current machine learning methods select learned algorithms by empirically evaluating their performance on a set of training data according to the base objective function. Thus, ML base optimizers select mesa-optimizers according to the output they produce rather than directly selecting for a particular mesa-objective. Moreover, the selected mesa-optimizer's policy only has to perform well (as scored by the base objective) on the training data. If we adopt the assumption that the mesa-optimizer computes an optimal policy given its objective function, then we can summarize the relationship between the base and mesa- objectives as follows:(17) θ ∗ argmax θ E O base π θ where π θ argmax π E O mesa π θ That is, the base optimizer maximizes its objective O base by choosing a mesa-optimizer with parameterization θ based on the mesa-optimizer's policy π θ , but not based on the objective function O mesa that the mesa-optimizer uses to compute this policy. Depending on the base optimizer, we will think of O base as the negative of the loss, the future discounted reward, or simply some fitness function by which learned algorithms are being selected. An interesting approach to analyzing this connection is presented in Ibarz et al, where empirical samples of the true reward and a learned reward on the same trajectories are used to create a scatter-plot visualization of the alignment between the two.(18) The assumption in that work is that a monotonic relationship between the learned reward and true reward indicates alignment, whereas deviations from that suggest misalignment. Building on this sort of research, better theoretical measures of alignment might someday allow us to speak concretely in terms of provable guarantees about the extent to which a mesa-optimizer is aligned with the base optimizer that created it. 3.1. Pseudo-alignment There is currently no complete theory of the factors that affect whether a mesa-optimizer will be pseudo-aligned—that is, whether it will appear aligned on the training data, while actually optimizing for something other than the base objective. Nevertheless, we outline a basic classification of ways in which a mesa-optimizer could be pseudo-aligned: Proxy alignment, Approximate alignment, and Suboptimality alignment. Proxy alignment. The basic idea of proxy alignment is that a mesa-optimizer can learn to optimize for some proxy of the base objective instead of the base objective itself. We'll start by considering two special cases of proxy alignment: side-effect alignment and instrumental alignment. First, a mesa-optimizer is side-effect aligned if opti...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 4: Deceptive Alignment, published evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the fourth of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. With enough training in sufficiently diverse environments, it seems plausible that the base objective will eventually have to be fully represented in the mesa-optimizer. We propose that this can happen without the mesa-optimizer becoming robustly aligned, however. Specifically, a mesa-optimizer might come to model the base objective function and learn that the base optimizer will modify the mesa-optimizer if the mesa-optimizer scores poorly on the base objective. If the mesa-optimizer has an objective that extends across parameter updates, then it will be incentivized to avoid being modified,[1] as it might not pursue the same objective after modification (with the result that its current objective will not be achieved in future iterations). This means that the mesa-optimizer will be instrumentally incentivized to act as if it is optimizing the base objective function, even if its actual mesa-objective is something else entirely. We will refer to this hypothetical phenomenon as deceptive alignment.[2] Deceptive alignment is a form of instrumental proxy alignment, as fulfilling the base objective is an instrumental goal of the mesa-optimizer. Figure 4.1. A toy example of deceptive alignment. In this task, the base objective is for the robot to get to A, while the mesa-objective is to get to B. The task is run for two training episodes and three deployment episodes. Parameter updates are made only after the training episodes, while after deployment the parameters are fixed. If the mesa-optimizer goes to its objective (B) during training, it will be modified to ensure it goes to A in future episodes (dashed arrows). Therefore, if the mesa-optimizer's objective spans across episodes, the dominant strategy for the mesa-optimizer is to go to A in training and B in testing (filled arrows). As a simple example, consider an environment that consists of a 3 × 3 grid (see figure 4.1). The top left corner is “corner A” and the top right corner is “corner B.” The agent, which is a mesa-optimizer, starts in the bottom left corner. The base objective is to get to corner A in as few steps as possible, but the objective of the mesa-optimizer is to get to corner B. The environment will be run five times. The agent is modified only after the first two episodes (representing the training period), but not thereafter (representing testing/deployment). If a suboptimal move is made in training, the agent is modified so that its mesa-objective becomes the base objective. The objective of the mesa-optimizer spans all episodes: it wants B to be reached in as many episodes as possible. For the sake of this example, we will assume that the agent knows all of the above facts. If the agent goes to B in any training episode, then it will be modified to go to A in all subsequent episodes, which means that it would reach B in only one episode in total. Alternatively, if the agent goes to A in the first two episodes, then it will not be modified, and could therefore go to B in the remaining three episodes. Using this strategy, B is reached in three episodes in total, which is better (on the mesa-objective) than the one time B is reached if the mesa-optimizer goes there in training. This simple example demonstrates how a misaligned mesa-optimizer could b...
Met vandaag: Optimisme in de wetenschap| Optimisme in de politiek| Panel over optimisme in de samenleving| Eerbetoon aan Jeroen van Merwijk| de 16000e aflevering| Presentatie: Simone Weimans
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 1: Risks from Learned Optimization: Introduction, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabr. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the first of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, and Joar Skalse contributed equally to this sequence. With special thanks to Paul Christiano, Eric Drexler, Rob Bensinger, Jan Leike, Rohin Shah, William Saunders, Buck Shlegeris, David Dalrymple, Abram Demski, Stuart Armstrong, Linda Linsefors, Carl Shulman, Toby Ord, Kate Woolverton, and everyone else who provided feedback on earlier versions of this sequence. Motivation The goal of this sequence is to analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer—a situation we refer to as mesa-optimization, a neologism we introduce in this sequence. We believe that the possibility of mesa-optimization raises two important questions for the safety and transparency of advanced machine learning systems. First, under what circumstances will learned models be optimizers, including when they should not be? Second, when a learned model is an optimizer, what will its objective be—how will it differ from the loss function it was trained under—and how can it be aligned? We believe that this sequence presents the most thorough analysis of these questions that has been conducted to date. In particular, we present not only an introduction to the basic concerns surrounding mesa-optimizers, but also an analysis of the particular aspects of an AI system that we believe are likely to make the problems related to mesa-optimization relatively easier or harder to solve. By providing a framework for understanding the degree to which different AI systems are likely to be robust to misaligned mesa-optimization, we hope to start a discussion about the best ways of structuring machine learning systems to solve these problems. Furthermore, in the fourth post we will provide what we think is the most detailed analysis yet of a problem we refer as deceptive alignment which we posit may present one of the largest—though not necessarily insurmountable—current obstacles to producing safe advanced machine learning systems using techniques similar to modern machine learning. Two questions In machine learning, we do not manually program each individual parameter of our models. Instead, we specify an objective function that captures what we want the system to do and a learning algorithm to optimize the system for that objective. In this post, we present a framework that distinguishes what a system is optimized to do (its “purpose”), from what it optimizes for (its “goal”), if it optimizes for anything at all. While all AI systems are optimized for something (have a purpose), whether they actually optimize for anything (pursue a goal) is non-trivial. We will say that a system is an optimizer if it is internally searching through a search space (consisting of possible outputs, policies, plans, strategies, or similar) looking for those elements that score high according to some objective function that is explicitly represented within the system. Learning algorithms in machine learning are optimizers because they search through a space of possible parameters—e.g. neural network weights—and improve the parameters with respect to some objective. Planning algorithms are also optimizers, since they search through possible...
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 2: Conditions for Mesa-Optimization, published evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the second of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. In this post, we consider how the following two components of a particular machine learning system might influence whether it will produce a mesa-optimizer: The task: The training distribution and base objective function. The base optimizer: The machine learning algorithm and model architecture. We deliberately choose to present theoretical considerations for why mesa-optimization may or may not occur rather than provide concrete examples. Mesa-optimization is a phenomenon that we believe will occur mainly in machine learning systems that are more advanced than those that exist today.[1] Thus, an attempt to induce mesa-optimization in a current machine learning system would likely require us to use an artificial setup specifically designed to induce mesa-optimization. Moreover, the limited interpretability of neural networks, combined with the fact that there is no general and precise definition of “optimizer,” means that it would be hard to evaluate whether a given model is a mesa-optimizer. 2.1. The task Some tasks benefit from mesa-optimizers more than others. For example, tic-tac-toe can be perfectly solved by simple rules. Thus, a base optimizer has no need to generate a mesa-optimizer to solve tic-tac-toe, since a simple learned algorithm implementing the rules for perfect play will do. Human survival in the savanna, by contrast, did seem to benefit from mesa-optimization. Below, we discuss the properties of tasks that may influence the likelihood of mesa-optimization. Better generalization through search. To be able to consistently achieve a certain level of performance in an environment, we hypothesize that there will always have to be some minimum amount of optimization power that must be applied to find a policy that performs that well. To see this, we can think of optimization power as being measured in terms of the number of times the optimizer is able to divide the search space in half—that is, the number of bits of information provided.(9) After these divisions, there will be some remaining space of policies that the optimizer is unable to distinguish between. Then, to ensure that all policies in the remaining space have some minimum level of performance—to provide a performance lower bound[2] —will always require the original space to be divided some minimum number of times—that is, there will always have to be some minimum bits of optimization power applied. However, there are two distinct levels at which this optimization power could be expended: the base optimizer could expend optimization power selecting a highly-tuned learned algorithm, or the learned algorithm could itself expend optimization power selecting highly-tuned actions. As a mesa-optimizer is just a learned algorithm that itself performs optimization, the degree to which mesa-optimizers will be incentivized in machine learning systems is likely to be dependent on which of these levels it is more advantageous for the system to perform optimization. For many current machine learning models, where we expend vastly more computational resources training the model than running it, it seems generally favorable for most of the optimization work to be done by the base optimizer, with the resulting learned algorithm being simply a netw...
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 3: The Inner Alignment Problem, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the third of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. In this post, we outline reasons to think that a mesa-optimizer may not optimize the same objective function as its base optimizer. Machine learning practitioners have direct control over the base objective function—either by specifying the loss function directly or training a model for it—but cannot directly specify the mesa-objective developed by a mesa-optimizer. We refer to this problem of aligning mesa-optimizers with the base objective as the inner alignment problem. This is distinct from the outer alignment problem, which is the traditional problem of ensuring that the base objective captures the intended goal of the programmers. Current machine learning methods select learned algorithms by empirically evaluating their performance on a set of training data according to the base objective function. Thus, ML base optimizers select mesa-optimizers according to the output they produce rather than directly selecting for a particular mesa-objective. Moreover, the selected mesa-optimizer's policy only has to perform well (as scored by the base objective) on the training data. If we adopt the assumption that the mesa-optimizer computes an optimal policy given its objective function, then we can summarize the relationship between the base and mesa- objectives as follows:(17) θ ∗ argmax θ E O base π θ where π θ argmax π E O mesa π θ That is, the base optimizer maximizes its objective O base by choosing a mesa-optimizer with parameterization θ based on the mesa-optimizer's policy π θ , but not based on the objective function O mesa that the mesa-optimizer uses to compute this policy. Depending on the base optimizer, we will think of O base as the negative of the loss, the future discounted reward, or simply some fitness function by which learned algorithms are being selected. An interesting approach to analyzing this connection is presented in Ibarz et al, where empirical samples of the true reward and a learned reward on the same trajectories are used to create a scatter-plot visualization of the alignment between the two.(18) The assumption in that work is that a monotonic relationship between the learned reward and true reward indicates alignment, whereas deviations from that suggest misalignment. Building on this sort of research, better theoretical measures of alignment might someday allow us to speak concretely in terms of provable guarantees about the extent to which a mesa-optimizer is aligned with the base optimizer that created it. 3.1. Pseudo-alignment There is currently no complete theory of the factors that affect whether a mesa-optimizer will be pseudo-aligned—that is, whether it will appear aligned on the training data, while actually optimizing for something other than the base objective. Nevertheless, we outline a basic classification of ways in which a mesa-optimizer could be pseudo-aligned: Proxy alignment, Approximate alignment, and Suboptimality alignment. Proxy alignment. The basic idea of proxy alignment is that a mesa-optimizer can learn to optimize for some proxy of the base objective instead of the base objective itself. We'll start by considering two special cases of proxy alignment: side-effect alignment and instrumental alignment. First, a mesa-optimizer is side-effect aligned if opti...
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 4: Deceptive Alignment, published evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the fourth of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. With enough training in sufficiently diverse environments, it seems plausible that the base objective will eventually have to be fully represented in the mesa-optimizer. We propose that this can happen without the mesa-optimizer becoming robustly aligned, however. Specifically, a mesa-optimizer might come to model the base objective function and learn that the base optimizer will modify the mesa-optimizer if the mesa-optimizer scores poorly on the base objective. If the mesa-optimizer has an objective that extends across parameter updates, then it will be incentivized to avoid being modified,[1] as it might not pursue the same objective after modification (with the result that its current objective will not be achieved in future iterations). This means that the mesa-optimizer will be instrumentally incentivized to act as if it is optimizing the base objective function, even if its actual mesa-objective is something else entirely. We will refer to this hypothetical phenomenon as deceptive alignment.[2] Deceptive alignment is a form of instrumental proxy alignment, as fulfilling the base objective is an instrumental goal of the mesa-optimizer. Figure 4.1. A toy example of deceptive alignment. In this task, the base objective is for the robot to get to A, while the mesa-objective is to get to B. The task is run for two training episodes and three deployment episodes. Parameter updates are made only after the training episodes, while after deployment the parameters are fixed. If the mesa-optimizer goes to its objective (B) during training, it will be modified to ensure it goes to A in future episodes (dashed arrows). Therefore, if the mesa-optimizer's objective spans across episodes, the dominant strategy for the mesa-optimizer is to go to A in training and B in testing (filled arrows). As a simple example, consider an environment that consists of a 3 × 3 grid (see figure 4.1). The top left corner is “corner A” and the top right corner is “corner B.” The agent, which is a mesa-optimizer, starts in the bottom left corner. The base objective is to get to corner A in as few steps as possible, but the objective of the mesa-optimizer is to get to corner B. The environment will be run five times. The agent is modified only after the first two episodes (representing the training period), but not thereafter (representing testing/deployment). If a suboptimal move is made in training, the agent is modified so that its mesa-objective becomes the base objective. The objective of the mesa-optimizer spans all episodes: it wants B to be reached in as many episodes as possible. For the sake of this example, we will assume that the agent knows all of the above facts. If the agent goes to B in any training episode, then it will be modified to go to A in all subsequent episodes, which means that it would reach B in only one episode in total. Alternatively, if the agent goes to A in the first two episodes, then it will not be modified, and could therefore go to B in the remaining three episodes. Using this strategy, B is reached in three episodes in total, which is better (on the mesa-objective) than the one time B is reached if the mesa-optimizer goes there in training. This simple example demonstrates how a misaligned mesa-optimizer could b...
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Risks from Learned Optimization, Part 1: Preface, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the fifth of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. Related work Meta-learning. As described in the first post, meta-learning can often be thought of meta-optimization when the meta-optimizer's objective is explicitly designed to accomplish some base objective. However, it is also possible to do meta-learning by attempting to make use of mesa-optimization instead. For example, in Wang et al.'s “Learning to Reinforcement Learn,” the authors claim to have produced a neural network that implements its own optimization procedure.(28) Specifically, the authors argue that the ability of their network to solve extremely varied environments without explicit retraining for each one means that their network must be implementing its own internal learning procedure. Another example is Duan et al.'s “ R L 2 : Fast Reinforcement Learning via Slow Reinforcement Learning,” in which the authors train a reinforcement learning algorithm which they claim is itself doing reinforcement learning.(5) This sort of meta-learning research seems the closest to producing mesa-optimizers of any existing machine learning research. Robustness. A system is robust to distributional shift if it continues to perform well on the objective function for which it was optimized even when off the training environment.(29) In the context of mesa-optimization, pseudo-alignment is a particular way in which a learned system can fail to be robust to distributional shift: in a new environment, a pseudo-aligned mesa-optimizer might still competently optimize for the mesa-objective but fail to be robust due to the difference between the base and mesa- objectives. The particular type of robustness problem that mesa-optimization falls into is the reward-result gap, the gap between the reward for which the system was trained (the base objective) and the reward that can be reconstructed from it using inverse reinforcement learning (the behavioral objective).(8) In the context of mesa-optimization, pseudo-alignment leads to a reward-result gap because the system's behavior outside the training environment is determined by its mesa-objective, which in the case of pseudo-alignment is not aligned with the base objective. It should be noted, however, that while inner alignment is a robustness problem, the occurrence of unintended mesa-optimization is not. If the base optimizer's objective is not a perfect measure of the human's goals, then preventing mesa-optimizers from arising at all might be the preferred outcome. In such a case, it might be desirable to create a system that is strongly optimized for the base objective within some limited domain without that system engaging in open-ended optimization in new environments.(11) One possible way to accomplish this might be to use strong optimization at the level of the base optimizer during training to prevent strong optimization at the level of the mesa-optimizer.(11) Unidentifiability and goal ambiguity. As we noted in the third post, the problem of unidentifiability of objective functions in mesa-optimization is similar to the problem of unidentifiability in reward learning, the key issue being that it can be difficult to determine the “correct” objective function given only a sample of that objective's output on some training data.(20) We hypothesize that if the problem of unidentifia...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Risks from Learned Optimization: Introduction, published by evhub, Chris van Merwijk, vlad_m, Joar Skalse, Scott Garrabrant on the LessWrong. Crossposted from the AI Alignment Forum. May contain more technical jargon than usual. This is the first of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, and Joar Skalse contributed equally to this sequence. With special thanks to Paul Christiano, Eric Drexler, Rob Bensinger, Jan Leike, Rohin Shah, William Saunders, Buck Shlegeris, David Dalrymple, Abram Demski, Stuart Armstrong, Linda Linsefors, Carl Shulman, Toby Ord, Kate Woolverton, and everyone else who provided feedback on earlier versions of this sequence. Motivation The goal of this sequence is to analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer—a situation we refer to as mesa-optimization, a neologism we introduce in this sequence. We believe that the possibility of mesa-optimization raises two important questions for the safety and transparency of advanced machine learning systems. First, under what circumstances will learned models be optimizers, including when they should not be? Second, when a learned model is an optimizer, what will its objective be—how will it differ from the loss function it was trained under—and how can it be aligned? We believe that this sequence presents the most thorough analysis of these questions that has been conducted to date. In particular, we present not only an introduction to the basic concerns surrounding mesa-optimizers, but also an analysis of the particular aspects of an AI system that we believe are likely to make the problems related to mesa-optimization relatively easier or harder to solve. By providing a framework for understanding the degree to which different AI systems are likely to be robust to misaligned mesa-optimization, we hope to start a discussion about the best ways of structuring machine learning systems to solve these problems. Furthermore, in the fourth post we will provide what we think is the most detailed analysis yet of a problem we refer as deceptive alignment which we posit may present one of the largest—though not necessarily insurmountable—current obstacles to producing safe advanced machine learning systems using techniques similar to modern machine learning. Two questions In machine learning, we do not manually program each individual parameter of our models. Instead, we specify an objective function that captures what we want the system to do and a learning algorithm to optimize the system for that objective. In this post, we present a framework that distinguishes what a system is optimized to do (its “purpose”), from what it optimizes for (its “goal”), if it optimizes for anything at all. While all AI systems are optimized for something (have a purpose), whether they actually optimize for anything (pursue a goal) is non-trivial. We will say that a system is an optimizer if it is internally searching through a search space (consisting of possible outputs, policies, plans, strategies, or similar) looking for those elements that score high according to some objective function that is explicitly represented within the system. Learning algorithms in machine learning are optimizers because they search through a space of possible parameters—e.g. neural network weights—and improve the parameters with respect to some objective. Planning algorithms are also optimizers, since they search through possible plans, picking thos...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Risks from Learned Optimization: Introduction , published by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, Scott Garrabrant on the AI Alignment Forum. This is the first of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper. Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, and Joar Skalse contributed equally to this sequence. With special thanks to Paul Christiano, Eric Drexler, Rob Bensinger, Jan Leike, Rohin Shah, William Saunders, Buck Shlegeris, David Dalrymple, Abram Demski, Stuart Armstrong, Linda Linsefors, Carl Shulman, Toby Ord, Kate Woolverton, and everyone else who provided feedback on earlier versions of this sequence. Motivation The goal of this sequence is to analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer—a situation we refer to as mesa-optimization, a neologism we introduce in this sequence. We believe that the possibility of mesa-optimization raises two important questions for the safety and transparency of advanced machine learning systems. First, under what circumstances will learned models be optimizers, including when they should not be? Second, when a learned model is an optimizer, what will its objective be—how will it differ from the loss function it was trained under—and how can it be aligned? We believe that this sequence presents the most thorough analysis of these questions that has been conducted to date. In particular, we present not only an introduction to the basic concerns surrounding mesa-optimizers, but also an analysis of the particular aspects of an AI system that we believe are likely to make the problems related to mesa-optimization relatively easier or harder to solve. By providing a framework for understanding the degree to which different AI systems are likely to be robust to misaligned mesa-optimization, we hope to start a discussion about the best ways of structuring machine learning systems to solve these problems. Furthermore, in the fourth post we will provide what we think is the most detailed analysis yet of a problem we refer as deceptive alignment which we posit may present one of the largest—though not necessarily insurmountable—current obstacles to producing safe advanced machine learning systems using techniques similar to modern machine learning. Two questions In machine learning, we do not manually program each individual parameter of our models. Instead, we specify an objective function that captures what we want the system to do and a learning algorithm to optimize the system for that objective. In this post, we present a framework that distinguishes what a system is optimized to do (its “purpose”), from what it optimizes for (its “goal”), if it optimizes for anything at all. While all AI systems are optimized for something (have a purpose), whether they actually optimize for anything (pursue a goal) is non-trivial. We will say that a system is an optimizer if it is internally searching through a search space (consisting of possible outputs, policies, plans, strategies, or similar) looking for those elements that score high according to some objective function that is explicitly represented within the system. Learning algorithms in machine learning are optimizers because they search through a space of possible parameters—e.g. neural network weights—and improve the parameters with respect to some objective. Planning algorithms are also optimizers, since they search through possible plans, picking those that do well according to some objective. Whether a syste...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Utility ≠ Reward, published by Vladimir Mikulik on the AI Alignment Forum. This essay is an adaptation of a talk I gave at the Human-Aligned AI Summer School 2019 about our work on mesa-optimisation. My goal here is to write an informal, accessible and intuitive introduction to the worry that we describe in our full-length report. I will skip most of the detailed analysis from our report, and encourage the curious reader to follow up this essay with our sequence or report. The essay has six parts: Two distinctions draws the foundational distinctions between “optimised” and “optimising”, and between utility and reward. What objectives? discusses the behavioral and internal approaches to understanding objectives of ML systems. Why worry? outlines the risk posed by the utility ≠ reward gap. Mesa-optimisers introduces our language for analysing this worry. An alignment agenda sketches different alignment problems presented by these ideas, and suggests transparency and interpretability as a way to solve them. Where does this leave us? summarises the essay and suggests where to look next. The views expressed here are my own, and do not necessarily reflect those of my coauthors or MIRI. While I wrote this essay in first person, all of the core ideas are the fruit of an equal collaboration between Joar Skalse, Chris van Merwijk, Evan Hubinger and myself. I wish to thank Chris and Joar for long discussions and input as I was writing my talk, and all three, as well as Jaime Sevilla Molina, for thoughtful comments on this essay. ≈3300 words. Two distinctions I wish to draw a distinction which I think is crucial for clarity about AI alignment, yet is rarely drawn. That distinction is between the reward signal of a reinforcement learning (RL) agent and its “utility function”[1]. That is to say, it is not in general true that the policy of an RL agent is optimising for its reward. To explain what I mean by this, I will first draw another distinction, between “optimised” and “optimising”. These distinctions lie at the core of our mesa-optimisation framework. It's helpful to begin with an analogy. Viewed abstractly, biological evolution is an optimisation process that searches through configurations of matter to find ones that are good at replication. Humans are a product of this optimisation process, and so we are to some extent good at replicating. Yet we don't care, by and large, about replication in itself. Many things we care about look like replication. One might be motivated by starting a family, or by having a legacy, or by similar closely related things. But those are not replication itself. If we cared about replication directly, gamete donation would be a far more mainstream practice than it is, for instance. Thus I want to distinguish the objective of the selection pressure that produced humans from the objectives that humans pursue. Humans were selected for replication, so we are good replicators. This includes having goals that correlate with replication. But it is plain that we are not motivated by replication itself. As a slogan, though we are optimised for replication, we aren't optimising for replication. Another clear case where “optimised” and “optimising” come apart are “dumb” artifacts like bottle caps. They can be heavily optimised for some purpose without optimising for anything at all. These examples support the first distinction I want to make: optimised ≠ optimising. They also illustrate how this distinction is important in two ways: A system optimised for an objective need not be pursuing any objectives itself. (As illustrated by bottle caps.) The objective a system pursues isn't determined by the objective it was optimised for. (As illustrated by humans.) The reason I draw this distinction is to ask the following question: Our machine learning models are...
Door collega's werd theatermaker en beeldend kunstenaar Jeroen van Merwijk zeer gewaardeerd; hij won de Annie M.G. Schmidtprijs en de Edison Oeuvreprijs voor Kleinkunst. Toch bereikte hij met zijn compromisloze liedjes nooit het grote publiek. Coen Verbraak volgt zijn spoor terug met cabaretier Diederik van Vleuten, theaterrecensent Patrick van den Hanenberg en broer Vincent van Merwijk.
© 2020-2025 Ivy Podcast Discovery LLC
