POPULARITY
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI scaling myths, published by Nicholas Kruus on June 29, 2024 on The Effective Altruism Forum. "So far, bigger and bigger language models have proven more and more capable. But does the past predict the future? "One popular view is that we should expect the trends that have held so far to continue for many more orders of magnitude, and that it will potentially get us to artificial general intelligence, or AGI. "This view rests on a series of myths and misconceptions. The seeming predictability of scaling is a misunderstanding of what research has shown. Besides, there are signs that LLM developers are already at the limit of high-quality training data. And the industry is seeing strong downward pressure on model size. While we can't predict exactly how far AI will advance through scaling, we think there's virtually no chance that scaling alone will lead to AGI..." Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance, published by Nicholas Kruus on May 15, 2024 on The Effective Altruism Forum. Extended version of a short paper accepted at DPFM, ICLR'24. Authored by Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip H.S. Torr, Adel Bibi, Samuel Albanie, and Matthias Bethge. Similar to "The Importance of (Exponentially More) Computing Power." Abstract: Web-crawled pretraining datasets underlie the impressive "zero-shot" evaluation performance of multimodal models, such as CLIP for classification/retrieval and Stable-Diffusion for image generation. However, it is unclear how meaningful the notion of "zero-shot" generalization is for such multimodal models, as it is not known to what extent their pretraining datasets encompass the downstream concepts targeted for during "zero-shot" evaluation. In this work, we ask: How is the performance of multimodal models on downstream concepts influenced by the frequency of these concepts in their pretraining datasets? We comprehensively investigate this question across 34 models and five standard pretraining datasets (CC-3M, CC-12M, YFCC-15M, LAION-400M, LAION-Aesthetics), generating over 300GB of data artifacts. We consistently find that, far from exhibiting "zero-shot" generalization, multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance, following a sample inefficient log-linear scaling trend. This trend persists even when controlling for sample-level similarity between pretraining and downstream datasets, and testing on purely synthetic data distributions. Furthermore, upon benchmarking models on long-tailed data sampled based on our analysis, we demonstrate that multimodal models across the board perform poorly. We contribute this long-tail test set as the "Let it Wag!" benchmark to further research in this direction. Taken together, our study reveals an exponential need for training data which implies that the key to "zero-shot" generalization capabilities under large-scale training paradigms remains to be found. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Summary: Mistakes in the Moral Mathematics of Existential Risk (David Thorstad), published by Nicholas Kruus on April 10, 2024 on The Effective Altruism Forum. This post summarizes "Three Mistakes in the Moral Mathematics of Existential Risk," a Global Priorities Institute Working Paper by David Thorstad. This post is part of my sequence of GPI Working Paper summaries. For more, Thorstad's blog, Reflective Altruism, has a five-part series on this paper. Introduction Many prominent figures in the effective altruism community argue existential risk mitigation offers astronomical value. Thorstad believes there are many philosophical ways to push back on this conclusion[1] and even mathematical ones. Thorstad argues leading models of existential risk mitigation neglect morally relevant parameters, mislocating debates and inflating existential risk reduction's value by many orders of magnitude. He broadly assumes we aren't in the time of perils (which he justifies in this paper) and treats extinction risks as only those that kill all humans.[2] Mistake 1: Cumulative Risk Existential risks recur many times throughout history, meaning they can be presented as a per-century risk repeated each century or as a cumulative risk of occurring during a total time interval (e.g., the cumulative risk of extinction before Earth becomes less habitable). Mistake 1: Expected value calculations of existential risk mitigation reduce the cumulative risk, not the per-century risk. Thorstad identifies two problems with this choice. If humans live a long time, small reductions in cumulative risk require astronomical reductions in per-century risk. This is because the chance we survive for the total time interval in question depends on cumulative risk, and our cumulative survival chance must exceed our reduction in cumulative risk. Reducing cumulative risks with our actions today requires changing the risk for many, many centuries to come. So, even if we can substantially shift the risk of extinction this century or even nearby ones, we'll likely have a hard time doing so for existential risk a thousand or million centuries from now. For instance, if we want to create a meager one-in-a-hundred-million absolute reduction[3] in existential risk before Earth becomes less habitable,[4] the per-century risk must be nearly one-in-a-million or lower.[5] Many longtermists estimate this century's existential risk to be ~15-20% or higher,[6] in which case we'd need to drive the per-century risk down a hundred thousand times. Hence, many expected value calculations of existential risk mitigation demand vastly greater reductions in per-century risk than they initially seem to. Mistake 2: Background Risk Millett and Snyder-Beattie (MSB) offer one of the most cited papers discussing biorisk - biological extinction risk - featuring a favorable cost-effectiveness estimate. While Thorstad believes many complaints about MSB's model exist, he raises two. Mistake 2: Existential risk mitigation calculations (including MSB's model) ignore background risk. In MSB's model, the background risk is the risk of extinction from all non-biological sources. But, modifying this model to include background risk changes the estimated cost-effectiveness considerably. Without background risk, a 1% relative reduction in biorisk has a meaningful impact on per-century risk: it discounts per-century risk by 1%. But, when you include non-biological background risk, the same reduction in biorisk changes the per-century risk far less: per-century risk becomes the discounted biorisk plus the full background risk. Since many longtermists believe the per-century risk is very high (~15-20% or higher)[6] and thus much greater than biorisk, this substantially reduces biorisk mitigation's estimated cost-effectiveness: For reference, GiveWell e...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Summary: Against the Singularity Hypothesis (David Thorstad), published by Nicholas Kruus on March 27, 2024 on The Effective Altruism Forum. This post summarizes "Against the Singularity Hypothesis," a Global Priorities Institute Working Paper by David Thorstad. This post is part of my sequence of GPI Working Paper summaries. For more, Thorstad's blog, Reflective Altruism, has a three-part series on this paper. Introduction The effective altruism community has allocated substantial resources to catastrophic risks from AI, partly motivated by the singularity hypothesis about AI's rapid advancement. While many[1] AI experts and philosophers have defended the singularity hypothesis, Thorstad argues the case for it is surprisingly thin. Thorstad describes the singularity hypothesis in (roughly) the following three parts:[2] Self-Improvement: Artificial agents will become able to increase their own quantity of general intelligence. Intelligence Explosion: For a sustained period, their general intelligence will grow at an accelerating rate, creating exponential or hyperbolic growth that causes them to quickly surpass human intelligence by orders of magnitude. Singularity: This will produce a discontinuity in human history, after which humanity's fate - living in a digital form, extinct, or powerless - depends largely on our interactions with artificial agents. Growth Thorstad offers five reasons to doubt the intelligence growth rate proposed by the singularity hypothesis. Extraordinary claims require extraordinary evidence: Proposing that exponential or hyperbolic growth will occur for a prolonged period,[3] is an extraordinary claim that requires many excellent reasons to suspect it's correct. Until this high burden of evidence is met, it's appropriate to place very low credence on the singularity hypothesis. Good ideas become harder to find: Idea-generating becomes increasingly difficult as low-hanging fruit is picked. For example, spending on drug and agricultural research has seen rapidly diminishing returns.[4] AI will likely be no exception, as hardware improvement (e.g. Moore's law) is slowing. Even if the rate of diminishing research productivity is small, its effects become substantial as it compounds over many cycles of self-improvement.[5] Bottlenecks: No algorithm can run quicker than its slowest component, so, unless every component can be sped up at once, bottlenecks may arise. Even a single bottleneck would halt an intelligence explosion, and we should expect them to emerge because… There is limited room for improvement in certain processes (e.g., search algorithms) There are physical resource constraints (we shouldn't expect supply chains' output to increase a thousandfold or more very quickly) Physical constraints: Regardless of path, improving AI will eventually face intractable limitations from resource constraints and laws of physics, likely slowing intelligence growth. Consider Moore's law's demise: Circuits' energy requirements have massively increased - increasing costs and overheating.[6] Capital is drying up, as semiconductor plant prices have skyrocketed.[7] Our best transistors' diameter is now that of ten atoms, making manufacturing increasingly difficult and soon subject to quantum uncertainties.[8] Sublinearity: Technological capabilities[9] have been rapidly improving, meaning, if intelligence grows proportionally to them, then continuing current trends would create exponential intelligence growth. But intelligence grows sublinearly to these capabilities, not proportionally. Consider almost any performance metric plausibly correlated with intelligence - e.g., Chess, Go, protein folding, weather and oil reserve prediction - historically, exponential increases in the quantity of computation power yield merely linear gains.[10] If these performa...
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How much parenting harms productivity and how you can reduce it, published by Nicholas Kruus on February 26, 2024 on The Effective Altruism Forum. Intro Many EAs factor children's effects on their personal impact when deciding whether to have them ( example). To offer some insight for potential parents, I tried to summarize the best research I could find on parenthood's impact on people's productivity, though I was surprised at the lack of robust literature (especially more recently). The following information comes from four studies: one published in Science[1], one published in Nature[2], one from the Federal Reserve of St. Louis[3], and one published in Social Studies of Science[4]. They all focus on academics and quantitatively measure productivity with research output metrics (the footnotes contain more detail about each). How parenting impacts productivity TLDR: The trend for having one child seems to be a short-term reduction in productivity (median: 17%, mean: ~23%) for mothers that peters out after ~10 years. There is usually little effect on fathers, but fathers who are primary caregivers (or otherwise more engaged with their children) suffer similar short-term (12 years after having children): Parenting has no effect on productivity for mothers or fathers, so long as they have their children on purpose after they turn 30 years old. Social Studies of Science: Overall: 8% and 12% decline in research productivity and visibility[4], respectively, for men and women combined. For women, the decrease was 15%. To illustrate the cumulative effect of this, mothers were 2 years behind their childless counterparts in the number of papers they published 18 years after having their children. How to minimize productivity impacts Have kids later Economists who become mothers before 30 suffer a 13% decrease in overall (short- & long-term) productivity[3], whereas those having children after 30 do not ( Fed of St. Louis). Employment at an institution 100 ranks higher correlates with an additional 1-year delay before having children. However, this might be explained by personality: Perhaps, the type of people who wait to have children are the type of people who become employed at higher-ranked institutions ( Science). Take parental leave Taking parental leave shorter than 1 month does not mitigate productivity losses, but parental leave longer than 1 month and less than 12 months correlated with an 11%-17% productivity[2] improvement ( Nature). Be a lazier parent and divide labor bet...
Tere tulemast kuulama LIVE ülekannet aasta suurimalt Euroopa discgolfi sündmuselt! Olime oma stuudioga kohal suurvõistluse esimesel päeval, kus Sandra meelitas stuudiosse spordilegendid ehk otsesaates käisid meil külas:⇨ legendaarne spordikommentaator Kalev Kruus; ⇨ võistluse üks korraldajatest ja Eesti discgolfi ristiisa, Silver Leetma; ⇨ EM'i rajameister ja endine tippsportlane, Seppo Paju; ⇨ Soome valitsev meister, Niklas Anttila; ⇨ Eesti üks parimaid discgolfareid, Albert Tamm; ⇨ Eesti koondise korvpallur ja harrastusdiscgolfar, Kristjan Kitsing; ⇨ valitsev Rootsi meister ja EM meeste arvestuse võitja, Dennis Augustsson. Mõnusat kuulamist jagub nii discgolfi fännidele kui ka neile, kes spordialaga alles tutvust tegemas. Saadet juhib Sandra Raju.Liitu perega ➡︎ https://www.instagram.com/be1stpodcast/PS! Video oli otseeetris 16.08 Sportlandi Facebooki lehel. Loosimine on selleks korraks läbi.
Homme toimub Tallinna eri paigus koolinoorte teatripäev.
Kalev käis külas ja viiekümnes episood, juubelihurraa!
Tallinnas Adamson-Ericu muuseumis on avatud näitus "Valve Janov. Algusest avangardi 1942-1965". Vaatame näitust koos Enn Lillemetsaga, kes on üks näituse kuraatoritest. Teine kuraator on Ülle Kruus ning näituse kujundas Inga Heamägi.
The next phase of solar power technology is fast approaching. Our focus in today's podcast will be what lies ahead specifically in the field of photovoltaics embedded in existing infrastructure: sidewalks, driveways, trails, garage roofs and more. It's coming fast. --- Send in a voice message: https://anchor.fm/james-herlihy/message
27 Times Cinema at Venezia 78.Rebeka Kruus – 27 Times Cinema #Giornate2021 was first posted on September 11, 2021 at 2:44 pm.©2015 "Fred English Channel". Use of this feed is for personal non-commercial use only. If you are not reading this article in your feed reader, then the site is guilty of copyright infringement. Please contact me at radio@fred.fm
In this episode, our Digital Transformation Adviser Florian Marcus talks to Dermtest´s CEO Priit Kruus. Dermtest gives doctors the digital tools to manage the skin, skin problems, and wounds better. The company has ca 60 different hospitals, family doctors and private clinics in their client portfolio.
Coinbase went public today, making it easier for individual investors to transact in Cryptocurrencies like Bitcoin, Ethereum, Tether and Litecoin. Will Coinbase make Crypto more accessible like Apple and Microsoft made clunky home computers easy to use in the 1980s ? And why do you need a cryptocurrency account anyway ? Privacy, secrecy, one global currency ? Will internet based currencies supplant existing currencies ? It looks like money is about to be revolutionized by FinTech. --- Send in a voice message: https://anchor.fm/james-herlihy/message
Elina kohtub armastusest tehtud Jane Kruusiga, kellega tuleb juttu kaugetest ja lähedastest paikadest, laulmisest ja väelauludest, sooja põgenemisest, armastuse leidmisest, sõprusest ja hoidmisest. Toeta meie podcasti valmimist ja saa ligipääs erisaadetele: patreon.com/armastusest Jälgi meid: Instagram: instagram.com/podcastarmastusestFacebook: facebook.com/podcastarmastusestTwitter: twitter.com/armastusest Kuula meie Spotify muusikanimekirja.
Eesti vokaalkunsti jäädvustajana tuntud Jüri Kruus saab 3.
Eesti vokaalkunsti jäädvustajana tuntud Jüri Kruus saab 3.
Laiemale üldsusele oli Georg Ots tuntud eelkõige operetirollide kaudu – operetiprintsina. Albumile "Unistuste prints" on selle koostaja, restaureerija ja produtsent Jüri Kruus koondanud populaarsemad laulud ja duetid operettidest. (2010)
Laiemale üldsusele oli Georg Ots tuntud eelkõige operetirollide kaudu – operetiprintsina. Albumile "Unistuste prints" on selle koostaja, restaureerija ja produtsent Jüri Kruus koondanud populaarsemad laulud ja duetid operettidest. (2010)
Kuigi laulusalm ütleb, et Karuks istus vangitornis, pole selles grammigi tõde. Asjaosaline ise rõhutab, et juba lapsena oli ta hea poiss ja palju muret temaga ei olnud. Niisiis on meil külas Tiit Karuks - ajakirjanik erus, nagu ta ise ütleb - ja temaga koos tütar Epp Kruus.
Kuigi laulusalm ütleb, et Karuks istus vangitornis, pole selles grammigi tõde. Asjaosaline ise rõhutab, et juba lapsena oli ta hea poiss ja palju muret temaga ei olnud. Niisiis on meil külas Tiit Karuks - ajakirjanik erus, nagu ta ise ütleb - ja temaga koos tütar Epp Kruus.
Kullamäe ja Kruus avaldavad delikaatse info by Delfi Tasku
Albumile "Unistuste prints" on selle koostaja, restaureerija ja produtsent Jüri Kruus koondanud populaarsemad laulud ja duetid operettidest Georg Otsa esituses. Operetiprintsina jõudis Georg Ots üleliidulisele kinoekraanile Lenfilmi vändatud menufilmis "Mister X".
Albumile "Unistuste prints" on selle koostaja, restaureerija ja produtsent Jüri Kruus koondanud populaarsemad laulud ja duetid operettidest Georg Otsa esituses. Operetiprintsina jõudis Georg Ots üleliidulisele kinoekraanile Lenfilmi vändatud menufilmis "Mister X".
Oma debüüdiga ohtralt vastukaja tekitanud ansambli The Sound liikmed räägivad ära, mida nad MATTIAS NAANI virtuaalsest rünnakust arvavad ning kelle ja kuidas muusikat peaks kirjutama.
Õhtulehe jalgpallisaadet "Kolmas poolaeg" külastas hinnatud spordikommentaator Kalev Kruus. Ajakirjanike Siim Kera ning Kaarel Tälliga jagus tal jututeemasid küllalt - Ragnar Klavan Meistrite liiga finaalis, Madridi Real, Viasat Sport Balticu telekanali hingeelu, kalastamine, triatlon ja palju muud. Otseloomulikult ei puudunud saatest Coolbeti panustamisrubriik.
Stuudios on notar Priidu Pärna. Mis probleemid on uutel korteriühistutel tekkinud?Adamson-Ericu muuseumi direktor Ülle Kruus räägib, mida saab muuseumis veel teha peale näituste vaatamise. Kavas on loengud ja kontserdid, teatrietendused ja töötoad, kus inimesed saavad väljapaistvate kunstnike juhendamisel ise kunstiga tegelda.Saame tuttavaks vanima vabariigi tasemel kaasa tegeva mälumänguri Peeter-Erik Kuboga, kes aasta algul tähistas 80. sünnipäeva. Mälumänge on ta korraldanud üle 50 aasta. Saame teada, kuidas mälumäng ajas muutunud on ja miks tema ikka selles kaasa lööb.
Stuudios on notar Priidu Pärna. Mis probleemid on uutel korteriühistutel tekkinud?Adamson-Ericu muuseumi direktor Ülle Kruus räägib, mida saab muuseumis veel teha peale näituste vaatamise. Kavas on loengud ja kontserdid, teatrietendused ja töötoad, kus inimesed saavad väljapaistvate kunstnike juhendamisel ise kunstiga tegelda.Saame tuttavaks vanima vabariigi tasemel kaasa tegeva mälumänguri Peeter-Erik Kuboga, kes aasta algul tähistas 80. sünnipäeva. Mälumänge on ta korraldanud üle 50 aasta. Saame teada, kuidas mälumäng ajas muutunud on ja miks tema ikka selles kaasa lööb.
Priit Kruus on oma tiimiga loonud tehnoloogia, mis muudab melanoomi avastamise palju tõhusamaks.
Ekspressi külalispeatoimetaja Siim Kallas sai täna lõpuks selguse, et Reformierakond toetab tema kandidatuuri Riigikogus, aga valimisvõitluse lõpuni siduvat toetust ei antud. Mida selline määramatus tähendab ja mis üleüldse toimus tänasel Reformierakonna koosolekul toimus, sellest tuligi Kallas Ekspressi raadiosaatesse rääkima. Muuhulgas annab Kallas aru, et kui palju maksab tema käekell. Nimelt on tema konkurendi Marina Kaljuranna käekott endiselt paljudel teemana keelel, kuid see omakorda pahandas naisõiguslasi, kelle arvates võiks meedia uurida ka meespoliitikute käekellade maksumusi. Selgus, et Kallase kell pole VÄGA kallis. Saate teises pooles on külas TV3e kõige tuntum spordikommentaator Kalev Kruus, kes räägib spordifänni keerulisest elust ja valikutest olümpia eel. Muuhulgas räägib Kruus, et kellelt ta medaleid loodab, mida ta arvab üha süvenemast dopingukollist ja milliseid ülekandeid ta kindlasti kõigile vaadata soovitab. Saate lõpus laulab USA lauljatar Mitski looga „Your Best American Girlfriend“ ja see on lihtsalt väga-väga hea lugu. Saadet juhtisid Dannar Leitmaa ja Otto Oliver Olgo.
Kajalood mõõdab täna õpetajat vanuses 30+, kes on asunud oma elupõliseidametikaaslasi rehabiliteerima. Teatrilaval! Mida tahab oma elus saavutada mees, kes on õpetajatöö kõrvalt asutanud noorte kirjandusajakirja Värske Rõhk, on kirjutanud põhikoolile kirjandusõpikuid ning teinud pere ülalpidamiseks transamehe, parketipanija ja raadioreporteri tööd? Õpetaja Priit Kruus (foto: Vallo Kruuse/Ekspress) on Kaja Kärneri saates 3. oktoobril kell 12.05.
Kajalood mõõdab täna õpetajat vanuses 30+, kes on asunud oma elupõliseidametikaaslasi rehabiliteerima. Teatrilaval! Mida tahab oma elus saavutada mees, kes on õpetajatöö kõrvalt asutanud noorte kirjandusajakirja Värske Rõhk, on kirjutanud põhikoolile kirjandusõpikuid ning teinud pere ülalpidamiseks transamehe, parketipanija ja raadioreporteri tööd? Õpetaja Priit Kruus (foto: Vallo Kruuse/Ekspress) on Kaja Kärneri saates 3. oktoobril kell 12.05.
Eesti Maaülikooli teadlased Anne Must ja Märt Kruus räägivad liigirikkast mardikarühmast jooksiklastest, keda peaks olema võimalik meelitada põllumeestele appi kahjuri- ja umbrohutõrjes. Must ja Kruus on koos Enno Meriveega kirjutanud jooksiklastest pika rohkete fotodega loo „Seemnesööjad“ ajakirja Eesti Loodus septembrinumbrisse. (Toomas Jüriado, Hendrik Relve.)