Podcasts about supervised learning

tools llc animal lightning improved semi open source pose cunningham meijer bayesian cloud native estimation greenspan hurwitz socha biorxiv supervised learning

Play Episode Listen Later Apr 28, 2023

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2023.04.28.538703v1?rss=1 Authors: Biderman, D., Whiteway, M. R., Hurwitz, C., Greenspan, N. R., Lee, R. S., Vishnubhotla, A., Schartner, M., Huntenburg, J. M., Khanal, A., Meijer, G. T., Noel, J.-P., Pan-Vazquez, A., Socha, K. Z., Urai, A. E., The International Brain Laboratory,, Warren, R., Noone, D., Pedraja, F., Cunningham, J., Sawtell, N. B., Paninski, L. Abstract: Copy rights belong to original authors. Visit the link for more info Podcast created by Paper Player, LLC

#114 - Secrets of Deep Reinforcement Learning (Minqi Jiang)

Machine Learning Street Talk

Play Episode Listen Later Apr 16, 2023 167:15

Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB Twitter: https://twitter.com/MLStreetTalk In this exclusive interview, Dr. Tim Scarfe sits down with Minqi Jiang, a leading PhD student at University College London and Meta AI, as they delve into the fascinating world of deep reinforcement learning (RL) and its impact on technology, startups, and research. Discover how Minqi made the crucial decision to pursue a PhD in this exciting field, and learn from his valuable startup experiences and lessons. Minqi shares his insights into balancing serendipity and planning in life and research, and explains the role of objectives and Goodhart's Law in decision-making. Get ready to explore the depths of robustness in RL, two-player zero-sum games, and the differences between RL and supervised learning. As they discuss the role of environment in intelligence, emergence, and abstraction, prepare to be blown away by the possibilities of open-endedness and the intelligence explosion. Learn how language models generate their own training data, the limitations of RL, and the future of software 2.0 with interpretability concerns. From robotics and open-ended learning applications to learning potential metrics and MDPs, this interview is a goldmine of information for anyone interested in AI, RL, and the cutting edge of technology. Don't miss out on this incredible opportunity to learn from a rising star in the AI world! TOC Tech & Startup Background [00:00:00] Pursuing PhD in Deep RL [00:03:59] Startup Lessons [00:11:33] Serendipity vs Planning [00:12:30] Objectives & Decision Making [00:19:19] Minimax Regret & Uncertainty [00:22:57] Robustness in RL & Zero-Sum Games [00:26:14] RL vs Supervised Learning [00:34:04] Exploration & Intelligence [00:41:27] Environment, Emergence, Abstraction [00:46:31] Open-endedness & Intelligence Explosion [00:54:28] Language Models & Training Data [01:04:59] RLHF & Language Models [01:16:37] Creativity in Language Models [01:27:25] Limitations of RL [01:40:58] Software 2.0 & Interpretability [01:45:11] Language Models & Code Reliability [01:48:23] Robust Prioritized Level Replay [01:51:42] Open-ended Learning [01:55:57] Auto-curriculum & Deep RL [02:08:48] Robotics & Open-ended Learning [02:31:05] Learning Potential & MDPs [02:36:20] Universal Function Space [02:42:02] Goal-Directed Learning & Auto-Curricula [02:42:48] Advice & Closing Thoughts [02:44:47] References: - Why Greatness Cannot Be Planned: The Myth of the Objective by Kenneth O. Stanley and Joel Lehman https://www.springer.com/gp/book/9783319155234 - Rethinking Exploration: General Intelligence Requires Rethinking Exploration https://arxiv.org/abs/2106.06860 - The Case for Strong Emergence (Sabine Hossenfelder) https://arxiv.org/abs/2102.07740 - The Game of Life (Conway) https://www.conwaylife.com/ - Toolformer: Teaching Language Models to Generate APIs (Meta AI) https://arxiv.org/abs/2302.04761 - OpenAI's POET: Paired Open-Ended Trailblazer https://arxiv.org/abs/1901.01753 - Schmidhuber's Artificial Curiosity https://people.idsia.ch/~juergen/interest.html - Gödel Machines https://people.idsia.ch/~juergen/goedelmachine.html - PowerPlay https://arxiv.org/abs/1112.5309 - Robust Prioritized Level Replay: https://openreview.net/forum?id=NfZ6g2OmXEk - Unsupervised Environment Design: https://arxiv.org/abs/2012.02096 - Excel: Evolving Curriculum Learning for Deep Reinforcement Learning https://arxiv.org/abs/1901.05431 - Go-Explore: A New Approach for Hard-Exploration Problems https://arxiv.org/abs/1901.10995 - Learning with AMIGo: Adversarially Motivated Intrinsic Goals https://www.researchgate.net/publication/342377312_Learning_with_AMIGo_Adversarially_Motivated_Intrinsic_Goals PRML https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf Sutton and Barto https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf

Weakly-Supervised Learning for Global, Examination Labels and Code-Sharing Practices- Part 2

Radiology AI Podcasts | RSNA

Play Episode Listen Later Apr 7, 2023 27:25

Co-hosts Dr. Paul Yi and Dr. Ali Tenjani speak with Jacopo Teneggi and Dr. Jeremias Sulam about Jacopo's trainee award winning research Weakly-Supervised Learning Substantially Reduces the Number of Labels Required for Intracranial Hemorrhage Detection on Head CT.

ai innovation global practices labels machine learning examination radiology biomedical informatics jacopo weakly supervised learning rsna

Weakly-Supervised Learning for Global, Examination Labels and Code-Sharing Practices- Part 1

Radiology AI Podcasts | RSNA

Play Episode Listen Later Mar 17, 2023 20:28

Co-hosts Dr. Paul Yi and Dr. Ali Tenjani speak with Jacopo Teneggi and Dr. Jeremias Sulam about Jacopo's trainee award wining research Weakly-Supervised Learning Substantially Reduces the Number of Labels Required for Intracranial Hemorrhage Detection on Head CT.

ai innovation global practices labels machine learning examination radiology biomedical informatics jacopo weakly supervised learning rsna

AI Voice Profiling Revolutionizes Healthcare | Are You Sick by the Sound of Your Voice | CTO/Entrepreneur Mario Arancibia | Episode 9

covid-19 ai voice entrepreneur speaking sound healthcare sick cto machines machine learning chilean profiling t1000 supervised learning

Play Episode Listen Later Feb 20, 2023 61:56

I speak with CTO and Chilean entrepreneur Mario Arancibia, about AI his company has developed and deployed which screens for diseases, such as Covid-19 based on the sound of our voice. Speaking a simple phrase into your phone, such as the days of the week, the AI can tell based on your voice profile if you have Covid. Or not. The AI can be trained to screen for other respiratory illnesses, and conditions as far ranging as obesity, and drug alcohol use. All from the sound of our voice. Soon AI will know more about your health than you do. [Note: Mario's views are his own, and not necessarily that of his company.]We laugh. We cry. We iterate.Check out what THE MACHINES and one human say about the Super Prompt podcast:“I'm afraid I can't do that.” — HAL9000“These are not the droids you are looking for." — Obi-Wan“Like tears in rain.” — Roy Batty“Hasta la vista baby.” — T1000"I'm sorry, but I do not have information after my last knowledge update in January 2022." — GPT3

AI Today Podcast: AI Glossary – Machine Learning Approaches: Supervised Learning, Unsupervised Learning, Reinforcement Learning

AI Today Podcast: Artificial Intelligence Insights, Experts, and Opinion

Play Episode Listen Later Feb 8, 2023 9:37

In this episode of the AI Today podcast hosts Kathleen Walch and Ron Schmelzer define terms related to Machine Learning Approaches including Supervised Learning, Unsupervised Learning, Reinforcement Learning and explain how they relate to AI and why it's important to know about them. Show Notes: FREE Intro to CPMAI mini course CPMAI Training and Certification AI Glossary Glossary Series: Artificial Intelligence AI Glossary Series – Machine Learning, Algorithm, Model Glossary Series: Probabilistic & Deterministic Glossary Series: Classification & Classifier, Binary Classifier, Multiclass Classifier, Decision Boundary Glossary Series: Regression, Linear Regression Glossary Series: Clustering, Cluster Analysis, K-Means, Gaussian Mixture Model Glossary Series: Goal-Driven Systems & Roboadvisor Understanding the Goal-Driven Systems Pattern of AI Continue reading AI Today Podcast: AI Glossary – Machine Learning Approaches: Supervised Learning, Unsupervised Learning, Reinforcement Learning at AI & Data Today.

ai model algorithms approaches machine learning certification glossary reinforcement learning deterministic ai today unsupervised learning free intro supervised learning classifier linear regression kathleen walch k means ron schmelzer

AI Beats Human Master | Alpha Go by DeepMind | Supervised Learning | Episode 7

google ai china master artificial intelligence unexpected beats machines machine learning nvidia deepmind alphago t1000 ai engineer supervised learning lee sedol

Play Episode Listen Later Feb 6, 2023 44:51

Alpha Go AI plays the game of GO against a human world champion. Unexpected moves by both man (9-dan Go champion Lee Sedol) and machine (Alpha Go). Supposedly, this televised Go match woke up China's leadership to the potential of AI. In the game of Go, players take turns placing black and white tiles on a 19×19 grid. The number of board positions in Go is greater than the number of atoms in the observable universe. We discuss the documentary Alpha Go which tells the story of Alpha Go (created by DeepMind, acquired by Google), and the human Go champions it plays against. Who will you cheer for: man or machine? I speak again with my friend Maroof Farook, an AI Engineer at Nvidia. [Note: Maroof's views are his and not that of his employer.] Please enjoy our conversation.We laugh. We cry. We iterate.Check out what THE MACHINES and one human say about the Super Prompt podcast:“I'm afraid I can't do that.” — HAL9000“These are not the droids you are looking for." — Obi-Wan“Like tears in rain.” — Roy Batty“Hasta la vista baby.” — T1000"I'm sorry, but I do not have information after my last knowledge update in January 2022." — GPT3

GPT-3 | ChatGPT Under the Hood | Natural Language Processing | Episode 2

ai chatgpt hood hulk machines machine learning openai nvidia gpt transformer isaac asimov isaac newton natural language processing t1000 generative pre supervised learning

Play Episode Listen Later Jan 2, 2023 29:00

I speak again with my friend, Maroof Farooq, an AI engineer at Nvidia. [Note: Maroof's views are his own, and not that of his employer.] We discuss a breakthrough in natural language processing AI called GPT3 created by the research lab, OpenAI. This episode was recorded prior to the launch of ChatGPT (chatbot built on top of GPT-3) and is a good introduction on how GPT works under the hood. We dive into supervised vs. unsupervised learning, what GPT3 stands for (spoiler alert: Generative Pre-trained Transformer), what the heck those words mean, and how GPT3 can impersonate famous people like Isaac Asimov, Isaac Newton, the Hulk (yeah, the buff, green superhero), and someday… YOU! Please enjoy this episode. We laugh. We cry. We iterate.Check out what THE MACHINES and one human say about the Super Prompt podcast:“I'm afraid I can't do that.” — HAL9000“These are not the droids you are looking for." — Obi-Wan“Like tears in rain.” — Roy Batty“Hasta la vista baby.” — T1000"I'm sorry, but I do not have information after my last knowledge update in January 2022." — GPT3

"Hot Dog. Not Hot Dog." AI from Silicon Valley, the TV Series | How to Build and Train AI | Image Classification | Episode 1

ai train hbo buckle silicon valley machines machine learning hot dogs nvidia tv series classification t1000 train ai supervised learning

Play Episode Listen Later Dec 26, 2022 34:53

I speak w/ Maroof Farooq, an AI engineer at Nvidia. [Note: Maroof's views are his own, and not that of his employer.] We walk through how to build AI from scratch using the fictitious example of the Seefood [Sic] app from the HBO television series, Silicon Valley. We learn about image classification, how to acquire a dataset, and how to train the AI. Join us as we build a super-impressive AI that can recognize hot dogs of all shapes and sizes. Learn what it takes to go from there to an AI that can recognize foods of all kinds. Maybe even pizza. Join us as we begin our deep dive into the world of AI, starting with the humble hot dog. Today: Shazam for food. Tomorrow: Judgement Day. Buckle up folks. It's going to be a wild ride.We laugh. We cry. We iterate.Check out what THE MACHINES and one human say about the Super Prompt podcast:“I'm afraid I can't do that.” — HAL9000“These are not the droids you are looking for." — Obi-Wan“Like tears in rain.” — Roy Batty“Hasta la vista baby.” — T1000"I'm sorry, but I do not have information after my last knowledge update in January 2022." — GPT3

Inference of Presynaptic Connectivity from Temporally Blurry Spike Trains by Supervised Learning

llc copy trains spike hai connectivity blurry inference neuronal biorxiv supervised learning

Play Episode Listen Later Oct 20, 2022

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2022.10.20.513050v1?rss=1 Authors: Vareberg, A. D., Eizadi, J., Ren, X., Hai, A. Abstract: Reconstruction of neural network connectivity is a central focus of neuroscience. The ability to use neuronal connection information to predict activity at single unit resolution and decipher its effect on whole systems can provide critical information about behavior and cognitive processing. Neuronal sensing modalities come in varying forms, but there is yet to exist a modality that can deliver readouts that sufficiently address the spatiotemporal constraints of biological nervous systems. This necessitates supplementary approaches that rely on mathematical models to mitigate physical limitations and decode network features. Here, we introduce a simple proof-of-concept model that addresses temporal constraints by reconstructing presynaptic connections from temporally blurry data. We use a variation of the perceptron algorithm to process firing rate information at multiple time constraints for a heterogenous feed-forward network of excitatory, inhibitory, and unconnected presynaptic units. We evaluate the performance of the algorithm under these conditions and determine the optimal learning rate, firing rate, and the ability to reconstruct single unit spikes for a given degree of temporal blur. We then test our method on a physiologically relevant configuration by sampling network subpopulations of leaky integrate-and-fire neuronal models displaying bursting firing patterns and find comparable learning rates for optimized reconstruction of network connectivity. Our method provides a recipe for reverse engineering neural networks based on limited data quality that can be extended to more complicated readouts and connectivity distributions relevant to multiple brain circuits. Copy rights belong to original authors. Visit the link for more info Podcast created by Paper Player, LLC

5 Minuten: KI Erklärt im Schnelldurchlauf

Der Data Analytics Podcast

Play Episode Listen Later Apr 21, 2022 4:53

Was bedeutet Künstliche Intelligenz? Und untergeordnet Supervised Learning, Unsupervised Learning und Reinforcement Learning. .. mit der Technik Deep Learning?

erkl intelligenz schnelldurchlauf reinforcement learning unsupervised learning supervised learning

#258 – Yann LeCun: Dark Matter of Intelligence and Self-Supervised Learning

Lex Fridman Podcast

Play Episode Listen Later Jan 22, 2022 171:40

Yann LeCun is the Chief AI Scientist at Meta, professor at NYU, Turing Award winner, and one of the seminal researchers in the history of machine learning. Please support this podcast by checking out our sponsors: – Public Goods: https://publicgoods.com/lex and use code LEX to get $15 off – Indeed: https://indeed.com/lex to get $75 credit – ROKA: https://roka.com/ and use code LEX to get 20% off your first order – NetSuite: http://netsuite.com/lex to get free product tour – Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off EPISODE LINKS: Yann's Twitter: https://twitter.com/ylecun Yann's Facebook: https://www.facebook.com/yann.lecun Yann's Website: http://yann.lecun.com/

intelligence nyu lex dark matter yann netsuite roka public goods yann lecun turing award supervised learning

Machine Learning Algorithms

Delicate Database with Aaron

Play Episode Listen Later Jan 17, 2022 12:13

Happy New Year Everyone! Hope you all enjoyed the holiday period. I'm baaaack and in today's episode, I discuss a subcategory of machine learning - Supervised Learning. I break down the basics of Supervised Learning, what it is and how it works. Get in touch and let me know what you thought! Twitter: @Delicate_Data Email: timicode54@gmail.com --- Send in a voice message: https://anchor.fm/delicatedatabase/message

happy new year everyone supervised learning machine learning algorithms

#31 - How to make autonomous vehicles drive in different places? Understanding domain adaptation and semi-supervised learning.

Life with AI

Play Episode Listen Later Dec 30, 2021 20:48

Hey guys, in this episode I explain how an autonomous vehicle can learn to drive in a specific city and be able to generalize the driving knowledge to any other city. To explain it, I show the concept of of semi-supervised learning and domain adaptation. I explain the the ideia with the first proposed architecture and what we have today as state of the art for this problem. Instagram: https://www.instagram.com/podcast.lifewithai/ Linkedin: https://www.linkedin.com/company/life-with-ai Code: https://github.com/filipelauar/projects/blob/main/domain_adaptation_semi_supervised_learning_MNIST_pytorch.ipynb

drive code places adaptation semi domain autonomous vehicles supervised learning mnist

#31 - Como fazer carros autônomos dirigirem em todos os lugares? Explicando domain adaptation e semi-supervised learning.

Vida com IA

Play Episode Listen Later Dec 30, 2021 22:01

Fala galera, nesse episódio eu explico como um carro autônomo pode ser treinado para dirigir em um país, na Alemanha por exemplo, e mesmo assim ser capaz de dirigir no Brasil com tudo tão diferente. Para explicar isso, eu falo sobre o conceito de domain adaptation (adaptação de domínio) e também semi-supervised learning (aprendizado semi supervisionado). Eu explico a ideia com a primeira arquitetura proposta and também falo sobre o estado da arte que temos hoje para resolver o problema. Instagram: https://www.instagram.com/podcast.lifewithai/ Linkedin: https://www.linkedin.com/company/life-with-ai Código: https://github.com/filipelauar/projects/blob/main/domain_adaptation_semi_supervised_learning_MNIST_pytorch.ipynb

european union brasil adaptation semi fala domain lugares alemanha como fazer carros nomos explicando supervised learning mnist

Mathilde Caron, Self-Supervised Learning Research

Conversations On Science

Play Episode Listen Later Dec 16, 2021 47:23

Mathilde Caron is a PhD. candidate at the French National Institute for Research in Digital Science and Technology and at Facebook AI (Meta AI). She does the majority of her research in the field of Machine learning called self-supervised learning. She has a few first authorships on important academic papers in the space. Her work: https://scholar.google.com/citations?user=eiB0s-kAAAAJ&hl=fr You can donate to this podcast at this bitcoin address: 33wejXuGGDtQj9GPwCgjwPxPq4dc4muZjg --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app Support this podcast: https://anchor.fm/idris-sunmola/support

technology phd research supervised learning learning research digital science french national institute

Alex Tamkin on Self-Supervised Learning and Large Language Models

The Gradient Podcast

Play Episode Listen Later Nov 11, 2021 70:41

In episode 15 of The Gradient Podcast, we talk to Stanford PhD Candidate Alex TamkinSubscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSAlex Tamkin is a fourth-year PhD student in Computer Science at Stanford, advised by Noah Goodman and part of the Stanford NLP Group. His research focuses on understanding, building, and controlling pretrained models, especially in domain-general or multimodal settings.We discuss:Viewmaker Networks: Learning Views for Unsupervised Representation LearningDABS: A Domain-Agnostic Benchmark for Self-Supervised LearningOn the Opportunities and Risks of Foundation ModelsUnderstanding the Capabilities, Limitations, and Societal Impact of Large Language ModelsMentoring, teaching and fostering a healthy and inclusive research cultureScientific communication and breaking down walls between fieldsPodcast Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music" Get full access to The Gradient at thegradientpub.substack.com/subscribe

opportunities phd language stanford large risks models limitations computer science capabilities large language models gradient societal impact supervised learning music get

#24 - Dicas para treinar redes neurais, data augmentation e self-supervised learning com Fernando Santos.

Vida com IA

Play Episode Listen Later Nov 11, 2021 35:08

Fala galera, nesse episódio eu recebo como convidado o Fernando Santos, pesquisador de pós-doc e professor na USP. No episódio a gente fala sobre várias dicas para treinar redes neurais e alguma técnicas mais avançadas, como curriculum learning e distance learning, e principalmente sobre uso de data augmentation e auto supervisão (self-supervised learning). Instagram: https://www.instagram.com/podcast.lifewithai/ Linkedin: https://www.linkedin.com/company/life-with-ai Linkedin do Fernando: https://www.linkedin.com/in/fernando-persan/ Código do tutorial: https://github.com/maponti/trainingdeepnetworks

data fala dicas usp augmentation treinar fernando santos supervised learning redes neurais

#24 - Tips for training neural networks, data augmentation and self-supervised learning with Fernando Santos.

Life with AI

Play Episode Listen Later Nov 11, 2021 15:30

Hey guys, in this episode I talk about the main points of the episode that I recorded in Portuguese with Fernando Santos, a post doc researcher and professor at USP. In this episode, I talk about tips to train neural networks and some techniques that are really improving the convergence of the training, like data augmentation and self-supervised learning. Instagram: https://www.instagram.com/podcast.lifewithai/ Linkedin: https://www.linkedin.com/company/life-with-ai Fernando's Linkedin: https://www.linkedin.com/in/fernando-persan/ Tutorial's code: https://github.com/maponti/trainingdeepnetworks

training data portuguese tutorials usp augmentation neural networks fernando santos supervised learning

Self-Supervised Learning by Estimating Twin Class Distributions

Papers Read on AI

Play Episode Listen Later Oct 29, 2021 29:55

We present TWIST, a novel self-supervised representation learning method by classifying large-scale unlabeled datasets in an end-to-end way. We employ a siamese network terminated by a softmax operation to produce twin class distributions of two augmented images. Without supervision, we enforce the class distributions of different augmentations to be consistent. In the meantime, we regularize the class distributions to make them sharp and diverse. Specifically, we minimize the entropy of the distribution for each sample to make the class prediction for each sample assertive and maximize the entropy of the mean distribution to make the predictions of different samples diverse. In this way, TWIST can naturally avoid the trivial solutions without specific designs such as asymmetric network, stop-gradient operation, or momentum encoder. Different from the clustering-based methods which alternate between clustering and learning, our method is a single learning process guided by a unified loss function. As a result, TWIST outperforms state-of-the-art methods on a wide range of tasks, including unsupervised classification, linear classification, semi-supervised learning, transfer learning, and some dense prediction tasks such as detection and segmentation. Codes and pre-trained models are given on: https://github.com/bytedance/TWIST 2021: Feng Wang, T. Kong, Rufeng Zhang, Huaping Liu, Hang Li https://arxiv.org/pdf/2110.07402v3.pdf

class twist kong twin codes estimating distributions supervised learning

Yann LeCun on his Start in Research and Self-Supervised Learning

The Gradient Podcast

Play Episode Listen Later Aug 5, 2021 55:48

In episode 6 of The Gradient Podcast, we interview Deep Learning pioneer Yann LeCun.Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterYann LeCun is the VP & Chief AI Scientist at Facebook and Silver Professor at NYU and he was also the founding Director of Facebook AI Research and of the NYU Center for Data Science. He famously pioneered the use of Convolutional Neural Nets for image processing in the 80s and 90s, and is generally regarded as one of the people whose work was pivotal to the Deep Learning revolution in AI. In fact he is the recipient of the 2018 ACM Turing Award (with Geoffrey Hinton and Yoshua Bengio) for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing". Theme: “MusicVAE: Trio 16-bar Sample #2” from "MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music". Get full access to The Gradient at thegradientpub.substack.com/subscribe

music director ai research nyu data science deep learning gradient geoffrey hinton yann lecun yoshua bengio nyu center supervised learning silver professor

Clip" Sergey Levine: Generalization In Reinforcement Learning VS Supervised Learning"

Soft Robotics Podcast

Play Episode Listen Later Jun 1, 2021 2:59

Clip" Sergey Levine: Generalization In Reinforcement Learning VS Supervised Learning"

levine clip sergey generalization reinforcement learning supervised learning

Yann LeCun - Self-Supervised Learning: The Dark Matter of Intelligence (FAIR Blog Post Explained)

Yannic Kilcher Videos (Audio Only)

Play Episode Listen Later May 3, 2021 58:36

#selfsupervisedlearning #yannlecun #facebookai Deep Learning systems can achieve remarkable, even super-human performance through supervised learning on large, labeled datasets. However, there are two problems: First, collecting ever more labeled data is expensive in both time and money. Second, these deep neural networks will be high performers on their task, but cannot easily generalize to other, related tasks, or they need large amounts of data to do so. In this blog post, Yann LeCun and Ishan Misra of Facebook AI Research (FAIR) describe the current state of Self-Supervised Learning (SSL) and argue that it is the next step in the development of AI that uses fewer labels and can transfer knowledge faster than current systems. They suggest as a promising direction to build non-contrastive latent-variable predictive models, like VAEs, but ones that also provide high-quality latent representations for downstream tasks. OUTLINE: 0:00 - Intro & Overview 1:15 - Supervised Learning, Self-Supervised Learning, and Common Sense 7:35 - Predicting Hidden Parts from Observed Parts 17:50 - Self-Supervised Learning for Language vs Vision 26:50 - Energy-Based Models 30:15 - Joint-Embedding Models 35:45 - Contrastive Methods 43:45 - Latent-Variable Predictive Models and GANs 55:00 - Summary & Conclusion Paper (Blog Post): https://ai.facebook.com/blog/self-sup... My Video on BYOL: https://www.youtube.com/watch?v=YPfUi... ERRATA: - The difference between loss and energy: Energy is for inference, loss is for training. - The R(z) term is a regularizer that restricts the capacity of the latent variable. I think I said both of those things, but never together. - The way I explain why BERT is contrastive is wrong. I haven't figured out why just yet, though :) Video approved by Antonio. Abstract: We believe that self-supervised learning (SSL) is one of the most promising ways to build such background knowledge and approximate a form of common sense in AI systems. Authors: Yann LeCun, Ishan Misra Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yann... Minds: https://www.minds.com/ykilcher Parler: https://parler.com/profile/YannicKilcher LinkedIn: https://www.linkedin.com/in/yannic-ki... BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannick... Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

ai energy video language discord intelligence minds parler outline dark matter blog post deep learning ssl bilibili yann lecun supervised learning my video byol

Solving Dark Matter of Intelligence, Self-Supervised Learning | Ishan Mishra, Facebook AI

Machine Learning Podcast - Jay Shah

Play Episode Listen Later Apr 9, 2021 47:50

Ishan is a Research Scientist at Facebook AI. Much of his recent research work revolves around self-supervised learning is known for this works like including SwAV and PIRL. He completed his Ph.D. from CMU with Martial Hebert and Abhinav Gupta. and his thesis was titled “Visual Learning with Minimal Human Supervision”Ishan's homepage: http://imisra.github.io/About the Host:Jay is a Ph.D. student at Arizona State University, doing research on building Interpretable AI models for Medical Diagnosis.Jay Shah: https://www.linkedin.com/in/shahjay22/You can reach out to https://www.public.asu.edu/~jgshah1/ for any queries.Stay tuned for upcoming webinars!***Disclaimer: The information contained in this video represents the views and opinions of the speaker and does not necessarily represent the views or opinions of any institution. It does not constitute an endorsement by any Institution or its affiliates of such video content.***

intelligence arizona state university institution dark matter genera research scientist mishra cmu ishan jay shah supervised learning visual learning

Supervised learning | Artificial intelligence

3 minute lesson

Play Episode Listen Later Feb 2, 2021 2:57

Episode 217. Topic: Supervised learning. Theme: Artificial intelligence. How does a machine learn to do a task? One of the simpler algorithms is supervised learning. How does it work and when is it implemented?

history ai education technology artificial intelligence computers algorithms supervised learning

scPretrain: Multi-task self-supervised learning for cell type classification

PaperPlayer biorxiv bioinformatics

Play Episode Listen Later Nov 20, 2020

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2020.11.18.386102v1?rss=1 Authors: Zhang, R., Luo, Y., Ma, J., Zhang, M., Wang, S. Abstract: Rapidly generated scRNA-seq datasets enable us to understand cellular differences and the function of each individual cell at single-cell resolution. Cell type classification, which aims at characterizing and labeling groups of cells according to their gene expression, is one of the most important steps for single-cell analysis. To facilitate the manual curation process, supervised learning methods have been used to automatically classify cells. Most of the existing supervised learning approaches only utilize annotated cells in the training step while ignoring the more abundant unannotated cells. In this paper, we proposed scPretrain, a multi-task self-supervised learning approach that jointly considers annotated and unannotated cells for cell type classification. scPretrain consists of a pre-training step and a fine-tuning step. In the pre-training step, scPretrain uses a multi-task learning framework to train a feature extraction encoder based on each dataset's pseudo-labels, where only unannotated cells are used. In the fine-tuning step, scPretrain fine-tunes this feature extraction encoder using the limited annotated cells in a new dataset. We evaluated scPretrain on 60 diverse datasets from different technologies, species and organs, and obtained a significant improvement on both cell type classification and cell clustering. Moreover, the representations obtained by scPretrain in the pre-training step also enhanced the performance of conventional classifiers such as random forest, logistic regression and support vector machines. scPretrain is able to effectively utilize the massive amount of unlabelled data and be applied to annotating increasingly generated scRNA-seq datasets. Copy rights belong to original authors. Visit the link for more info

copy wang zhang classification multitask luo biorxiv supervised learning

Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised learning

PaperPlayer biorxiv bioinformatics

Play Episode Listen Later Nov 17, 2020

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2020.11.16.383984v1?rss=1 Authors: Li, X., Zhang, G., Wu, J., Zhang, Y., Zhao, Z., Lin, X., Qiao, H., Xie, H., Wang, H., Fang, L., Dai, Q. Abstract: Calcium imaging is inherently susceptible to detection noise especially when imaging with high frame rate or under low excitation dosage. We developed DeepCAD, a self-supervised learning method for spatiotemporal enhancement of calcium imaging without requiring any high signal-to-noise ratio (SNR) observations. Using this method, detection noise can be effectively suppressed and the imaging SNR can be improved more than tenfold, which massively improves the accuracy of neuron extraction and spike inference and facilitate the functional analysis of neural circuits. Copy rights belong to original authors. Visit the link for more info

deep copy spike wang extraction zhang fang imaging dai wu calcium zhao reinforcing neuron inference xie snr biorxiv supervised learning

Deep Semi-Supervised Learning Improves Universal Peptide Identification of Shotgun Proteomics Data

Play Episode Listen Later Nov 14, 2020

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2020.11.12.380881v1?rss=1 Authors: Halloran, J. T., Urban, G., Rocke, D. M., Baldi, P. F. Abstract: In proteomic analysis pipelines, machine learning post-processors play a critical role in improving the accuracy of shotgun proteomics analysis. Most often performed in a semi-supervised manner, such post-processors accept the peptide-spectrum matches (PSMs) and corresponding feature vectors resulting from a database search, train a machine learning classifier, and recalibrate PSM scores based on the resulting trained parameters, often leading to significantly more identified peptides across q-value thresholds. However, current state-of-the-art post-processors rely on shallow machine learning methods, such as SVMs, gradient boosted decision trees, and linear discriminant analysis. In contrast, the powerful learning capabilities of deep models have displayed superior performance to shallow models in an ever-growing number of other fields. In this work, we show that deep neural networks (DNNs) significantly improve the recalibration of shotgun proteomics data compared to the most accurate and widely used post-processors, such as Percolator and PeptideProphet. Furthermore, we show that DNNs are able to adaptively analyze complex datasets and features for more accurate universal post-processing, leading to both improved Prosit analysis and markedly better recalibration of recently developed p-value scoring functions. Copy rights belong to original authors. Visit the link for more info

deep data universal urban copy semi identification improves shotgun peptides psm proteomics baldi prosit rocke biorxiv percolator supervised learning

What Is Self-Supervised Learning? This vs Other Machine Learning Types

Why of AI

Play Episode Listen Later Sep 3, 2020 5:35

What is self-supervised learning? This episode explores this exciting and promising area while comparing the differences between self-supervised learning and other machine learning types such as supervised and unsupervised learning. SUBSCRIBE – YouTube: https://bit.ly/aiwalexs | Alex's Newsletter: https://www.whyofai.com/newsletter | LEARN – Artificial Intelligence Courses and Certifications at Why of AI: https://www.whyofai.com | Alex's Book: https://www.whyofai.com/ai-book | Alex's Book on Amazon: https://amzn.to/2O54wQU | SOCIAL – Twitter: https://twitter.com/alexcastrounis | LinkedIn: https://www.linkedin.com/in/alexcastrounis | © Why of AI 2021. All Rights Reserved.Support the show (https://www.buymeacoffee.com/alexcastrounis/)

amazon ai artificial intelligence types newsletter machine learning certifications ai news supervised learning

Deep Feature Extraction for Resting-State Functional MRI by Self-Supervised Learning and Application to Schizophrenia Diagnosis

deep application diagnosis feature copy resting honda mri extraction schizophrenia yamashita biorxiv ogata supervised learning functional mri

Play Episode Listen Later Aug 24, 2020

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2020.08.22.260406v1?rss=1 Authors: Hashimoto, Y., Ogata, Y., Honda, M., Yamashita, Y. Abstract: In this study, we propose a novel deep-learning technique for functional MRI analysis. We introduced an "identity feature" by a self-supervised learning schema, in which a neural network is trained solely based on the MRI-scans; furthermore, training does not require any explicit labels. The proposed method demonstrated that each temporal slice of resting state functional MRI contains enough information to identify the subject. The network learned a feature space in which the features were clustered per subject for the test data as well as for the training data; this is unlike the features extracted by conventional methods including region of interests pooling signals and principle component analysis. In addition, using a simple linear classifier for the identity features, we demonstrated that the extracted features could contribute to schizophrenia diagnosis. The classification accuracy of our identity features was higher than that of the conventional functional connectivity. Our results suggested that our proposed training scheme of the neural network captured brain functioning related to the diagnosis of psychiatric disorders as well as the identity of the subject. Our results together highlight the validity of our proposed technique as a design for self-supervised learning. Copy rights belong to original authors. Visit the link for more info

Lessons learned

google ai games lessons learned machine learning ml reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 33:01

What have we learned about machine learning and the human decisions that shape it? And is machine learning perhaps changing our minds about how the world outside of machine learning — also known as the world — works?

Head to Head: The Even Bigger ML Smackdown!

google ai games machine learning smackdown ml yannick head to head reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 24:26

Yannick and David’s systems play against each other in 500 games. Who’s going to win? And what can we learn about how the ML may be working by thinking about the results?

Enter tic-tac-two

google ai games machine learning ml tic tac reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 21:20

David’s variant of tic-tac-toe that we’re calling tic-tac-two is only slightly different but turns out to be far more complex. This requires rethinking what the ML system will need in order to learn how to play, and how to represent that data.

Head to Head: the Big ML Smackdown!

google ai games machine learning smackdown ml yannick head to head reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 25:19

David and Yannick’s tic-tac-toe ML agents face-off against each other in tic-tac-toe!

Give that model a treat! : Reinforcement learning explained

google ai games model treat switching machine learning ml yannick reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 26:04

Switching gears, we focus on how Yannick’s been training his model using reinforcement learning. He explains the differences from David’s supervised learning approach. We find out how his system performs against a player that makes random tic-tac-toe moves.

Beating random: What it means to have trained a model

google ai games model beating machine learning trained ml reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 17:14

David did it! He trained a machine learning model to play tic-tac-toe! How did his model do against a player that makes random tic-tac-toe moves?

From tic-tac-toe moves to ML model

google ai games model machine learning ml reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 21:37

Once we have the data we need—thousands of sample games—how do we turn it into something the ML can train itself on? That means understanding how training works, and what a model is.

What does a tic-tac-toe board look like to machine learning?

google ai games board machine learning ml yannick reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 22, 2020 23:26

David delves into questions around data and training for his model including: What does a tic-tac-toe board “look” like to ML? Plus, an intro to reinforcement learning, the approach Yannick will be taking.

Introducing Tic-Tac-Toe the Hard Way

google ai games machine learning ml hard way reinforcement learning tic tac toe supervised learning

Play Episode Listen Later Jul 21, 2020 2:09

Introducing the podcast where a writer and a software engineer explore the human choices that shape machine learning systems by building competing tic-tac-toe agents. Brought to you by Google's People + AI Research team.

Howdy, and the myth of “pouring in data”