Podcasts about iclr

learning energy intelligence new york university makes shortcuts functions deep learning generalization deep neural networks classifier natural language understanding iclr

Play Episode Listen Later Jul 26, 2022 89:04

This week we are joined by Kyunghyun Cho. He is an associate professor of computer science and data science at New York University, a research scientist at Facebook AI Research and a CIFAR Associate Fellow. On top of this he also co-chaired the recent ICLR 2020 virtual conference.We talk about a variety of topics in this weeks episode including the recent ICLR conference, energy functions, shortcut learning and the roles popularized Deep Learning research areas play in answering the question “What is Intelligence?”.Underrated ML Twitter: https://twitter.com/underrated_mlKyunghyun Cho Twitter: https://twitter.com/kchonyc?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5EauthorPlease let us know who you thought presented the most underrated paper in the form below:https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:“Shortcut Learning in Deep Neural Networks” - https://arxiv.org/pdf/2004.07780.pdf"Bayesian Deep Learning and a Probabilistic Perspective of Generalization” - https://arxiv.org/abs/2002.08791"Classifier-agnostic saliency map extraction" - https://arxiv.org/abs/1805.08249“Deep Energy Estimator Networks” - https://arxiv.org/abs/1805.08306“End-to-End Learning for Structured Prediction Energy Networks” - https://arxiv.org/abs/1703.05667“On approximating nabla f with neural networks” - https://arxiv.org/abs/1910.12744“Adversarial NLI: A New Benchmark for Natural Language Understanding“ - https://arxiv.org/abs/1910.14599“Learning the Difference that Makes a Difference with Counterfactually-Augmented Data” - https://arxiv.org/abs/1909.12434“Learning Concepts with Energy Functions” - https://openai.com/blog/learning-concepts-with-energy-functions/

Metaphor generation and ML for child welfare

Play Episode Listen Later Jul 26, 2022 73:38

We open season two of Underrated ML with Anna Huang on the show. Anna Huang is a Research Scientist at Google Brain, working on the Magenta project. Her research focuses on designing generative models to make creating music more approachable. She is the creator of Music Transformer and also the ML model Coconet that powered Google's first AI Doodle the Bach Doodle.She holds a PhD in computer science from Harvard University and was a recipient of the NSF Graduate Research Fellowship. She spent the later parts of her PhD as a visiting research student at the Montreal Institute of Learning Algorithms (MILA). She publishes in machine learning, human-computer interaction, and music, at conferences such as ICLR, IUI, CHI, and ISMIR.She has been a judge on the Eurovision AI Song Contest and her compositions have won awards including first place in the San Francisco Choral Artists' a cappella composition contest. She holds a masters in media arts and sciences from the MIT Media Lab, and a B.S. in computer science and B.M. in music composition both from the University of Southern California. She grew up in Hong Kong, where she learned to play the guzheng.On the episode we discuss Metaphoria by Kate Gero and Lydia Chilton, which is a fascinating tool allowing users to generate metaphors from only a select number of words. We also discuss the current trends regarding the dangers of AI with a case study on child welfare.Underrated ML Twitter: https://twitter.com/underrated_mlAnna Huang Twitter: https://twitter.com/huangczaPlease let us know who you thought presented the most underrated paper in the form below: https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:Gero, Katy Ilonka, and Lydia B. Chilton. "Metaphoria: An Algorithmic Companion for Metaphor Creation." CHI 2019. [paper][online paper] [talk] [demo]"A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions" - [paper]Additional Links:Compton, Kate, and Michael Mateas. "Casual Creators." ICCC 2015. [paper]Fiebrink, Rebecca, Dan Trueman, and Perry R. Cook. "A Meta-Instrument for Interactive, On-the-Fly Machine Learning." NIME 2009. [paper][talk][tool]Huang, Cheng-Zhi Anna, et al. "The Bach Doodle: Approachable music composition with machine learning at scale." ISMIR 2019. [paper][blog][doodle]

Interestingness predictions and getting to grips with data privacy

phd predictions barcelona rank princeton university electrical engineering data privacy grips naila nle generative models cvpr iclr

Play Episode Listen Later Jul 26, 2022 68:52

This week we are joined by Naila Murray. Naila obtained a B.Sc. in Electrical Engineering from Princeton University in 2007. In 2012, she received her PhD from the Universitat Autonoma de Barcelona, in affiliation with the Computer Vision Center. She joined NAVER LABS Europe (then Xerox Research Centre Europe) in January 2013, working on topics including fine-grained visual categorization, image retrieval, and visual attention. From 2015 to 2019 she led the computer vision team at NLE. She currently serves as NLE's director of science. She serves/served as area chair for ICLR 2018, ICCV 2019, ICLR 2019, CVPR 2020, ECCV 2020, and programme chair for ICLR 2021. Her research interests include representation learning and multi-modal search.We discuss using sparse pairwise comparisons to learn a ranking function that is robust to outliers. We also take a look at using generative models in order to utilise once inaccessible datasets.Underrated ML Twitter: https://twitter.com/underrated_mlNaila Murray Twitter: https://twitter.com/NailaMurrayPlease let us know who you thought presented the most underrated paper in the form below: https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:"Interestingness Prediction by Robust Learning to Rank" [paper]"Generative Models for Effective ML on Private Decentralized datasets" - [paper]

The ICLR's Relationship with the Insurance Industry

Zurich Canada's Perspectives

Play Episode Listen Later Jul 20, 2022 26:22

Listen to our latest Risk Insights episode led by Chris Snider, Zurich Resilience Solutions and guest Glenn McGillivray, Managing Director at the Institute for Catastrophic Loss Reduction. This episode covers the ICLR and their relationship with the personal and commercial insurance industry by highlighting topics such as wildfire floods and wind exposures.

relationships institute managing directors insurance industry iclr risk insights

Rosanne Liu: Paths in AI Research and ML Collective

The Gradient Podcast

Play Episode Listen Later Jun 10, 2022 75:08

In episode 29 of The Gradient Podcast, we chat with Rosanne Liu. Rosanne is a research scientist in Google Brain, and co-founder and executive director of ML Collective, a nonprofit organization for open collaboration and accessible mentorship. Before that she was a founding member of Uber AI. Outside of research, she supports underrepresented communities, and organizes symposiums, workshops, and a weekly reading group “Deep Learning: Classics and Trends” since 2018. She is currently thinking deeply how to democratize AI research even further, and improve the diversity and fairness of the field, while working on multiple fronts of machine learning research including understanding training dynamics, rethinking model capacity and scaling. Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterOutline:(01:30) How did you go into AI / research(6:45) AI research: the unreasonably narrow path and how not to be miserable(16:30) ML Collective Overview(21:45) Deep Learning: Classics and Trends Reading Group(26:25) More details about ML Collective(39:35) ICLR 2022 Diversity, Equity & Inclusion(48:00) Narrowness vs Variety in research(57:20) Favorite Papers (58:50) Measuring the Intrinsic Dimension of Objective Landscapes (01:01:40) Natural Adversarial Objects (01:03:00) Interests outside of AI - Writing(01:08:05) Interests outside of AI - Narrating Travels with Charley(01:13:22) Outro Get full access to The Gradient at thegradientpub.substack.com/subscribe

ai diversity collective measuring variety paths interests equity inclusion gradient ai research google brain ai writing iclr

Data Rights, Quantification and Governance for Ethical AI with Margaret Mitchell - #572

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later May 12, 2022 41:56

Today we close out our coverage of the ICLR series joined by Meg Mitchell, chief ethics scientist and researcher at Hugging Face. In our conversation with Meg, we discuss her participation in the WikiM3L Workshop, as well as her transition into her new role at Hugging Face, which has afforded her the ability to prioritize coding in her work around AI ethics. We explore her thoughts on the work happening in the fields of data curation and data governance, her interest in the inclusive sharing of datasets and creation of models that don't disproportionately underperform or exploit subpopulations, and how data collection practices have changed over the years. We also touch on changes to data protection laws happening in some pretty uncertain places, the evolution of her work on Model Cards, and how she's using this and recent Data Cards work to lower the barrier to entry to responsibly informed development of data and sharing of data. The complete show notes for this episode can be found at twimlai.com/go/572

ai ethical governance quantification margaret mitchell data rights iclr

Studying Machine Intelligence with Been Kim - #571

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later May 9, 2022 52:13

Today we continue our ICLR coverage joined by Been Kim, a staff research scientist at Google Brain, and an ICLR 2022 Invited Speaker. Been, whose research has historically been focused on interpretability in machine learning, delivered the keynote Beyond interpretability: developing a language to shape our relationships with AI, which explores the need to study AI machines as scientific objects, in isolation and with humans, which will provide principles for tools, but also is necessary to take our working relationship with AI to the next level. Before we dig into Been's talk, she characterizes where we are as an industry and community with interpretability, and what the current state of the art is for interpretability techniques. We explore how the Gestalt principles appear in neural networks, Been's choice to characterize communication with machines as a language as opposed to a set of principles or foundational understanding, and much much more. The complete show notes for this episode can be found at twimlai.com/go/571

ai studying gestalt google brain machine intelligence iclr

Advances in Neural Compression with Auke Wiggers - #570

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later May 2, 2022 37:09

Today we're joined by Auke Wiggers, an AI research scientist at Qualcomm. In our conversation with Auke, we discuss his team's recent research on data compression using generative models. We discuss the relationship between historical compression research and the current trend of neural compression, and the benefit of neural codecs, which learn to compress data from examples. We also explore the performance evaluation process and the recent developments that show that these models can operate in real-time on a mobile device. Finally, we discuss another ICLR paper, “Transformer-based transform coding”, that proposes a vision transformer-based architecture for image and video coding, and some of his team's other accepted works at the conference. The complete show notes for this episode can be found at twimlai.com/go/570

ai advances transformer qualcomm neural compression auke wiggers iclr

Optimizing Privacy with Synthetic Data with Alexandra Ebert

Data Bytes

Play Episode Listen Later Apr 28, 2022 38:21

Overview Today's guest is Alexandra Ebert, Chief Trust Officer at MOSTLY AI. Alexandra's work focused on public policy issues in the emerging field of synthetic data and Ethical AI. In this conversation we discuss the importance of communication skills for data professionals, what synthetic data is and how it can assist in privacy and fairness, and wrap up by discussing the importance of open data for research, education, and policy. This episode is great for data leaders looking to increase the privacy practice of their data teams, and provides practitioners and consumers with insights on the current state of Ethical AI. About Alexandra Ebert Alexandra Ebert is a Responsible AI, synthetic data & privacy expert and serves as Chief Trust Officer at MOSTLY AI. As a member of the company's executive leadership team, she is engaged in public policy issues in the emerging field of synthetic data and Ethical AI and is responsible for engaging with the privacy community, with regulators, the media, and with customers. She regularly speaks at international conferences on AI, privacy, and digital banking and hosts The Data Democratization Podcast, where she discusses emerging digital policy trends as well as Responsible AI and privacy best practices with regulators, policy experts and senior executives. Apart from her work at MOSTLY AI, she serves as the chair of the IEEE Synthetic Data IC expert group and was pleased to be invited to join the group of AI experts for the #humanAIze initiative, which aims to make AI more inclusive and accessible to everyone. Before joining the company, she researched GDPR's impact on the deployment of artificial intelligence in Europe and its economic, societal, and technological consequences. Besides being an advocate for privacy protection, Alexandra is deeply passionate about Ethical AI and ensuring the fair and responsible use of machine learning algorithms. She is the co-author of an ICLR paper and a popular blog series on fairness in AI and fair synthetic data, which was featured in Forbes, IEEE Spectrum, and by distinguished AI expert Andrew Ng. About MOSTLY AI: MOSTLY AI developed the world's most accurate synthetic data platform: a game-changing new anonymization technology that empowers businesses to unlock their big data assets, without putting their customers' privacy at risk. Social Handles Linkedin https://www.linkedin.com/in/alexandraebert/ Learn more about our mission and become a member here: https://www.womenindata.org/ --- Support this podcast: https://anchor.fm/women-in-data/support

europe ai data forbes privacy optimizing gdpr synthetic ebert responsible ai andrew ng ieee spectrum iclr alexandra ebert

Pytorch Geometric with Matthias Fey

Psyda Podcast with Minhaaj

Play Episode Listen Later Oct 9, 2021 91:46

Matthias Fey is the creator of the Pytorch Geometric library and a postdoctoral researcher in deep learning at TU Dortmund Germany. He is a core contributor to the Open Graph Benchmark dataset initiative in collaboration with Stanford University Professor Jure Leskovec. 00:00 Intro 00:50 Pytorch Geometric Inception 02:57 Graph NNs vs CNNs, Transformers, RNNs 05:00 Implementation of GNNs as an extension of other ANNs 08:15 Image Synthesis from Textual Inputs as GNNs 10:48 Image classification Implementations on augmented Data in GNNs 13:40 Multimodal Data implementation in GNNs 16:25 Computational complexity of GNN Models 18:55 GNNAuto Scale Paper, Big Data Scalability 24:39 Open Graph Benchmark Dataset Initiative with Stanford, Jure Leskovec and Large Networks 30:14 PyG in production, Biology, Chemistry and Fraud Detection 33:10 Solving Cold Start Problem in Recommender Systems using GNNs 38:21 German Football League, Bundesliga & Playing in Best team of Worst League 41:54 Pytorch Geometric in ICLR and NeurIPS and rise in GNN-based papers 43:27 Intrusion Detection, Anomaly Detection, and Social Network Monitoring as GNN implementation 46:10 Raw data conversion to Graph format as Input in PyG 50:00 Boilerplate templates for PyG for Citizen Data Scientists 53:37 GUI for beginners and Get Started Wizards 56:43 AutoML for PyG and timeline for Tensorflow Version 01:02:40 Explainability concerns in PyG and GNNs in general 01:04:40 CSV files in PyG and Structured Data Explainability 01:06:32 Playing Bass, Octoberfest & 99 Red Balloons 01:09:50 Collaboration with Stanford, OGB & Core Team 01:15:25 Leaderboards on Benchmark Datasets at OGB Website, Arvix Dataset 01:17:11 Datasets from outside Stanford, Harvard, Facebook etc 01:19:00 Kaggle vs Self-owned Competition Platform 01:20:00 Deploying Arvix Model for Recommendation of Papers 01:22:40 Future Directions of Research 01:26:00 Collaborations, Jurgen Schmidthuber & Combined Research 01:27:30 Sharing Office with a Dog, 2 Rabbits and How to train Cats

Graph Neural Networks with Ankit Jain

Psyda Podcast with Minhaaj

Play Episode Listen Later Sep 20, 2021 119:03

Ankit is an experienced AI Researcher/Machine Learning Engineer who is passionate about using AI to build scalable machine learning products. In his 10 years of AI career, he has researched and deployed several state-of-the-art machine learning models which have impacted 100s of millions of users. Currently, He works as a senior research scientist at Facebook where he works on a variety of machine learning problems across different verticals. Previously, he was a researcher at Uber AI where he worked on application of deep learning methods to different problems ranging from food delivery, fraud detection to self-driving cars. He has been a featured speaker in many of the top AI conferences and universities like UC Berkeley, IIT Bombay and has published papers in several top conferences like Neurips, ICLR. Additionally, he has co-authored a book on machine learning titled TensorFlow Machine Learning Projects. He has undergraduate and graduate degrees from IIT Bombay (India) and UC Berkeley respectively. Outside of work, he enjoys running and has run several marathons. 00:00 Intro 00:17 IIT vs FAANG companies, Competition Anxiety 05:40 Work Load between India and US, Educational Culture 07:50. Uber Eats, Food Recommendation Systems and Graph Networks 11:00 Accuracy Matrices for Recommendation Systems 12:42 Weather as a predictor of Food Orders and Pizza Fad 15:48 Raquel Urtusun and Zoubin Gharamani, Autonomous Driving and Google Brain 17:30 Graph Learning in Computer Vision & Beating the Benchmarks 19:15 Latent Space Representations and Fraud Detection 21:30 Multimodal Data & Prediction Accuracy 23:20 Multimodal Graph Recommendation at Uber Eats 23:50 Post-Order Data Analysis for Uber Eats 27:30 Plugging out of Matrix and Marathon Running 31:44 Finding Collusion between Riders and Drivers with Graph Learning 35:40 Reward Sensitivity Analysis for Drivers in Uber through LSTM Networks 42:00 PyG 2.0, Jure Leskovec, and DeepGraph, Tensorflow Support 46:46 Pytorch vs Tensorflow, Scalability and ease of use. 52:10 Work at Facebook, End to End Experiments 55:19 Optimisation of Cross-functional Solutions for Multiple Teams 57:30 Content Understanding teams and Behaviour Prediction 59:50 Cold Start Problem and Representation Mapping 01:03:30 NeurIPS paper on Meta-Learning and Global Few-Shot Model 01:07:00 Experimentation Ambience at Facebook, Privacy and Data Mine 01:09:03 Cons of working at FAANG 01:10:20 High School Math Teacher as Inspiration and Mentoring Others 01:18:25 TensorFlow Book and Upcoming Blog 01:16:40 Working at Oil Rig in the Ocean Straight Out of College 01:20:08 Promises of AI and Benefits to Society at Large 01:25:50 Facebook accused of Polarisation, Manipulation and Racism 01:28:10 Revenue Models - Product vs Advertising 01:31:15 Metaverse and Long-term Goals 01:33:10 Facebook Ray-Ban Stories and Market for Smart Glasses 01:36:40 Possibility of Facebook OS for Facebook Hardware 01:38:00 LibraCoin & Moving Fast - Breaking Things at Facebook 01:39:09 Orkut vs Facebook - A case study on Superior Tech Stack 01:42:00 Careers in Data Science & How to Get into It 01:45:00 Irrelevance of College Degrees and Prestigious Universities as Pre-requisites 01:49:50 Decreasing Attention Span & Lack of Curiosity 01:54:40 Arranged Marriages & Shifting Relationship Trends

#35 レビューのレビュー

nextstep.fm

Play Episode Listen Later Aug 13, 2021 132:17

Starring:k_katsumi, sonson_twit, d_date 1. @k_katsumi 遅刻 2. 二人でお送りする・・・・nextstep.fm'が・・・・． 3. github.dev試した？github.comでリポジトリを開いた状態でピリオドキーを押してみよう． 4. swiftのビルドは，死ぬほどリソースを食う 5. M1 Macのメモリが足りない 6. ブラウザでタブ開きすぎ問題 7. 色味・・・・ってわかる？ 8. ダークモード使ってますか？ 9. ワクチン副反応どうでしたか 10. 年齢と副反応 11. これからmRNAは，パッチアップデートように摂取する必要があるのだろうか 12. ワクチンは，一括パッチできる？ 13. 名前をなぜギリシア文字にしたのか 14. ワクチンのsemantic version - https://semver.org 15. 救急時のベッド探索問題 16. ベッドと医療機器はスケールアウトしても，使える人はスケールアウトできない． 17. 人はデータ，アルゴリズムによる統治を受け入れられるか？ 18. 信号機はすでにそうだよね． 19. @d_date 最近気づいたんですが・・・運用を考えない人って多くないですか？ 20. コードか？ドキュメントか？ 21. 両方いるってのは，当たり前なんだけど． 22. コードだけあってもさ，大人の事情は解決できない 23. slack, money forwardのビジネスモデル 24. そもそも銀行がAPIを個人に公開すればいいだけなんだけど 25. @k_katsumi https://swift-format.com Online swift code formatter 26. @sonson_twit Three.js - https://threejs.org 27. OpenGLを生で書くってもうないよね・・・・． 28. @sonson_twit JSのthisでつまづいた 29. ネットの情報源 30. なぞの対話形式のサイト・・・・・・ 31. Python, Ruby, JSのことを調べると・・・ひどいサイトにあたる 32. @d_date そういう技術まとめサイトにあたらない・・・・・． 33. 本屋いったら，プログラミングコーナーにPythonしかない・・・・・・・． 34. UIKit Tutorial https://developer.apple.com/tutorials/app-dev-training/ 35. @d_date 論文読み慣れてたら，Swift evolutionとか読みやすいのでは？ 36. @sonson_twit いや・・・やっぱり分野のコンテキストがわからないとつらいですよ． 37. 機械学習の学術分野は，急速に拡大しすぎて，すでに査読システムが限界 39. 学会のdouble blindレビューを揶揄したおもしろ動画　https://twitter.com/docmilanfar/status/1417650565941579777?s=20 40. 機械学習の分野のジャーナルの査読と，会議の査読の違い 41. ジャーナルのレビューはやりとりがあるけど・・・・会議のレビューは一発勝負に近い． 42. ICLRのオープンレビュー https://openreview.net/forum?id=HygnDhEtvr 43. インターネット時代の査読とは 44. 査読システムが取れる責任は小さい・・・・数学と自然科学は違う 45. @sonson_twit の失態 46. 結局カットせず 47. 不正使用よくない! 48. @k_katsumi え，週間でやるんですか

online swift api starring python mrna js opengl iclr

Evgeny Burnaev | Евгений Бурнаев

Play Episode Listen Later May 27, 2021 84:09

Evgeny Burnaev, Ph.D. Associate Professor of Center for Computational and Data-Intensive Science and Engineering (CDISE) at Skolkovo Institute of Science and Technology (Skoltech). Evgeny graduated from the Moscow Institute of Physics and Technology in 2006. After getting a Candidate of Sciences degree from the Institute for Information Transmission Problem in 2008, he stayed with the Institute as a head of the Data Analysis and Predictive Modeling Lab. Since 2007 Evgeny carried out a number of successful industrial projects with Airbus, SAFT, IHI, and Sahara Force India Formula 1 team among others. The corresponding data analysis algorithms, developed by Evgeny and his scientific group, formed a core of the algorithmic software library for metamodeling and optimization. Thanks to the developed functionality, engineers can construct fast mathematical approximations to long-running computer codes (realizing physical models) based on available data and perform design space exploration for trade-off studies. The software library passed the final Technology Readiness Level certification in Airbus. According to Airbus experts, the application of the library “provides the reduction of up to 10% of lead time and cost in several areas of the aircraft design process”. Nowadays a spin-off company Datadvance develops a Software platform for Design Space Exploration with GUI based on this algorithmic core. Evgeny's current research focuses on the development of new algorithms in machine learning and artificial intelligence such as deep networks for an approximation of physical models, generative modeling, and manifold learning, with applications to computer vision and 3D reconstruction, neurovisualization. The results are published in top computer science conferences (ICML, ICLR, NeurIPS, CVPR, ICCV, and ECCV) and journals. Evgeny Burnaev was honored with several awards for his research, including the Moscow Government Prize for Young Scientists in the category for the Transmission, Storage, Processing and Protection of Information for leading the project “The development of methods for predictive analytics for processing industrial, biomedical and financial data”, Geometry Processing Dataset Award for the work “ABC Dataset: A Big CAD Model Dataset For Geometric Deep Learning”, Symposium on Geometry Processing (2019), the Best Paper Award for the research in eSports at the IEEE Internet of People conference (2019), the Ilya Segalovich Yandex Science Prize “The best research director of postgraduate students in the field of computer sciences” (2020), the Best Paper Award for the research on modeling of point clouds and predicting properties of 3D shapes at the Int. Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR) (2020). FIND EVGENY ON SOCIAL MEDIA LinkedIn | Facebook | Instagram | Twitter © Copyright 2022 Den of Rich. All rights reserved.

#154 - Evgeny Burnaev

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later May 27, 2021 84:09

Evgeny Burnaev, Ph.D. Associate Professor of Center for Computational and Data-Intensive Science and Engineering (CDISE) at Skolkovo Institute of Science and Technology (Skoltech). Evgeny graduated from the Moscow Institute of Physics and Technology in 2006. After getting a Candidate of Sciences degree from the Institute for Information Transmission Problem in 2008, he stayed with the Institute as a head of the Data Analysis and Predictive Modeling Lab.Since 2007 Evgeny carried out a number of successful industrial projects with Airbus, SAFT, IHI, and Sahara Force India Formula 1 team among others. The corresponding data analysis algorithms, developed by Evgeny and his scientific group, formed a core of the algorithmic software library for metamodeling and optimization. Thanks to the developed functionality, engineers can construct fast mathematical approximations to long-running computer codes (realizing physical models) based on available data and perform design space exploration for trade-off studies. The software library passed the final Technology Readiness Level certification in Airbus. According to Airbus experts, the application of the library “provides the reduction of up to 10% of lead time and cost in several areas of the aircraft design process”. Nowadays a spin-off company Datadvance develops a Software platform for Design Space Exploration with GUI based on this algorithmic core.Evgeny's current research focuses on the development of new algorithms in machine learning and artificial intelligence such as deep networks for an approximation of physical models, generative modeling, and manifold learning, with applications to computer vision and 3D reconstruction, neurovisualization. The results are published in top computer science conferences (ICML, ICLR, NeurIPS, CVPR, ICCV, and ECCV) and journals.Evgeny Burnaev was honored with several awards for his research, including Moscow Government Prize for Young Scientists in the category for the Transmission, Storage, Processing and Protection of Information for leading the project “The development of methods for predictive analytics for processing industrial, biomedical and financial data”, Geometry Processing Dataset Award for the work “ABC Dataset: A Big CAD Model Dataset For Geometric Deep Learning”, Symposium on Geometry Processing (2019), the Best Paper Award for the research in eSports at the IEEE Internet of People conference (2019), the Ilya Segalovich Yandex Science Prize “The best research director of postgraduate students in the field of computer sciences” (2020), the Best Paper Award for the research on modeling of point clouds and predicting properties of 3D shapes at the Int. Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR) (2020).FIND EVGENY ON SOCIAL MEDIALinkedIn | Facebook | Instagram | Twitter

Learning Long-Time Dependencies with RNNs w/ Konstantin Rusch - #484

Play Episode Listen Later May 17, 2021 36:15

Today we conclude our 2021 ICLR coverage joined by Konstantin Rusch, a PhD Student at ETH Zurich. In our conversation with Konstantin, we explore his recent papers, titled coRNN and uniCORNN respectively, which focus on a novel architecture of recurrent neural networks for learning long-time dependencies. We explore the inspiration he drew from neuroscience when tackling this problem, how the performance results compared to networks like LSTMs and others that have been proven to work on this problem and Konstantin’s future research goals. The complete show notes for this episode can be found at twimlai.com/go/484.

learning neuroscience longtime neurobiology konstantin neurons phd student eth zurich dependencies rusch lstm rnns schmidhuber iclr lstms

What the Human Brain Can Tell Us About NLP Models with Allyson Ettinger - #483

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later May 13, 2021 36:33

Today we continue our ICLR ‘21 series joined by Allyson Ettinger, an Assistant Professor at the University of Chicago. One of our favorite recurring conversations on the podcast is the two-way street that lies between machine learning and neuroscience, which Allyson explores through the modeling of cognitive processes that pertain to language. In our conversation, we discuss how she approaches assessing the competencies of AI, the value of control of confounding variables in AI research, and how the pattern matching traits of Ml/DL models are not necessarily exclusive to these systems. Allyson also participated in a recent panel discussion at the ICLR workshop How Can Findings About The Brain Improve AI Systems?, centered around the utility of brain inspiration for developing AI models. We discuss ways in which we can try to more closely simulate the functioning of a brain, where her work fits into the analysis and interpretability area of NLP, and much more! The complete show notes for this episode can be found at twimlai.com/go/483.

university chicago ai language assistant professor neuroscience models analysis nlp linguistics black box qualcomm competency cognitive science human brain representations ettinger interpretability iclr

Probabilistic Numeric CNNs with Roberto Bondesan - #482

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later May 10, 2021 39:31

Today we kick off our ICLR 2021 coverage joined by Roberto Bondesan, an AI Researcher at Qualcomm. In our conversation with Roberto, we explore his paper Probabilistic Numeric Convolutional Neural Networks, which represents features as Gaussian processes, providing a probabilistic description of discretization error. We discuss some of the other work the team at Qualcomm presented at the conference, including a paper called Adaptive Neural Compression, as well as work on Guage Equvariant Mesh CNNs. Finally, we briefly discuss quantum deep learning, and what excites Roberto and his team about the future of their research in combinatorial optimization. The complete show notes for this episode can be found at https://twimlai.com/go/482

optimization qualcomm deep learning compression probabilistic gaussian numeric iclr

Ivan Oseledets | Иван Оселедец

Play Episode Listen Later Apr 26, 2021 88:16

Ivan Oseledets is a Professor of the Center for Computational and Data-Intensive Science and Engineering at Skoltech. Ivan graduated from Moscow Institute of Physics and Technology in 2006, got Candidate of Sciences degree in 2007, and Doctor of Sciences in 2012, both from Marchuk Institute of Numerical Mathematics of Russian Academy of Sciences. He joined Skoltech CDISE in 2013. Ivan's research covers a broad range of topics. He proposed a new decomposition of high-dimensional arrays (tensors) – tensor-train decomposition, and developed many efficient algorithms for solving high-dimensional problems. These algorithms are used in different areas of chemistry, biology, data analysis and machine learning. His current research focuses on development of new algorithms in machine learning and artificial intelligence such as construction of adversarial examples, theory of generative adversarial networks and compression of neural networks. It resulted in publications in top computer science conferences such as ICML, NIPS, ICLR, CVPR, RecSys, ACL and ICDM. Ivan is an Associate Editor of SIAM Journal on Mathematics in Data Science, SIAM Journal on Scientific Computing, Advances in Computational Mathematics (Springer). He is also an area chair of ICLR 2020 conference. Ivan got several awards for his research and industrial cooperation, including two gold medals of Russian academy of Sciences (for students in 2005 and young researchers in 2009), Dynasty Foundation award (2012), SIAM Outstanding Paper Prize (2018), Russian President Award for young researchers in science and innovation (2018), Ilya Segalovich award for Best PhD thesis supervisor (2019), Best Professor award from Skoltech (2019), the best cooperation project leader award from Huawei (2015, 2017). He also has been a Pi and Co-Pi of several grants and industrial projects (230 million of rubles since 2017). In 2021, Ivan became one of the winners of the Humboldt Research Award, is an award given by the Alexander von Humboldt Foundation of Germany. Ivan is actively involved in education and research supervision: he introduced and is teaching three courses of Skoltech curriculum, and five of his PhD students have successfully defended their theses, including two PhD students at Skoltech. FIND IVAN ON SOCIAL MEDIA LinkedIn | Facebook | Instagram | Twitter © Copyright 2022 Den of Rich. All rights reserved.

#128 - Ivan Oseledets

vision phd building psychology salem machines brains boston college stable biases serre generalization hamilton college paul j inductive neurips iclr

Play Episode Listen Later Apr 26, 2021 88:17

Ivan Oseledets is a Professor of the Center for Computational and Data-Intensive Science and Engineering at Skoltech.Ivan graduated from Moscow Institute of Physics and Technology in 2006, got Candidate of Sciences degree in 2007, and Doctor of Sciences in 2012, both from Marchuk Institute of Numerical Mathematics of Russian Academy of Sciences. He joined Skoltech CDISE in 2013.Ivan's research covers a broad range of topics. He proposed a new decomposition of high-dimensional arrays (tensors) – tensor-train decomposition, and developed many efficient algorithms for solving high-dimensional problems. These algorithms are used in different areas of chemistry, biology, data analysis and machine learning. His current research focuses on development of new algorithms in machine learning and artificial intelligence such as construction of adversarial examples, theory of generative adversarial networks and compression of neural networks. It resulted in publications in top computer science conferences such as ICML, NIPS, ICLR, CVPR, RecSys, ACL and ICDM.Ivan is an Associate Editor of SIAM Journal on Mathematics in Data Science, SIAM Journal on Scientific Computing, Advances in Computational Mathematics (Springer). He is also an area chair of ICLR 2020 conference.Ivan got several awards for his research and industrial cooperation, including two gold medals of Russian academy of Sciences (for students in 2005 and young researchers in 2009), Dynasty Foundation award (2012), SIAM Outstanding Paper Prize (2018), Russian President Award for young researchers in science and innovation (2018), Ilya Segalovich award for Best PhD thesis supervisor (2019), Best Professor award from Skoltech (2019), the best cooperation project leader award from Huawei (2015, 2017). He also has been a Pi and Co-Pi of several grants and industrial projects (230 million of rubles since 2017). In 2021, Ivan became one of the winners of the Humboldt Research Award, is an award given by the Alexander von Humboldt Foundation of Germany.Ivan is actively involved in education and research supervision: he introduced and is teaching three courses of Skoltech curriculum, and five of his PhD students have successfully defended their theses, including two PhD students at Skoltech.FIND IVAN ON SOCIAL MEDIALinkedIn | Facebook | Instagram | Twitter

Episode 61: Meta Reinforcement Learning with Louis Kirsch

Datacast

Play Episode Listen Later Apr 18, 2021 61:04

Show Notes(2:05) Louis went over his childhood as a self-taught programmer and his early days in school as a freelance developer.(4:22) Louis described his overall undergraduate experience getting a Bachelor’s degree in IT Systems Engineering from Hasso Plattner Institute, a highly-ranked computer science university in Germany.(6:10) Louis dissected his Bachelor thesis at HPI called “Differentiable Convolutional Neural Network Architectures for Time Series Classification,” — which addresses the problem of automatically designing architectures for time series classification efficiently, using a regularization technique for ConvNet that enables joint training of network weights and architecture through back-propagation.(7:40) Louis provided a brief overview of his publication “Transfer Learning for Speech Recognition on a Budget,” — which explores Automatic Speech Recognition training by model adaptation under constrained GPU memory, throughput, and training data.(10:31) Louis described his one-year Master of Research degree in Computational Statistics and Machine Learning at the University College London supervised by David Barber.(12:13) Louis unpacked his paper “Modular Networks: Learning to Decompose Neural Computation,” published at NeurIPS 2018 — which proposes a training algorithm that flexibly chooses neural modules based on the processed data.(15:13) Louis briefly reviewed his technical report, “Scaling Neural Networks Through Sparsity,” which discusses near-term and long-term solutions to handle sparsity between neural layers.(18:30) Louis mentioned his report, “Characteristics of Machine Learning Research with Impact,” which explores questions such as how to measure research impact and what questions the machine learning community should focus on to maximize impact.(21:16) Louis explained his report, “Contemporary Challenges in Artificial Intelligence,” which covers lifelong learning, scalability, generalization, self-referential algorithms, and benchmarks.(23:16) Louis talked about his motivation to start a blog and discussed his two-part blog series on intelligence theories (part 1 on universal AI and part 2 on active inference).(27:46) Louis described his decision to pursue a Ph.D. at the Swiss AI Lab IDSIA in Lugano, Switzerland, where he has been working on Meta Reinforcement Learning agents with Jürgen Schmidhuber.(30:06) Louis created a very extensive map of reinforcement learning in 2019 that outlines the goal, methods, and challenges associated with the RL domain.(33:50) Louis unpacked his blog post reflecting on his experience at NeurIPS 2018 and providing updates on the AGI roadmap regarding topics such as scalability, continual learning, meta-learning, and benchmarks.(37:04) Louis dissected his ICLR 2020 paper “Improving Generalization in Meta Reinforcement Learning using Learned Objectives,” which introduces a novel algorithm called MetaGenRL, inspired by biological evolution.(44:03) Louis elaborated on his publication “Meta-Learning Backpropagation And Improving It,” which introduces the Variable Shared Meta-Learning framework that unifies existing meta-learning approaches and demonstrates that simple weight-sharing and sparsity in a network are sufficient to express powerful learning algorithms.(51:14) Louis expands on his idea to bootstrap AI that entails how the task, the general meta learner, and the unsupervised objective should interact (proposed at the end of his invited talk at NeurIPS 2020).(54:14) Louis shared his advice for individuals who want to make a dent in AI research.(56:05) Louis shared his three most useful productivity tips.(58:36) Closing segment.Louis’s Contact InfoWebsiteTwitterLinkedInGoogle ScholarGitHubMentioned ContentPapers and ReportsDifferentiable Convolutional Neural Network Architectures for Time Series Classification (2017)Transfer Learning for Speech Recognition on a Budget (2017)Modular Networks: Learning to Decompose Neural Computation (2018)Contemporary Challenges in Artificial Intelligence (2018)Characteristics of Machine Learning Research with Impact (2018)Scaling Neural Networks Through Sparsity (2018)Improving Generalization in Meta Reinforcement Learning using Learned Objectives (2019)Meta-Learning Backpropagation And Improving It (2020)Blog PostsTheories of Intelligence — Part 1 and Part 2 (July 2018)Modular Networks: Learning to Decompose Neural Computation (May 2018)How to Make Your ML Research More Impactful (Dec 2018)A Map of Reinforcement Learning (Jan 2019)NeurIPS 2018, Updates on the AI Roadmap (Jan 2019)MetaGenRL: Improving Generalization in Meta Reinforcement Learning (Oct 2019)General Meta-Learning and Variable Sharing (Nov 2020)PeopleJeff Clune (for his push on meta-learning research)Kenneth Stanley (for his deep thoughts on open-ended learning)Jürgen Schmidhuber (for being a visionary scientist)Book“Grit” (by Angela Duckworth)

Episode 61: Meta Reinforcement Learning with Louis Kirsch

DataCast

Play Episode Listen Later Apr 18, 2021 61:04

Show Notes(2:05) Louis went over his childhood as a self-taught programmer and his early days in school as a freelance developer.(4:22) Louis described his overall undergraduate experience getting a Bachelor’s degree in IT Systems Engineering from Hasso Plattner Institute, a highly-ranked computer science university in Germany.(6:10) Louis dissected his Bachelor thesis at HPI called “Differentiable Convolutional Neural Network Architectures for Time Series Classification,” — which addresses the problem of automatically designing architectures for time series classification efficiently, using a regularization technique for ConvNet that enables joint training of network weights and architecture through back-propagation.(7:40) Louis provided a brief overview of his publication “Transfer Learning for Speech Recognition on a Budget,” — which explores Automatic Speech Recognition training by model adaptation under constrained GPU memory, throughput, and training data.(10:31) Louis described his one-year Master of Research degree in Computational Statistics and Machine Learning at the University College London supervised by David Barber.(12:13) Louis unpacked his paper “Modular Networks: Learning to Decompose Neural Computation,” published at NeurIPS 2018 — which proposes a training algorithm that flexibly chooses neural modules based on the processed data.(15:13) Louis briefly reviewed his technical report, “Scaling Neural Networks Through Sparsity,” which discusses near-term and long-term solutions to handle sparsity between neural layers.(18:30) Louis mentioned his report, “Characteristics of Machine Learning Research with Impact,” which explores questions such as how to measure research impact and what questions the machine learning community should focus on to maximize impact.(21:16) Louis explained his report, “Contemporary Challenges in Artificial Intelligence,” which covers lifelong learning, scalability, generalization, self-referential algorithms, and benchmarks.(23:16) Louis talked about his motivation to start a blog and discussed his two-part blog series on intelligence theories (part 1 on universal AI and part 2 on active inference).(27:46) Louis described his decision to pursue a Ph.D. at the Swiss AI Lab IDSIA in Lugano, Switzerland, where he has been working on Meta Reinforcement Learning agents with Jürgen Schmidhuber.(30:06) Louis created a very extensive map of reinforcement learning in 2019 that outlines the goal, methods, and challenges associated with the RL domain.(33:50) Louis unpacked his blog post reflecting on his experience at NeurIPS 2018 and providing updates on the AGI roadmap regarding topics such as scalability, continual learning, meta-learning, and benchmarks.(37:04) Louis dissected his ICLR 2020 paper “Improving Generalization in Meta Reinforcement Learning using Learned Objectives,” which introduces a novel algorithm called MetaGenRL, inspired by biological evolution.(44:03) Louis elaborated on his publication “Meta-Learning Backpropagation And Improving It,” which introduces the Variable Shared Meta-Learning framework that unifies existing meta-learning approaches and demonstrates that simple weight-sharing and sparsity in a network are sufficient to express powerful learning algorithms.(51:14) Louis expands on his idea to bootstrap AI that entails how the task, the general meta learner, and the unsupervised objective should interact (proposed at the end of his invited talk at NeurIPS 2020).(54:14) Louis shared his advice for individuals who want to make a dent in AI research.(56:05) Louis shared his three most useful productivity tips.(58:36) Closing segment.Louis’s Contact InfoWebsiteTwitterLinkedInGoogle ScholarGitHubMentioned ContentPapers and ReportsDifferentiable Convolutional Neural Network Architectures for Time Series Classification (2017)Transfer Learning for Speech Recognition on a Budget (2017)Modular Networks: Learning to Decompose Neural Computation (2018)Contemporary Challenges in Artificial Intelligence (2018)Characteristics of Machine Learning Research with Impact (2018)Scaling Neural Networks Through Sparsity (2018)Improving Generalization in Meta Reinforcement Learning using Learned Objectives (2019)Meta-Learning Backpropagation And Improving It (2020)Blog PostsTheories of Intelligence — Part 1 and Part 2 (July 2018)Modular Networks: Learning to Decompose Neural Computation (May 2018)How to Make Your ML Research More Impactful (Dec 2018)A Map of Reinforcement Learning (Jan 2019)NeurIPS 2018, Updates on the AI Roadmap (Jan 2019)MetaGenRL: Improving Generalization in Meta Reinforcement Learning (Oct 2019)General Meta-Learning and Variable Sharing (Nov 2020)PeopleJeff Clune (for his push on meta-learning research)Kenneth Stanley (for his deep thoughts on open-ended learning)Jürgen Schmidhuber (for being a visionary scientist)Book“Grit” (by Angela Duckworth)

Episode 9: Drew Linsley, Brown, on inductive biases for vision and generalization

Generally Intelligent

Play Episode Listen Later Apr 2, 2021 72:22

Drew Linsley (Google Scholar) (Website) is a Paul J. Salem senior research associate at Brown, advised by Thomas Serre. He is working on building computational models of the visual system that serve the dual purpose of (1) explaining biological function and (2) extending artificial vision. Prior to his work in the Serre lab, he completed a PhD in computational neuroscience at Boston College and a BA in Psychology at Hamilton College. His most recent paper at NeurIPS is Stable and expressive recurrent vision models. It presents an alternative to back-propagation through time (BPTT) for recurrent vision models called "contractor recurrent back-propagation" (C-RBP), which has O(1) complexity for an N step model vs. O(N) for BPTT, and which learns long-range spatial dependencies in cases where BPTT cannot. Drew is also organizing an ICLR 2021 workshop named Generalization Beyond the Training Distribution in Brains and Machines on Friday, May 7th, 2021. Find them on the website and @ICLR_brains. Lastly, Drew is looking to work with collaborators in robotics, so feel free to reach out! Highlights from our conversation:

Facebook AI Research’s Tim & Heinrich on democratizing reinforcement learning research

Gradient Dissent - A Machine Learning Podcast by W&B

Play Episode Listen Later Mar 4, 2021 54:09

Since reinforcement learning requires hefty compute resources, it can be tough to keep up without a serious budget of your own. Find out how the team at Facebook AI Research (FAIR) is looking to increase access and level the playing field with the help of NetHack, an archaic rogue-like video game from the late 80s. Links discussed: The NetHack Learning Environment: https://ai.facebook.com/blog/nethack-learning-environment-to-advance-deep-reinforcement-learning/ Reinforcement learning, intrinsic motivation: https://arxiv.org/abs/2002.12292 Knowledge transfer: https://arxiv.org/abs/1910.08210 Tim Rocktäschel is a Research Scientist at Facebook AI Research (FAIR) London and a Lecturer in the Department of Computer Science at University College London (UCL). At UCL, he is a member of the UCL Centre for Artificial Intelligence and the UCL Natural Language Processing group. Prior to that, he was a Postdoctoral Researcher in the Whiteson Research Lab, a Stipendiary Lecturer in Computer Science at Hertford College, and a Junior Research Fellow in Computer Science at Jesus College, at the University of Oxford. https://twitter.com/_rockt Heinrich Kuttler is an AI and machine learning researcher at Facebook AI Research (FAIR) and before that was a research engineer and team lead at DeepMind. https://twitter.com/HeinrichKuttler https://www.linkedin.com/in/heinrich-kuttler/ Topics covered: 0:00 a lack of reproducibility in RL 1:05 What is NetHack and how did the idea come to be? 5:46 RL in Go vs NetHack 11:04 performance of vanilla agents, what do you optimize for 18:36 transferring domain knowledge, source diving 22:27 human vs machines intrinsic learning 28:19 ICLR paper - exploration and RL strategies 35:48 the future of reinforcement learning 43:18 going from supervised to reinforcement learning 45:07 reproducibility in RL 50:05 most underrated aspect of ML, biggest challenges? Get our podcast on these other platforms: Apple Podcasts: http://wandb.me/apple-podcasts Spotify: http://wandb.me/spotify Google: http://wandb.me/google-podcasts YouTube: http://wandb.me/youtube Soundcloud: http://wandb.me/soundcloud Tune in to our bi-weekly virtual salon and listen to industry leaders and researchers in machine learning share their research: http://wandb.me/salon Join our community of ML practitioners where we host AMA's, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack Our gallery features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, and industry leaders sharing best practices: https://wandb.ai/gallery

Conducting fundamental machine learning research as a non-profit with MLC's founder Rosanne Liu

Gradient Dissent - A Machine Learning Podcast by W&B

Play Episode Listen Later Feb 5, 2021 49:09

How Rosanne is working to democratize AI research and improve diversity and fairness in the field through starting a non-profit after being a founding member of Uber AI Labs, doing lots of amazing research, and publishing papers at top conferences. Rosanne is a machine learning researcher, and co-founder of ML Collective, a nonprofit organization for open collaboration and mentorship. Before that, she was a founding member of Uber AI. She has published research at NeurIPS, ICLR, ICML, Science, and other top venues. While at school she used neural networks to help discover novel materials and to optimize fuel efficiency in hybrid vehicles. ML Collective: http://mlcollective.org/ Controlling Text Generation with Plug and Play Language Models: https://eng.uber.com/pplm/ LCA: Loss Change Allocation for Neural Network Training: https://eng.uber.com/research/lca-loss-change-allocation-for-neural-network-training/ Topics covered 0:00 Sneak peek, Intro 1:53 The origin of ML Collective 5:31 Why a non-profit and who is MLC for? 14:30 LCA, Loss Change Allocation 18:20 Running an org, research vs admin work 20:10 Advice for people trying to get published 24:15 on reading papers and Intrinsic Dimension paper 36:25 NeurIPS - Open Collaboration 40:20 What is your reward function? 44:44 Underrated aspect of ML 47:22 How to get involved with MLC Get our podcast on these other platforms: Apple Podcasts: http://wandb.me/apple-podcasts Spotify: http://wandb.me/spotify Google: http://wandb.me/google-podcasts YouTube: http://wandb.me/youtube Tune in to our bi-weekly virtual salon and listen to industry leaders and researchers in machine learning share their research: http://wandb.me/salon Join our community of ML practitioners where we host AMA's, share interesting projects and meet other people working in Deep Learning: http://wandb.me/slack Our gallery features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, and industry leaders sharing best practices: https://wandb.ai/gallery

Trends in Reinforcement Learning with Pablo Samuel Castro - #443

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later Dec 30, 2020 86:38

Today we kick off our annual AI Rewind series joined by friend of the show Pablo Samuel Castro, a Staff Research Software Developer at Google Brain. Pablo joined us earlier this year for a discussion about Music & AI, and his Geometric Perspective on Reinforcement Learning, as well our RL office hours during the inaugural TWIMLfest. In today’s conversation, we explore some of the latest and greatest RL advancements coming out of the major conferences this year, broken down into a few major themes, Metrics/Representations, Understanding and Evaluating Deep Reinforcement Learning, and RL in the Real World. This was a very fun conversation, and we encourage you to check out all the great papers and other resources available on the show notes page.

ai technology tech data castro real world machine learning open source data science ml rl reinforcement learning google brain neurips icml iclr twiml

How can an AI understand language? Scott Leishman, XOKind | Productive AI Podcast with Troy Angrignon

Productive AI Podcast

Play Episode Listen Later Dec 16, 2020 56:15

How can an AI understand language? Computer-human communication is undergoing a revolution and AI can now listen to, understand, and speak back to us in much more powerful ways than it could before. On this episode, hear Scott Leishman discuss how AI can now write news articles, blog posts, poetry, and novels and how work done in the recent past is making it easier than ever to build incredibly powerful AI applications that can communicate with human beings. -- TIMING – 00:00 Introduction00:48 Scott’s background in computer science at FICO, Core Logic, and Nirvana Systems (which exited to Intel for $400M in 2016), and Intel06:56 What is Natural Language Processing (NLP)?11:40 What was the significance of GPT-3’s release this year?16:31 What can GPT-3 do? (explain it to somebody who doesn’t follow the field). 19:15 NLP is having its “ImageNet moment” – what does that mean? (Technical explanation)25:39 Simplifying NLP for less-technical listeners28:17 Standing on the shoulders of giants: Pre-trained models are making it easier to build AI applications30:05 What kinds of new uses cases are possible with the current state of the art NLP?33:29 Apple Knowledge Navigator – are we there yet?37:25 Where does NLP live in the AI stack?41:34 What are you doing with NLP at XOKind?49:47 What should people be doing to improve their chances of working in this space?54:05 Summary -- LINKS -- Books: Manning & Jurafsky is sort of the best known, comprehensive but is a bit dated at this point. Fortunately they are working on a new draft: https://web.stanford.edu/~jurafsky/slp3/ Conferences: the big ones for NLP are ACL, EMNLP (was just last week), CoNLL, but you’ll also see a lot of new work at ICLR and NeurIPS Papers. The field moves quick but arXiv is the first place to find new results. I’d highly recommend searching through something like arxiv-sanity instead for a subject/topic of interest. Mailing lists: I’m a big fan of Sebastian Reuder’s monthly update, you can sign up for at NLP news https://ruder.io/nlp-news/ Sites: I mentioned https://nlpprogress.com/ to keep tabs on current state of the art for given downstream tasksFor folks that want a good practical introduction I’d recommend Stanford’s undergraduate NLP course (complete with video lectures online): http://web.stanford.edu/class/cs224n/ Getting interested in ML in general, this course is pretty good too if you have some programming experience under your belt: https://course.fast.ai/ Hugging Face are doing a lot of great work in the NLP space, they have easy integrations for various models, a solid python library etc. Rasa are another open source solution, they now have APIs too for helping build conversation agents XOKind! Sign up for our mailing list on the front page here: https://www.xokind.com/ Job openings. List is here: https://www.xokind.com/careers/ (scroll down the page). Growing Frontend and Backend engineering is a current focus for us. Apple Knowledge Navigator Video: https://www.youtube.com/watch?v=HGYFEI6uLy0

Dubbel feest voor beleggers

Beurswatch | BNR

Play Episode Listen Later Nov 13, 2020 22:20

Dat het aantal coronabesmettingen weer in de lift zit, lijkt beleggers niet te raken. De blijdschap over het coronavaccin van Pfizer voert de boventoon op de beurzen. Ook de overwinning van Biden, die een gematigd beleid zal voeren, is niet slecht voor beleggers. Vooral vastgoedaandelen profiteerden van het goede nieuws. Olaf van den Heuvel (Aegon Asset Management) tipt dan ook het Franse vastgoedbedrijf Nexity (ticker: NEXI.PA). Wim Zwanenburg (Stroeve Lemberger) gaat voor het Ierse biotechbedrijf Icon (ticker: ICLR.O).

joe biden pfizer icon voor olaf offshore vooral franse feest dubbel abn ierse amro sbm beleggers nexi iclr

Beurswatch

Podcast | BNR

Play Episode Listen Later Nov 13, 2020 22:20

Dat het aantal coronabesmettingen weer in de lift zit, lijkt beleggers niet te raken. De blijdschap over het coronavaccin van Pfizer voert de boventoon op de beurzen. Ook de overwinning van Biden, die een gematigd beleid zal voeren, is niet slecht voor beleggers. Vooral vastgoedaandelen profiteerden van het goede nieuws. Olaf van den Heuvel (Aegon Asset Management) tipt dan ook het Franse vastgoedbedrijf Nexity (ticker: NEXI.PA). Wim Zwanenburg (Stroeve Lemberger) gaat voor het Ierse biotechbedrijf Icon (ticker: ICLR.O).

joe biden pfizer icon olaf offshore vooral franse abn ierse amro sbm nexi iclr

Naila Murray - Interestingness predictions and getting to grips with data privacy

phd predictions barcelona rank princeton university electrical engineering data privacy grips naila nle generative models cvpr iclr

Play Episode Listen Later Aug 14, 2020 68:52

This week we are joined by Naila Murray. Naila obtained a B.Sc. in Electrical Engineering from Princeton University in 2007. In 2012, she received her PhD from the Universitat Autonoma de Barcelona, in affiliation with the Computer Vision Center. She joined NAVER LABS Europe (then Xerox Research Centre Europe) in January 2013, working on topics including fine-grained visual categorization, image retrieval, and visual attention. From 2015 to 2019 she led the computer vision team at NLE. She currently serves as NLE's director of science. She serves/served as area chair for ICLR 2018, ICCV 2019, ICLR 2019, CVPR 2020, ECCV 2020, and programme chair for ICLR 2021. Her research interests include representation learning and multi-modal search.We discuss using sparse pairwise comparisons to learn a ranking function that is robust to outliers. We also take a look at using generative models in order to utilise once inaccessible datasets.Underrated ML Twitter: https://twitter.com/underrated_mlNaila Murray Twitter: https://twitter.com/NailaMurrayPlease let us know who you thought presented the most underrated paper in the form below: https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:"Interestingness Prediction by Robust Learning to Rank" [paper]"Generative Models for Effective ML on Private Decentralized datasets" - [paper]

Anna Huang - Metaphor generation and ML for child welfare

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later Jul 22, 2020 73:38

We open season two of Underrated ML with Anna Huang on the show. Anna Huang is a Research Scientist at Google Brain, working on the Magenta project. Her research focuses on designing generative models to make creating music more approachable. She is the creator of Music Transformer and also the ML model Coconet that powered Google’s first AI Doodle the Bach Doodle.She holds a PhD in computer science from Harvard University and was a recipient of the NSF Graduate Research Fellowship. She spent the later parts of her PhD as a visiting research student at the Montreal Institute of Learning Algorithms (MILA). She publishes in machine learning, human-computer interaction, and music, at conferences such as ICLR, IUI, CHI, and ISMIR.She has been a judge on the Eurovision AI Song Contest and her compositions have won awards including first place in the San Francisco Choral Artists’ a cappella composition contest. She holds a masters in media arts and sciences from the MIT Media Lab, and a B.S. in computer science and B.M. in music composition both from the University of Southern California. She grew up in Hong Kong, where she learned to play the guzheng.On the episode we discuss Metaphoria by Kate Gero and Lydia Chilton, which is a fascinating tool allowing users to generate metaphors from only a select number of words. We also discuss the current trends regarding the dangers of AI with a case study on child welfare.Underrated ML Twitter: https://twitter.com/underrated_mlAnna Huang Twitter: https://twitter.com/huangczaPlease let us know who you thought presented the most underrated paper in the form below: https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:Gero, Katy Ilonka, and Lydia B. Chilton. "Metaphoria: An Algorithmic Companion for Metaphor Creation." CHI 2019. [paper][online paper] [talk] [demo]"A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions" - [paper]Additional Links:Compton, Kate, and Michael Mateas. "Casual Creators." ICCC 2015. [paper]Fiebrink, Rebecca, Dan Trueman, and Perry R. Cook. "A Meta-Instrument for Interactive, On-the-Fly Machine Learning." NIME 2009. [paper][talk][tool]Huang, Cheng-Zhi Anna, et al. "The Bach Doodle: Approachable music composition with machine learning at scale." ISMIR 2019. [paper][blog][doodle]

Channel Gating for Cheaper and More Accurate Neural Nets with Babak Ehteshami Bejnordi - #385

Play Episode Listen Later Jun 22, 2020 55:58

Today we’re joined by Babak Ehteshami Bejnordi, a Research Scientist at Qualcomm. Babak works closely with former guest Max Welling and is currently focused on conditional computation, which is the main driver for today’s conversation. We dig into a few papers in great detail including one from this year’s CVPR conference, Conditional Channel Gated Networks for Task-Aware Continual Learning. We also discuss the paper TimeGate: Conditional Gating of Segments in Long-range Activities, and another paper from this year’s ICLR conference, Batch-Shaping for Learning Conditional Channel Gated Networks. We cover how gates are used to drive efficiency and accuracy, while decreasing model size, how this research manifests into actual products, and more! For more information on the episode, visit twimlai.com/talk/385. To follow along with the CVPR 2020 Series, visit twimlai.com/cvpr20. Thanks to Qualcomm for sponsoring today’s episode and the CVPR 2020 Series!

ai technology tech data series nets activities machine learning data science ml accurate cheaper segments qualcomm neural deepmind research scientist babak gating cvpr iclr max welling twiml

Episode 34: Deep Learning Generalization, Representation, and Abstraction with Ari Morcos

Datacast

Play Episode Listen Later Jun 14, 2020 97:03

Show Notes(2:32) Ari discussed his undergraduate studying Physiology and Neuroscience at UC San Diego, while doing neuroscience research on adult neurogenesis at the Gage Lab.(4:39) Ari discussed his decision to pursue a Ph.D. in Neurobiology at Harvard after college and extracted the importance of communication in research, thanks to his advisor Chris Harvey.(7:16) Ari explained his Ph.D. thesis titled “Population dynamics in parietal cortex during evidence accumulation for decision-making” - in which he developed methods to understand how neuronal circuits perform the computations necessary for complex behavior.(12:59) Ari talked about his process of learning machine learning and using that to analyze massive neuroscience datasets in his research.(15:22) Ari recounted attending NIPS 2015 and serendipitously meeting people from DeepMind, which he lated joined as a Research Scientist in their London office.(18:59) Ari’s research focuses on the generalization of neural networks, and shared his work called "On the Importance of Single Directions for Generalization” presented at ICLR 2018 (inspired by Chiyuan Zhang’s paper and Quoc Le’s paper previously).(28:51) Ari explained the differences between generalizing networks and memorizing networks, citing the results from his work "Insights on Representational Similarity in Neural Networks with Canonical Correlation” with Maithra Raghu and Samy Bengio presented at NeurIPS 2018 (Read Maithra’s paper on SVCCA that inspired it).(35:16) Another topic that Ari focuses on is representation learning and abstraction for intelligent systems. His team at DeepMind proposes a dataset and a challenge designed to probe abstract reasoning, as explained in “Measuring Abstract Reasoning in Neural Networks" presented at ICML 2018 (learn more about the IQ test Raven’s Progressive Matrices and take the challenge here).(42:21) An extension from the work above is "Learning to Make Analogies by Contrasting Abstract Relational Structure" - presented at ICLR 2019. With the same authors (led by Felix Hill along with David Barrett, Adam Santoro, Tim Lillicrap), Ari showed that while architecture choice can influence generalization performance, the choice of data and the manner in which it is presented to the model is even more critical.(48:18) Ari discussed "Neural Scene Representation and Rendering” (led by Ali Eslami and Danilo Rezende) that introduces Generative Query Network (GQN), a framework within which machines learn to represent scenes using only their own sensors (watch the video and check out the data).(55:09) Ari explained the findings in "Analyzing Biological and Artificial Neural Networks: Challenges with Opportunities for Synergy?” published at the Current Opinion in Neurobiology (joint work with David Barrett and Jakob Macke).(57:04) Ari shared the properties of pruning algorithms that influence stability and generalization, as claimed in “The Generalization-Stability Tradeoff in Neural Network Pruning” led by Brian Bartoldson.(01:00:56) Ari went over the generalization of lottery tickets in neural networks, which is inspired by the lottery ticket hypothesis from Jonathan Frankle and Michael Carbin at MIT. The two papers mentioned are collaboration with Haonan Yu, Yuandong Tian, Michela Paganini, and Sergey Edunov (Check out his talk at REWORK Deep Learning Summit in Montreal 2019).(01:09:00) Ari investigated "Training BatchNorm and Only BatchNorm” which looks at the performance of neural networks when trained only with the Batch Normalization parameters (joint work with Jonathan Frankle and David Schwab).(01:12:12) Ari mentioned "The Early Phase of Neural Network Training” (presented at ICML 2020) that uses the lottery ticket framework to rigorously examine the early part of the training (joint work with Jonathan Frankle and David Schwab). (01:16:25) Ari discussed at length “Representation Learning Through Latent Canonicalizations" (presented at ICLR 2020). This work seeks to learn representations in which semantically meaningful factors of variation (like color or shape) can be independently manipulated by learned linear transformations in latent space, termed “latent canonicalizes” (joint work with Or Litany, Srinath Sridhar, Leonidas Guibas, and Judy Hoffman).(01:22:15) Ari summarized "Selectivity Considered Harmful: Evaluating the Causal Impact of Class Selectivity in DNNs" - which investigates the causal impact of class selectivity on network function (led by Matthew Leavitt).(01:25:26) Ari reflected on his career and shared advice for individuals who want to make a dent in AI research.(01:28:10) Ari shared his excitement on self-supervised learning, which addresses the need of neural networks to require expensive labeled data.(01:29:47) Closing segment.His Contact InformationWebsiteGoogle ScholarLinkedInTwitterGitHubHis Recommended Resources“Understanding Deep Learning Requires Rethinking Generalization” by Chiyuan Zhang“Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability” by Maithra RaghuRaven’s Progressive Matrices IQ test"The Lottery Ticket Hypothesis” by Jonathan Frankle and Michael Carbin (Open-Source Framework)“Random Features for Large-Scale Kernel Machines” by Ali Rahimi and Ben Recht (NIPS 2017 Test Of Time Award)“beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework” by DeepMindSamy Bengio (Research Scientist at Google AI)Aleksander Madry (Professor of Computer Science at MIT)Jason Yosinski (Founding Member of Uber AI Labs)“The Idea Factory: Bell Labs and The Great Age of American Innovation" by Jon Gertner

Episode 34: Deep Learning Generalization, Representation, and Abstraction with Ari Morcos

DataCast

Play Episode Listen Later Jun 14, 2020 97:03

Show Notes(2:32) Ari discussed his undergraduate studying Physiology and Neuroscience at UC San Diego, while doing neuroscience research on adult neurogenesis at the Gage Lab.(4:39) Ari discussed his decision to pursue a Ph.D. in Neurobiology at Harvard after college and extracted the importance of communication in research, thanks to his advisor Chris Harvey.(7:16) Ari explained his Ph.D. thesis titled “Population dynamics in parietal cortex during evidence accumulation for decision-making” - in which he developed methods to understand how neuronal circuits perform the computations necessary for complex behavior.(12:59) Ari talked about his process of learning machine learning and using that to analyze massive neuroscience datasets in his research.(15:22) Ari recounted attending NIPS 2015 and serendipitously meeting people from DeepMind, which he lated joined as a Research Scientist in their London office.(18:59) Ari’s research focuses on the generalization of neural networks, and shared his work called "On the Importance of Single Directions for Generalization” presented at ICLR 2018 (inspired by Chiyuan Zhang’s paper and Quoc Le’s paper previously).(28:51) Ari explained the differences between generalizing networks and memorizing networks, citing the results from his work "Insights on Representational Similarity in Neural Networks with Canonical Correlation” with Maithra Raghu and Samy Bengio presented at NeurIPS 2018 (Read Maithra’s paper on SVCCA that inspired it).(35:16) Another topic that Ari focuses on is representation learning and abstraction for intelligent systems. His team at DeepMind proposes a dataset and a challenge designed to probe abstract reasoning, as explained in “Measuring Abstract Reasoning in Neural Networks" presented at ICML 2018 (learn more about the IQ test Raven’s Progressive Matrices and take the challenge here).(42:21) An extension from the work above is "Learning to Make Analogies by Contrasting Abstract Relational Structure" - presented at ICLR 2019. With the same authors (led by Felix Hill along with David Barrett, Adam Santoro, Tim Lillicrap), Ari showed that while architecture choice can influence generalization performance, the choice of data and the manner in which it is presented to the model is even more critical.(48:18) Ari discussed "Neural Scene Representation and Rendering” (led by Ali Eslami and Danilo Rezende) that introduces Generative Query Network (GQN), a framework within which machines learn to represent scenes using only their own sensors (watch the video and check out the data).(55:09) Ari explained the findings in "Analyzing Biological and Artificial Neural Networks: Challenges with Opportunities for Synergy?” published at the Current Opinion in Neurobiology (joint work with David Barrett and Jakob Macke).(57:04) Ari shared the properties of pruning algorithms that influence stability and generalization, as claimed in “The Generalization-Stability Tradeoff in Neural Network Pruning” led by Brian Bartoldson.(01:00:56) Ari went over the generalization of lottery tickets in neural networks, which is inspired by the lottery ticket hypothesis from Jonathan Frankle and Michael Carbin at MIT. The two papers mentioned are collaboration with Haonan Yu, Yuandong Tian, Michela Paganini, and Sergey Edunov (Check out his talk at REWORK Deep Learning Summit in Montreal 2019).(01:09:00) Ari investigated "Training BatchNorm and Only BatchNorm” which looks at the performance of neural networks when trained only with the Batch Normalization parameters (joint work with Jonathan Frankle and David Schwab).(01:12:12) Ari mentioned "The Early Phase of Neural Network Training” (presented at ICML 2020) that uses the lottery ticket framework to rigorously examine the early part of the training (joint work with Jonathan Frankle and David Schwab). (01:16:25) Ari discussed at length “Representation Learning Through Latent Canonicalizations" (presented at ICLR 2020). This work seeks to learn representations in which semantically meaningful factors of variation (like color or shape) can be independently manipulated by learned linear transformations in latent space, termed “latent canonicalizes” (joint work with Or Litany, Srinath Sridhar, Leonidas Guibas, and Judy Hoffman).(01:22:15) Ari summarized "Selectivity Considered Harmful: Evaluating the Causal Impact of Class Selectivity in DNNs" - which investigates the causal impact of class selectivity on network function (led by Matthew Leavitt).(01:25:26) Ari reflected on his career and shared advice for individuals who want to make a dent in AI research.(01:28:10) Ari shared his excitement on self-supervised learning, which addresses the need of neural networks to require expensive labeled data.(01:29:47) Closing segment.His Contact InformationWebsiteGoogle ScholarLinkedInTwitterGitHubHis Recommended Resources“Understanding Deep Learning Requires Rethinking Generalization” by Chiyuan Zhang“Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability” by Maithra RaghuRaven’s Progressive Matrices IQ test"The Lottery Ticket Hypothesis” by Jonathan Frankle and Michael Carbin (Open-Source Framework)“Random Features for Large-Scale Kernel Machines” by Ali Rahimi and Ben Recht (NIPS 2017 Test Of Time Award)“beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework” by DeepMindSamy Bengio (Research Scientist at Google AI)Aleksander Madry (Professor of Computer Science at MIT)Jason Yosinski (Founding Member of Uber AI Labs)“The Idea Factory: Bell Labs and The Great Age of American Innovation" by Jon Gertner

Neural Arithmetic Units & Experiences as an Independent ML Researcher with Andreas Madsen - #382

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later Jun 11, 2020 30:54

Today we’re joined by Andreas Madsen, an independent researcher based in Denmark whose research focuses on developing interpretable machine learning models. While we caught up with Andreas to discuss his ICLR spotlight paper, “Neural Arithmetic Units,” we also spend time exploring his experience as an independent researcher. We discuss the difficulties of working with limited resources, the importance of finding peers to collaborate with, and tempering expectations of getting papers accepted to conferences -- something that might take a few tries to get right. In his paper, Andreas notes that Neural Networks struggle to perform exact arithmetic operations over real numbers, but this can be helped with the addition of two NN components: the Neural Addition Unit (NAU), which can learn exact addition and subtraction; and the Neural Multiplication Unit (NMU) that can multiply subsets of a vector. The complete show notes can be found at twimlai.com/talk/382.

ai technology tech data experiences independent denmark researchers andreas machine learning data science ml units neural madsen neural networks nn arithmetic iclr twiml

ICLR 2020: Yoshua Bengio and the Nature of Consciousness

Machine Learning Street Talk

Play Episode Listen Later May 22, 2020 154:17

In this episode of Machine Learning Street Talk, Tim Scarfe, Connor Shorten and Yannic Kilcher react to Yoshua Bengio’s ICLR 2020 Keynote “Deep Learning Priors Associated with Conscious Processing”. Bengio takes on many future directions for research in Deep Learning such as the role of attention in consciousness, sparse factor graphs and causality, and the study of systematic generalization. Bengio also presents big ideas in Intelligence that border on the line of philosophy and practical machine learning. This includes ideas such as consciousness in machines and System 1 and System 2 thinking, as described in Daniel Kahneman’s book “Thinking Fast and Slow”. Similar to Yann LeCun’s half of the 2020 ICLR keynote, this talk takes on many challenging ideas and hopefully this video helps you get a better understanding of some of them! Thanks for watching! Please Subscribe for more videos! Paper Links: Link to Talk: https://iclr.cc/virtual_2020/speaker_7.html The Consciousness Prior: https://arxiv.org/abs/1709.08568 Thinking Fast and Slow: https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555 Systematic Generalization: https://arxiv.org/abs/1811.12889 CLOSURE: Assessing Systematic Generalization of CLEVR Models: https://arxiv.org/abs/1912.05783 Neural Module Networks: https://arxiv.org/abs/1511.02799 Experience Grounds Language: https://arxiv.org/pdf/2004.10151.pdf Benchmarking Graph Neural Networks: https://arxiv.org/pdf/2003.00982.pdf On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Please check out our individual channels as well! Machine Learning Dojo with Tim Scarfe: https://www.youtube.com/channel/UCXvHuBMbgJw67i5vrMBBobA Yannic Kilcher: https://www.youtube.com/channel/UCZHmQk67mSJgfCCTn7xBfe Henry AI Labs: https://www.youtube.com/channel/UCHB9VepY6kYvZjj0Bgxnpbw 00:00:00 Tim and Yannics takes 00:01:37 Intro to Bengio 00:03:13 System 2, language and Chomsky 00:05:58 Cristof Koch on conciousness 00:07:25 Francois Chollet on intelligence and consciousness 00:09:29 Meditation and Sam Harris on consciousness 00:11:35 Connor Intro 00:13:20 Show Main Intro 00:17:55 Priors associated with Conscious Processing 00:26:25 System 1 / System 2 00:42:47 Implicit and Verbalized Knowledge [DONT MISS THIS!] 01:08:24 Inductive Priors for DL 2.0 01:27:20 Systematic Generalization 01:37:53 Contrast with the Symbolic AI Program 01:54:55 Attention 02:00:25 From Attention to Consciousness 02:05:31 Thoughts, Consciousness, Language 02:06:55 Sparse Factor graph 02:10:52 Sparse Change in Abstract Latent Space 02:15:10 Discovering Cause and Effect 02:20:00 Factorize the joint distribution 02:22:30 RIMS: Modular Computation 02:24:30 Conclusion #machinelearning #deeplearning

Can AI help tackle climate change?

ETH Podcast

Play Episode Listen Later May 20, 2020 35:52

Climate change hasn’t been hitting the headlines quite as much in recent months – but that’s not because the situation has improved. ETH Zurich researchers Lynn Kaack and David Dao spoke to the ETH Podcast back in March about how we can use AI to help in the fight against climate change. This episode of the ETH podcast is about Lynn Kaack’s and David Dao’s work on the energy transition and forests, their work with the organisation Climate Change AI, and their take on research, activism and policy. We pushed back the podcast that had been produced before the lockdown due to our special series on COVID-19 and it has now been supplemented with current statements from the two researchers. Because we wanted to know what had changed for them.

covid-19 ai artificial intelligence climate change sustainability climate machine learning tackle global warming eth fridays for future deforestation eth zurich david dao iclr

ICLR 2020: Yann LeCun and Energy-Based Models

Machine Learning Street Talk

Play Episode Listen Later May 19, 2020 132:11

This week Connor Shorten, Yannic Kilcher and Tim Scarfe reacted to Yann LeCun's keynote speech at this year's ICLR conference which just passed. ICLR is the number two ML conference and was completely open this year, with all the sessions publicly accessible via the internet. Yann spent most of his talk speaking about self-supervised learning, Energy-based models (EBMs) and manifold learning. Don't worry if you hadn't heard of EBMs before, neither had we! Thanks for watching! Please Subscribe! Paper Links: ICLR 2020 Keynote Talk: https://iclr.cc/virtual_2020/speaker_7.html A Tutorial on Energy-Based Learning: http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf Concept Learning with Energy-Based Models (Yannic's Explanation): https://www.youtube.com/watch?v=Cs_j-oNwGgg Concept Learning with Energy-Based Models (Paper): https://arxiv.org/pdf/1811.02486.pdf Concept Learning with Energy-Based Models (OpenAI Blog Post): https://openai.com/blog/learning-concepts-with-energy-functions/ #deeplearning #machinelearning #iclr #iclr2020 #yannlecun

energy models explanation cs ml yann yann lecun iclr

ICLR: accessible, inclusive, virtual

Talking Machines

Play Episode Listen Later May 14, 2020 38:42

In episode eight of season six we talk with Alexander Rush and Shakir Mohamed about their work on ICLR this year which was first to take place in Ethiopia and then became totally virtual!

virtual inclusive ethiopia accessible iclr

Kyunghyun Cho - Energy functions and shortcut learning

learning energy intelligence new york university makes shortcuts functions deep learning generalization deep neural networks classifier natural language understanding iclr

Play Episode Listen Later May 11, 2020 89:04

This week we are joined by Kyunghyun Cho. He is an associate professor of computer science and data science at New York University, a research scientist at Facebook AI Research and a CIFAR Associate Fellow. On top of this he also co-chaired the recent ICLR 2020 virtual conference.We talk about a variety of topics in this weeks episode including the recent ICLR conference, energy functions, shortcut learning and the roles popularized Deep Learning research areas play in answering the question “What is Intelligence?”.Underrated ML Twitter: https://twitter.com/underrated_mlKyunghyun Cho Twitter: https://twitter.com/kchonyc?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5EauthorPlease let us know who you thought presented the most underrated paper in the form below:https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:“Shortcut “Learning in Deep Neural Networks” - https://arxiv.org/pdf/2004.07780.pdf"Bayesian Deep Learning and a Probabilistic Perspective of Generalization” - https://arxiv.org/abs/2002.08791"Classifier-agnostic saliency map extraction" - https://arxiv.org/abs/1805.08249“Deep Energy Estimator Networks” - https://arxiv.org/abs/1805.08306“End-to-End Learning for Structured Prediction Energy Networks” - https://arxiv.org/abs/1703.05667“On approximating nabla f with neural networks” - https://arxiv.org/abs/1910.12744“Adversarial NLI: A New Benchmark for Natural Language Understanding“ - https://arxiv.org/abs/1910.14599“Learning the Difference that Makes a Difference with Counterfactually-Augmented Data” - https://arxiv.org/abs/1909.12434“Learning Concepts with Energy Functions” - https://openai.com/blog/learning-concepts-with-energy-functions/

ep50 (ICLR): ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Leading NLP Ninja

Play Episode Listen Later Mar 14, 2020 28:01

ICLR 2020より，Replaced Token Detectionタスクによる事前学習によってGLUEとSQuADでSOTAを獲得したStanford x Googleのモデルを解説しました．今回紹介した記事はこちらのissueで解説しています． https://github.com/jojonki/arXivNotes/issues/391 サポーターも募集中です． https://www.patreon.com/jojonki --- Support this podcast: https://anchor.fm/lnlp-ninja/support

google training stanford generators iclr

101 - The lottery ticket hypothesis, with Jonathan Frankle

hypothesis lottery tickets carbin iclr

Play Episode Listen Later Jan 14, 2020 41:16

In this episode, Jonathan Frankle describes the lottery ticket hypothesis, a popular explanation of how over-parameterization helps in training neural networks. We discuss pruning methods used to uncover subnetworks (winning tickets) which were initialized in a particularly effective way. We also discuss patterns observed in pruned networks, stability of networks pruned at different time steps and transferring uncovered subnetworks across tasks, among other topics. A recent paper on the topic by Frankle and Carbin, ICLR 2019: https://arxiv.org/abs/1803.03635 Jonathan Frankle's homepage: http://www.jfrankle.com/

ep47 (ICLR): ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Leading NLP Ninja

Play Episode Listen Later Jan 12, 2020 25:34

ICLR 2020より，factorized embeddingとパラメタ共有によるパラメタ削減及び文順序予測タスクを採用したALBERTを解説しました．今回紹介した記事はこちらのissueで解説しています． https://github.com/jojonki/arXivNotes/issues/348 サポーターも募集中です． https://www.patreon.com/jojonki --- Support this podcast: https://anchor.fm/lnlp-ninja/support

language lite representations supervised learning iclr

ep46: FreeLB: Enhanced Adversarial Training for Language Understanding

Leading NLP Ninja

Play Episode Listen Later Jan 1, 2020 24:21

ICLR 2020より，BERT/RoBERTaの埋め込み空間に，敵対摂動を入れるVirtual Adversarial Trainingによって，モデル性能を改善するFreeLBという手法を解説しました．今回紹介した記事はこちらのissueで解説しています． https://github.com/jojonki/arXivNotes/issues/347 サポーターも募集中です． https://www.patreon.com/jojonki --- Support this podcast: https://anchor.fm/lnlp-ninja/support

training language enhanced adversarial iclr

JAICs on a Plane with a Cube of the Rubik

AI with AI

Play Episode Listen Later Oct 25, 2019 54:06

Welcome to Season 3.0! Andy and Dave discuss the AI in Advancement Advisory Council’s State of AI Advancement report, which takes a look at the impact of AI on roles within advancement. Researchers at Fudan and Changchun Institute of Optics announce a 500 MP camera (with associated cloud-powered AI) capable of identifying a face among tens of thousands. The U.S. National Science Foundation announces the National AI Research Institute, which anticipates approving $120M in grants next year. A recent solicitation from the Defense Innovation Unit seeks to understand trends in world events. And the JAIC has a new website. In research, OpenAI announces Dactyl, a robot hand capable of solving Rubik’s cube, as part of an effort to build a general purpose robot (transferring learning from simulation to the real world), and robust to perturbations such as broken fingers or intrusions by plush giraffes. Research accepted to ICLR 2020 demonstrates the application of deep learning to symbolic mathematics. Dan Gettinger of Bard College publishes The Drone Databook, cataloging the drones from 101 countries. The Carnegie Endowment for International Peace takes a look at the origins of AI Surveillance Technology in use around the globe. The Oliver Wyman Forum measures Global Cities’ AI Readiness, and Oxford Insights updates its Government AI Readiness Index. Arthur I Miller publishes the Artist in the Machine, while Marcus du Sautoy takes a look at The Creativity Code: Art and Innovation in the Age of AI. Lex Fridman and Gary Marcus have a discussion on AI. And Alexa will soon channel the voice of Samuel L Jackson. Click here to visit our website and explore the links mentioned in the episode.

ML and AI with Sherol Chen

Google Cloud Platform Podcast

Play Episode Listen Later Aug 13, 2019 30:04

On the show today, we speak with Developer Advocate and fellow Googler, Sherol Chen about machine learning and AI. Jon Foust and Aja Hammerly learn about the history and impact of AI and ML on technology and gaming. What does it mean to be human? What can machines do better than humans, and what can humans do better than machines? These are the large questions that we aim to solve in order to understand and use AI. Sherol goes on to explain the types of deep learning machines can achieve, from neural networks to decision trees. Sherol also went into depth about the potential social impact of AI as it assists doctors parsing through medical records and plans agricultural endeavors to maximize food production and safety. Sherol also elaborates on the ethical responsibilities we must realize when developing AI projects. For developers looking to build a new AI project, Sherol outlines the pros and cons of using existing tools like Cloud Speech-to-Text, AutoML and AutoML Tables. Sherol Chen Sherol advocates for Machine Learning for Google Cloud, and works in Research at Google Brain for Machine Learning in Music and Creativity for the Magenta team. She’s taught Artificial Intelligence at Stanford and around the world in six different countries. Her PhD work is in Computer Science, researching storytelling and Artificial Intelligence at the Expressive Intelligence Studio. Cool things of the week AMD EPYC processors come to Google—and to Google Cloud blog Kaggle Petfinder Dataset site Streaming data from Cloud Storage into BigQuery using Cloud Functions blog App Engine Standard Ruby site Thagomizer blog Interview AutoML Tables site AutoML Tables Promo Video video Can Machines Think? article AI Impact Challenge site NeurIPS site ICLR site ICML site Machine Learning Crash Course site TensorFlow site Project Magenta site Cloud Speech-to-Text site Cloud AutoML site Sherol’s Blog blog Question of the week You mentioned that you can run App Engine + Rails, how do you handle migrations? Where can you find us next? Jon will be at PAX Dev and PAX West, the internal game summit at Google in Sunnyvale, and taking some personal time to travel to Montreal. Aja will be hanging around at home, on the internet, and at Seattle.rb. Sound Effect Attribution “Coins 1.wav” by ProjectsU012 of Freesound.org “Wedding Bells.wav” by Maurice_J_K of Freesound.org “Small Group Laugh.wav” by Tim.Kahn of Freesound.org

music google ai interview research seattle creativity blog artificial intelligence streaming stanford montreal coins machine learning computer science chen ml kahn deep learning google cloud freesound aja magenta googlers wedding bells pax west developer advocate sunnyvale tensorflow her phd cloud storage google brain bigquery automl neurips icml amd epyc iclr maurice j k projectsu012 aja hammerly pax dev cloud automl

We All Live in a Neuro Subroutine (Side B)

AI with AI

Play Episode Listen Later Jun 7, 2019 26:05

Continuing in research topics, Andy and Dave discuss research from MIT that treats image classification adversarial examples not as bugs, but as features – and intentionally mislabeled pictures; the approach adds robustness to vulnerability, and provides evidence that adversarial vulnerability is caused by non-robust features and is not inherently tied to the standard training framework. The Bulletin of the Atomic Scientists releases The Global Competition for AI Dominance in its May 2019 issue. Isaac Godfrie provides a summary of “few shot” learning papers that were presented at ICLR 2019. A research paper shows the interface between machine learning and the physical sciences. A new survey from Alegion and Dimensional Research examines the data issues impacting AI/ML research (for example, 96% of companies surveyed said they ran into problem with data quality). Georgios Mastorakis examines issues that arise from taking a human-like approach to training algorithms. Mohri, Rostamizadeh, and Talwalkar release a graduate-level book on Foundations of Machine Learning through MIT Press. CollegeHumor produces “A Computer Co-Wrote this Sketch,” in which the characters appear to become aware of their situation. And finally, the Genetic and Evolutionary Computation Conference is scheduled for 13-17 July 2019 in Prague, Czech Republic. Click here to visit our website and explore the links mentioned in the episode.

mit foundations machine learning prague czech republic genetic sketch neuro bulletin ai ml side b mit press college humor atomic scientists global competition iclr

Elfnark’s Lottery Ticket

AI with AI

Play Episode Listen Later May 24, 2019 56:11

Andy and Dave take a look at the reintroduction of the “AI in Government Act,” a bill that intents to get more AI technical experts into the US Government. San Francisco bans facial recognition software (but leaves the door open in the future), while Moscow announces plans to weave AI facial recognition into its urban surveillance net. Facebook opens up its data to academic researchers for analysis. DARPA announces the Air Combat Evolution (ACE) program, to automate air-to-air combat; DARPA also announces Teaching AI to Leverage Overlooked Residuals (TAILOR), to make soldiers fitter, happier, and more productive. And IARPA announces Trojans in AI (TrojAI), an effort to inspect AI for malicious code. In research, Andy and Dave discuss research from Frankle at MIT that proposes a “Lottery Ticket” hypothesis, which suggests only certain “winning combinations” are necessary for training a neural networks, and that researchers have been training neural networks that are much larger than they need to be to increase the chances of includes one of these winning combinations. Leon Bottou at Facebook AI proposes a method for using AI to identify causal relationships in data (and which goes against common modern practice of combining data sets into one giant dataset). And research from Cambridge, George IT, and the University of Pennsylvania demonstrates that Magic: the Gathering is officially the world’s most complicated game (and is Turing complete). In reports of the week, the Stockholm International Peace Research Institute releases the Impact of AI on Strategic Stability and Nuclear Risk. IKV and Pax Christi release The State of AI. Analytics Vedhya has compiled a list of 25 open datasets for deep learning. Benedek Rozemberczki has curated a list of decision tree research papers. The IEEE Spectrum releases a report on Accelerating Autonomous Vehicle Technology. The May 2019 issue of The Scientist contains 15 articles on how Biology is tackling AI. David Kriesel provides A Brief Introduction to Neural Networks. COL Jasper Jeffers wins the 2019 Sci-Fi Writing Contest with AN41. The ICLR 2019 provides video on four talks, including Frankle’s Lottery Ticket hypothesis, and Bottou’s Casual Invariance. Melanie Mitchell gives a Ted Talk on the Collapse of AI and the possibility of an AI winter. And the National Academies-Royal Society Public Symposium will be meeting in DC on 24 May for an International Dialogue on AI.

The Deep End of Deep Learning

Talking Machines

Play Episode Listen Later Apr 25, 2019 19:23

In this episode as we prep for ICLR we take a break from our usual format to bring you a talk from Hugo LaRochelle at TedX Boston on Deep Learning.

learning ai deep research artificial intelligence computers intelligence artificial programming computer science networks ml deep learning deep end ai ml iclr tedx boston

Healthcare: 3 Low-Risk Ways To Invest In Biotech

Industry Focus

Play Episode Listen Later Jan 23, 2019 30:20

Investing in the biotech sector is not for the faint of heart. But these 3 investing strategies could help investors generate outsized returns with lower volatility. Stocks: IBB, XBI, IQV, PRAH, SYNH, ICLR, WST, VEEV, RGEN Check out more of our content here: TMF's podcast portal YouTube Twitter Join Our Motley Fool Podcast Facebook Group LinkedIn StockUp, The Motley Fool's weekly email newsletter

investing healthcare invest biotech motley fool low risk wst tmf prah veev iclr xbi

Designing Better Sequence Models with RNNs with Adji Bousso Dieng - TWiML Talk #160

This Week in Machine Learning & Artificial Intelligence (AI) Podcast

Play Episode Listen Later Jul 2, 2018 40:08

In this episode, I'm joined by Adji Bousso Dieng, PhD Student in the Department of Statistics at Columbia University. In this interview, Adji and I discuss two of her recent papers, the first, an accepted paper from this year’s ICML conference titled “Noisin: Unbiased Regularization for Recurrent Neural Networks,” which, as the name implies, presents a new way to regularize RNNs using noise injection. The second paper, an ICLR submission from last year titled “TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency,” debuts an RNN-based language model designed to capture the global semantic meaning relating words in a document via latent topics. We dive into the details behind both of these papers and I learn a ton along the way. For complete show notes, visit twimlai.com/talk/160.

54 - Simulating Action Dynamics with Neural Process Networks, with Antoine Bosselut

Play Episode Listen Later Mar 26, 2018 36:04

ICLR 2018 paper, by Antoine Bosselut, Omer Levy, Ari Holtzman, Corin Ennis, Dieter Fox, and Yejin Choi. This is not your standard NLP task. This work tries to predict which entities change state over the course of a recipe (e.g., ingredients get combined into a batter, so entities merge, and then the batter gets baked, changing location, temperature, and "cookedness"). We talk to Antoine about the work, getting into details about how the data was collected, how the model works, and what some possible future directions are. https://www.semanticscholar.org/paper/Simulating-Action-Dynamics-with-Neural-Process-Bosselut-Levy/dc01c9401d1caab7f5e6d2f1280f5815f6919977

action nlp dynamics networks antoine neural simulating iclr yejin choi

31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

Play Episode Listen Later Oct 6, 2017 11:19

ICLR 2017 paper by Hakan Inan, Khashayar Khosravi, Richard Socher, presented by Waleed. The paper presents some tricks for training better language models. It introduces a modified loss function for language modeling, where producing a word that is similar to the target word is not penalized as much as producing a word that is very different to the target (I've seen this in other places, e.g., image classification, but not in language modeling). They also give theoretical and empirical justification for tying input and output embeddings. https://www.semanticscholar.org/paper/Tying-Word-Vectors-and-Word-Classifiers-A-Loss-Fra-Inan-Khosravi/424aef7340ee618132cc3314669400e23ad910ba

loss language framework modeling tying vectors waleed iclr

26 - Structured Attention Networks, with Yoon Kim