Podcasts about Fastmail

Email hasn't gotten any less complicated since the last time we covered it, but we have tried a few new options for wrangling our ever-increasing number of inboxes. This week we dig into some of our current strategies, with a focus on Will's time using Fastmail, a paid-only service that purports to let you throw out your Gmails and Outlooks and more fully control your email addresses on domains that you own. We also touch on some of the other popular services like Hey and Proton Mail, grouse about Google's tenacious AI features, dig into our latest trip to the electronics flea market a bit, and more. Support the Pod! Contribute to the Tech Pod Patreon and get access to our booming Discord, a monthly bonus episode, your name in the credits, and other great benefits! You can support the show at: https://patreon.com/techpod

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Read the official blog post about the change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Read the official blog post about the change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

On Hands-On Tech, Mikah helps Lars, who is seeking help for a friend who is being flooded with spam emails and is looking for a way to filter out the spam emails! Mikah strongly recommends transitioning away from ISP-provided email addresses, as they often get shuffled between companies and can face service disruptions There are also some options to consider when moving from an old email account, such as setting up auto-responses directing to the new address or configuring your new email account to pull from your old account. Mikah suggests two things to do. The first is to create a Gmail account to leverage its excellent spam filtering capabilities, then configure it to pull mail from the Cox address and connect it to Outlook. The second option is to use local spam filtering software like Mailwasher, which offers Bayesian filtering that learns from user input to improve spam detection over time. Other software options to consider are Spambully, Spamfighter, and clean.io, though some may not work without MX record access. Also, be sure to stick around to the end of the episode for an upcoming change to Hands-On Tech. Send in your questions for Mikah to answer! hot@twit.tv Host: Mikah Sargent Download or subscribe to Hands-On Tech at https://twit.tv/shows/hands-on-tech Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit

As another year winds down, Mikah Sargent and Rosemary Orchard reflect on the most impressive and useful apps that defined their digital experiences in 2024. In this special episode, the hosts dive deep into their personal favorites across various categories. Balatro: A unique card game that even non-card players will love Flighty: The ultimate flight-tracking companion Carrot Weather: Bringing personality & precision to weather forecasting Fastmail: An email app packed with powerful features Croissant: Simplifying cross-platform social media posting Call Sheet: The ultimate companion for TV and movie enthusiasts Plus, a few more! Beyond their top picks, Mikah and Rosemary share insights into how these apps have improved their daily digital lives, offering listeners practical advice on finding and using the best iOS apps of the year. Feedback Chris wrote in asking about a method to easily format email replies with different colors for each response. Shortcuts Corner Jack asked about a CarPlay notification issue with a shortcut automation. His home arrival notification would activate on the screen but had no audio! Whether you're looking to optimize your iOS experience or just curious about the year's best apps, this episode of iOS Today is a must-listen! Hosts: Mikah Sargent and Rosemary Orchard Want access to the video version and exclusive features? Become a member of Club TWiT today! https://twit.tv/clubtwit Club TWiT members can discuss this episode and leave feedback in the Club TWiT Discord. You can also contribute to iOS Today by sending an email to iOSToday@TWiT.tv.

As another year winds down, Mikah Sargent and Rosemary Orchard reflect on the most impressive and useful apps that defined their digital experiences in 2024. In this special episode, the hosts dive deep into their personal favorites across various categories. Balatro: A unique card game that even non-card players will love Flighty: The ultimate flight-tracking companion Carrot Weather: Bringing personality & precision to weather forecasting Fastmail: An email app packed with powerful features Croissant: Simplifying cross-platform social media posting Call Sheet: The ultimate companion for TV and movie enthusiasts Plus, a few more! Beyond their top picks, Mikah and Rosemary share insights into how these apps have improved their daily digital lives, offering listeners practical advice on finding and using the best iOS apps of the year. Feedback Chris wrote in asking about a method to easily format email replies with different colors for each response. Shortcuts Corner Jack asked about a CarPlay notification issue with a shortcut automation. His home arrival notification would activate on the screen but had no audio! Whether you're looking to optimize your iOS experience or just curious about the year's best apps, this episode of iOS Today is a must-listen! Hosts: Mikah Sargent and Rosemary Orchard Want access to the video version and exclusive features? Become a member of Club TWiT today! https://twit.tv/clubtwit Club TWiT members can discuss this episode and leave feedback in the Club TWiT Discord. You can also contribute to iOS Today by sending an email to iOSToday@TWiT.tv.

As another year winds down, Mikah Sargent and Rosemary Orchard reflect on the most impressive and useful apps that defined their digital experiences in 2024. In this special episode, the hosts dive deep into their personal favorites across various categories. Balatro: A unique card game that even non-card players will love Flighty: The ultimate flight-tracking companion Carrot Weather: Bringing personality & precision to weather forecasting Fastmail: An email app packed with powerful features Croissant: Simplifying cross-platform social media posting Call Sheet: The ultimate companion for TV and movie enthusiasts Plus, a few more! Beyond their top picks, Mikah and Rosemary share insights into how these apps have improved their daily digital lives, offering listeners practical advice on finding and using the best iOS apps of the year. Feedback Chris wrote in asking about a method to easily format email replies with different colors for each response. Shortcuts Corner Jack asked about a CarPlay notification issue with a shortcut automation. His home arrival notification would activate on the screen but had no audio! Whether you're looking to optimize your iOS experience or just curious about the year's best apps, this episode of iOS Today is a must-listen! Hosts: Mikah Sargent and Rosemary Orchard Want access to the video version and exclusive features? Become a member of Club TWiT today! https://twit.tv/clubtwit Club TWiT members can discuss this episode and leave feedback in the Club TWiT Discord. You can also contribute to iOS Today by sending an email to iOSToday@TWiT.tv.

It was not Tom Lane's plan to become a computer person. Tom's plan was to be a pinball machine designer. And yet for the last 26 years Tom has been one of the most prolific engineering contributors to Postgres. In this episode of Talking Postgres with Claire Giordano, PostgreSQL luminary Tom Lane walks us through how he got his start as a developer and in Postgres—including his time working on desktop calculators at HP. And how he has code running on Mars (and most of us don't.) During Tom's PhD studies at Carnegie Mellon, nobody told him databases were so interesting! It wasn't until Tom needed a database to store stock trading information that he first got to work with Postgres. And that's when Tom's 26-year-long (and counting) Postgres story began.Links mentioned in this episode:Wikipedia: Tom Lane (computer scientist)Wikipedia: HP 9800 seriesCMU CS Department Coke Machine historyWikipedia: Honeywell 316Wikipedia: Teletype Model 33Wikipedia: Hydra (operating system)Wikipedia: William WulfWikipedia: Jon Bentley (computer scientist)Wikipedia: Mary Shaw (computer scientist)Wikipedia: UsenetGitHub: postgres commit by tglsfdcArticle: The Mars 2020 Engineering Cameras and Microphone on the Perseverance Rover: A Next-Generation Imaging System for Mars Exploration by J.N. Maki et al.Blog: Open Source on Mars: Community powers NASA's Ingenuity Helicopter by Klint FinleyPostgreSQL Mailing List message: pg_upgrade --check fails to warn about abstimePostgreSQL: Core Teampostgresql.git: commitdiffBlog: Proton to Fastmail by Tristan PartinTalking Postgres Ep18: How I got started as a dev (& in Postgres) with David RowleyPGConf EU 2024: Conference SchedulePGConf NYC 2024: Conference ScheduleTalking Postgres Ep19: Becoming a Postgres committer with Melanie PlagemanPostgreSQL: CommitfestsWikipedia: Cutting room floorPostgreSQL Mailing List message: Straight-from-the-horses-mouth deptPostgreSQL Mailing List message: [PATCH] Extend ALTER OPERATOR to support adding commutator, negator, hashes, and merges

WARNING: Jason's audio is very bad. He is very sorry. Who moved Andrew's cheese? How do we feel about Martin making this potential purchase? It's time for everyone's favorite gameshow, Roast My Blog! Do you have a kitchen? Tune in to see how this one weird trick can change your life! Using Apple Podcasts? All notes can always be found here (https://listen.hemisphericviews.com/121)! One Prime Plus 00:00:00 Welcome Ian M!

In this episode, the hosts revisit their past picks from previous episodes. They discuss the picks that they still use and enjoy, as well as the ones that are now defunct. Some notable picks include the Pebble smartwatch, Venmo, Fastmail, Overcast, and the Wirecutter. They also discuss the impact of certain TV shows, movies, and podcasts. The hosts express their appreciation for the picks suggested by each other and the impact they have had on their lives. dontpanic.io Proud member of the Coffee & Beer family of podcasts, streaming at coffeeandbeer.tv.

**Community Discussion:** [Discord](https://discord.gg/T38WpgkHGQ)**Highlighted Topics:**- Discussion on the perils of 'airdrop' investment scams in the cryptocurrency realm- Fastmail's anti-spam feature exploration and privacy violation detection- Overview of Robinhood Gold Card features and comparison with Privacy.com- Examination of the self-driving industry's push from Level 4 to Level 2 and 3, with a focus on various notable partnerships- Exploration of bringing self-driving tech to the heavy equipment industry as a safety measure- Breakdown and analysis of the "Prometheus" movie, its audience responses and box office results- Review of "Dungeons & Dragons", evaluating its creative potential and financial performance- Discussion on the utility of productivity checklists- Evaluation of the 'Second Brain' concept in information management**Frequent Topics:**- Introduction and assessment of Gmail's Gemini feature- Survey of promising cloud technologies including DigitalOcean, Azure, and Google Cloud- Prospects and challenges in AI and hardware startups investment- Analysis of self-driving industry in the Bay Area and Uber's influence- Insights into the construction of CI-CD systems and ensuring GPU availability- Deep-dive into the "Three-Body Problem", its Netflix adaptation and the book-to-screen balance- Review of the "Dune" movie, its visual impressions, parallels with the book and audience feedback- Consideration of the 'Checklist Manifesto' and the impact of to-do checklists in task management- Commentary on 'Obsidian' note-taking software as a productive tool**Books, Movies, and Software:**- "Prometheus" (Movie)- "Dungeons & Dragons" (Movie)- "Dune" (Movie)- "Three-Body Problem" (Book)- "Checklist Manifesto" (Book)- "Fastmail" (Software)- "Privacy.com" (Software)- "Obsidian" (Software)

What's the best way to roll central authentication? What's the best Google replacement suite? This week Noah and Steve dig into hosting questions, as always your calls go to the front of the line! -- During The Show -- 01:36 PHP, Kanban etc - Joe Nextcloud Deck Our Note Organizer (https://github.com/JoeMrCoffee/OurNoteOrganizer) 03:40 NIX Feedback - Alexander People time rank/prioritize "What problem does it solve" is a framework Effective evangelizing Making something "sticky" No bad questions 13:22 Caller Tony from Toronto Central Authentication? FreeIPA (https://www.freeipa.org/page/Main_Page) Samaba4 Distros Zentyal (https://zentyal.com/) 20:48 Grimnir from Mumble Volumio (https://volumio.com/) Locking it down SSH Samaba Home Assistant (https://www.home-assistant.io/) Adding music Separate Volumio from the PI 25:00 Nextcloud? - Craig Nextcloud (https://nextcloud.com/) is challenging on iOS Head Scale (https://headscale.net/) SpiderOak Immich (https://immich.app/) SeaFile (https://www.seafile.com/en/home/) Encrypt locally, then upload to "cloud" Fastmail (https://www.fastmail.com/) 36:20 Vivaldi & Hosting questions - Ben Hugo (https://gohugo.io/) Altispeed Hosting Vivaldi 41:25 Database Questions - Anton Argument against DIY OpenEMR (https://www.open-emr.org/) Open Source No lock in Form editor CPT/ICD10 codes WikiJS (https://js.wiki/) Weasis (https://weasis.org/en/getting-started/download-dicom-viewer/) 47:26 News Wire OSI Election Results - opensource.org (https://opensource.org/blog/results-of-2024-elections-of-osi-board-of-directors) Red Hat Nova - lore.kernel.org (https://lore.kernel.org/dri-devel/Zfsj0_tb-0-tNrJy@cassiopeiae/) Linux 6.9 RC - lkml.iu.edu (https://lkml.iu.edu/hypermail/linux/kernel/2403.3/00300.html) Regata OS 24 - betanews.com (https://betanews.com/2024/03/19/regata-os-24-arctic-fox-linux/) Wine 9.5 - gitlab.winehq.org (https://gitlab.winehq.org/wine/wine/-/releases/wine-9.5) Kafka UI 1.0 - GitHub (https://github.com/kafbat/kafka-ui) Firefox 124 - Mozilla (https://www.mozilla.org/en-US/firefox/124.0/releasenotes/) Gnome 45.5 - Gnome (https://discourse.gnome.org/t/gnome-45-5-released/20043) Gnome 46 - Gnome (https://release.gnome.org/46/) Emacs 29.3 - Gnu.org (https://lists.gnu.org/archive/html/emacs-devel/2024-03/msg00611.html) Cmake 3.29 - Cmake.org (https://cmake.org/cmake/help/latest/release/3.29.html) OpenVPN - OpenVPN (https://openvpn.net/community-downloads/) SysVInit 3.09 - Phoronix (https://www.phoronix.com/news/SysVinit-3.09) Docker 26 - Docker (https://docs.docker.com/engine/release-notes/26.0/) Lemur Pro - System76 (https://blog.system76.com/post/lemur-pro-ultraportable-laptops) Devika - Market Tech Post (https://www.marktechpost.com/2024/03/25/meet-devika-an-open-source-ai-software-engineer-that-aims-to-be-a-competitive-alternative-to-devin-by-cognition-ai/) GitHub (https://github.com/stitionai/devika) Ubuntu LTS 12 Year Support - How To Geek (https://www.howtogeek.com/ubuntu-linux-legacy-support-program/) 49:05 Shufflecake Shufflecake (https://shufflecake.net/) Linux encryption tool Makes hidden volumes Spiritual successor to TrueCrypt and VeriCrypt GPG encryption -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/382) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed)

Desde el 2021 vengo usando una suscripción de Fastmail para tener “mi propio dominio y servidor de correo”, y ya en aquel entonces podía haber montado una alternativa autoalojada, pero no me valía la pena… hoy quizás si. Me pueden contactar en: https://ernestoacosta.me/contacto.html Todos los medios donde publico contenido los encuentras en: https://ernestoacosta.me/

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/ask-the-tech-guys

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/all-twittv-shows

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/total-leo

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/ask-the-tech-guys

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/all-twittv-shows

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/total-mikah

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/total-leo

How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? What are the easiest methods to convert my old VHS tapes into digital files? Plus, Dick DeBartolo shares a cool beanie that you can wear with Bluetooth capabilities, and Chris Marquardt goes over the Unusual photo assignment review!  Seven things we learned analyzing 515 million Wordle. Inside Mark Zuckerberg's top-secret Hawaii compound. Four men indicted in $80 million 'pig butchering' scheme. Is there a difference in quality when I watch Netflix on certain devices? What brand of keyboards would Leo & Mikah recommend that are quiet when using it? How can I convert old home movie tapes to digital? Dick Debartolo and the Bluetooth beanie! At what point is it an issue when connecting to public WiFi? Can I download my purchases from a streaming service like Apple TV+? How can I get emails sent to my Fastmail account to show up in email clients such as Apple's Mail app? Which free two-factor authentication app would Leo & Mikah recommend that uses the "push" functionality and not type in a code? What's a great budget Android phone to upgrade to? Chris Marquardt and the Unusual photo assignment review! What's the best way to prepare myself to do cold-weather photography and capture the Northern lights? What are some recommendations to get children started in computer programming? Hosts: Leo Laporte and Mikah Sargent Guests: Dick DeBartolo and Chris Marquardt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/2005 Download or subscribe to this show at: https://twit.tv/shows/total-mikah

Apple previews Stolen Device Protection in iOS 17.3 beta, GM's Apple CarPlay dropout, and we unravel the Beeper Mini situation which promised iMessage on Android.Contact our hosts@hillithreads on Threads@Hillitech on TwitterWes on Mastodon@andrew_osu on Twitter@andrewohara941 on ThreadsSponsored by:Notion: Try out the incredible power of Notion AI today! For a limited time, try Notion AI for free when you visit: notion.com/appleinsiderFastmail: To learn more about Fastmail and receive 10% off your first year, visit: fastmail.com/appleinsiderLinks from the showStolen Device Protection to thwart iPhone and passcode thievesGM ditching Apple CarPlay is about money, not safetyBeeper Mini is down for 5% of users, but there's a fixBeeper Mini is down for 5% of users, but there's a fixUpgrade your AirPods Pro with a $99 USB-C charging caseHow to film Spatial Video for Apple Vision Pro on iPhone 15 ProApple News and Rumors since 1997Support the showSupport the show on Patreon or Apple Podcasts to get ad-free episodes every week, access to our private Discord channel, and early release of the show! We would also appreciate a 5-star rating and review in Apple PodcastsMore AppleInsider podcastsTune in to our HomeKit Insider podcast covering the latest news, products, apps and everything HomeKit related. Subscribe in Apple Podcasts, Overcast, or just search for HomeKit Insider wherever you get your podcasts.Subscribe and listen to our AppleInsider Daily podcast for the latest Apple news Monday through Friday. You can find it on Apple Podcasts, Overcast, or anywhere you listen to podcasts.Podcast artwork from Basic Apple Guy. Download the free wallpaper pack here. (00:00) - Intro (00:59) - Journal App (04:36) - iOS 17.3 Beta (26:23) - Matter Updates (30:15) - Sponsor: Fastmail (31:33) - Sponsor: Notion (33:10) - GM + CarPlay (45:13) - Beeper Mini (52:38) - AirPods Pro USB-C Case (57:11) - Apple Vision Pro ★ Support this podcast on Patreon ★

Amidst our live streaming conversation, we touch upon the topic of incorporating video into our podcast. The potential of expanding our presence on platforms like YouTube holds great appeal, and we explore the opportunities it presents for reaching a wider audience. Furthermore, this episode is sponsored by Shopify and Fastmail, […] The post The Wisdom of Charlie Munger – Business Brain 506 appeared first on Business Brain - The Entrepreneurs' Podcast.

First glimpse at Spatial Video captured on iPhone 15 Pro and viewable on Apple Vision Pro, MacBook Pro 16-inch M3 Max review, Nothing phone hacks iMessage onto Android, and welcome updates to watchOS 10.2 and iOS 17.2.Contact our hosts@stephenrobles on Threads@stephenrobles on TwitterStephen on Mastodon@hillithreads on Threads@Hillitech on TwitterWes on MastodonSponsored by:Notion: Try out the incredible power of Notion AI today! For a limited time, try Notion AI for free when you visit: notion.com/appleinsiderFastmail: To learn more about Fastmail and receive 10% off your first year, visit: fastmail.com/appleinsiderMasterClass: Buy one MasterClass annual membership and get one FREE when you visit: masterclass.com/appleinsiderLinks from the showApple Music Classical finally comes to the iPadApple explains the GPU enhancements in A17 Pro and M3MacBook Pro 16-inch M3 Max review: Performance, specs, speedFirst look at Vision Pro spatial video in new Apple demosApple seeds third iOS 17.2, iPadOS 17.2 developer betasNothing company brings iMessage blue bubbles to AndroidM3 Ultra could have up to 80 GPU coresEarly 2024 launch of 12.9-inch iPad rumored by KuoControversial Humane Ai Pin is here, costs $700Support the showSupport the show on Patreon or Apple Podcasts to get ad-free episodes every week, access to our private Discord channel, and early release of the show! We would also appreciate a 5-star rating and review in Apple PodcastsMore AppleInsider podcastsTune in to our HomeKit Insider podcast covering the latest news, products, apps and everything HomeKit related. Subscribe in Apple Podcasts, Overcast, or just search for HomeKit Insider wherever you get your podcasts.Subscribe and listen to our AppleInsider Daily podcast for the latest Apple news Monday through Friday. You can find it on Apple Podcasts, Overcast, or anywhere you listen to podcasts.Podcast artwork from Basic Apple Guy. Download the free wallpaper pack here.Those interested in sponsoring the show can reach out to us at: steve@appleinsider.com (00:00) - Intro (01:41) - Apple Music Classical on iPad (02:25) - Stephen's M3 Pro (07:43) - Dynamic Caching (11:04) - Spatial Video on iPhone (21:43) - Sponsor: Fastmail (23:37) - Sponsor: MasterClass (25:32) - iOS 17.2 watchOS 10.2 (34:13) - Nothing iMessage (40:54) - Sponsor: Notion (42:55) - M3 Ultra (44:47) - 12.9" iPad Air (47:58) - Humane Pin ★ Support this podcast on Patreon ★

- Analysis and insights: 'Rick and Morty' season.- Fitness-centric TV shows: 'Fast and Furious', 'Superman', 'Rick and Morty'.- Movie discussion: 'The Covenant', 'The Morning Show' Season 3, 'Foundation' series, 'Five Nights at Freddy's' movie trailer.- Video Game ecosystem: Player switch rationale within 'Finals', 'Apex', 'Fortnite', 'Call of Duty', 'Contra Strike'.- Discord join invitation for listeners.- Literary sessions: Isaac Asimov's book review and relation to Loki TV series.- Game launches: 'Alan Wake 2', alternatives like 'RoboCop: Rogue City', first impressions of 'Spider-Man 2' for PlayStation, PlayStation 5 Slim Edition.- Professional gaming culture update - gaming trends and streaming updates.- Electric Vehicle News - 'Cruise', 'Waymo', Ford F-150, Tesla Model 3.- Tech Talk - User experience of 'One Password', 'Bitwarden', 'FastMail'.- Personal Segment: Hiking trip experience in Boulder, Colorado.

Apple launched a new Pencil with USB-C, but less features than older models, your hosts analyze the current iPad lineup, discuss the Rewind Pendant that records all your (consensual) conversations, plus a very special guest for episode 500!Contact our hosts@stephenrobles on Threads@stephenrobles on TwitterStephen on Mastodon@williamgallagher_ on Threads@WGallagher on TwitterWilliam on Mastodon@thisisneilNeil on MastodonSponsored by:Fastmail: To learn more about Fastmail and receive 10% off your first year, visit: fastmail.com/appleinsiderLinks from the showApple Pencil USB-C vs Gen 1 and Gen 2: specs, price, featuresAnother leaker asserts impending iPad updateApple seeds iOS 17.1, iPadOS 17.1 release candidatesApple Stores could update iPhones without opening boxesApple's Thunderbolt 4 Pro cable design revealed under CT scanAirTag 2 rumored to arrive in early 2025TimePorter Watch Band Wall MountGitHub -Live Wallpaper Download Script for macOS Sonoma (and later)Rewind PendantNew Apple research highlights the health benefits of pickleball - AppleSupport the showSupport the show on Patreon or Apple Podcasts to get ad-free episodes every week, access to our private Discord channel, and early release of the show! We would also appreciate a 5-star rating and review in Apple PodcastsMore AppleInsider podcastsTune in to our HomeKit Insider podcast covering the latest news, products, apps and everything HomeKit related. Subscribe in Apple Podcasts, Overcast, or just search for HomeKit Insider wherever you get your podcasts.Subscribe and listen to our AppleInsider Daily podcast for the latest Apple news Monday through Friday. You can find it on Apple Podcasts, Overcast, or anywhere you listen to podcasts.Podcast artwork from Basic Apple Guy. Download the free wallpaper pack here. (00:00) - Intro (05:35) - Pencil + iPads (17:34) - iOS 17.1 (19:46) - iPhone Updates in Box (20:41) - NFC Tags (25:40) - Thunderbolt 4 CT Scan (28:44) - AirTag 2 (32:56) - Sponsor: Fastmail (34:40) - iPhone 15 Filming (43:57) - iOS 17 Ringtones (45:24) - Action Button (49:14) - FineWoven (51:26) - Sonoma Wallpapers (52:43) - Past and Future Rumors (57:22) - Apple Wearables (58:05) - Rewind.ai (01:06:31) - Pickleball ★ Support this podcast on Patreon ★

Thanks to the over 17,000 people who have joined the first AI Engineer Summit! A full recap is coming. Last call to fill out the State of AI Engineering survey! See our Community page for upcoming meetups in SF, Paris and NYC.This episode had good interest on Twitter.Fast.ai's “Practical Deep Learning” courses been watched by over >6,000,000 people, and the fastai library has over 25,000 stars on Github. Jeremy Howard, one of the creators of Fast, is now one of the most prominent and respected voices in the machine learning industry; but that wasn't always the case. Being non-consensus and right In 2018, Jeremy and Sebastian Ruder published a paper on ULMFiT (Universal Language Model Fine-tuning), a 3-step transfer learning technique for NLP tasks: The paper demonstrated that pre-trained language models could be fine-tuned on a specific task with a relatively small amount of data to achieve state-of-the-art results. They trained a 24M parameters model on WikiText-103 which was beat most benchmarks.While the paper had great results, the methods behind weren't taken seriously by the community: “Everybody hated fine tuning. Everybody hated transfer learning. I literally did tours trying to get people to start doing transfer learning and nobody was interested, particularly after GPT showed such good results with zero shot and few shot learning […] which I was convinced was not the right direction, but who's going to listen to me, cause as you said, I don't have a PhD, not at a university… I don't have a big set of computers to fine tune huge transformer models.”Five years later, fine-tuning is at the center of most major discussion topics in AI (we covered some like fine tuning vs RAG and small models fine tuning), and we might have gotten here earlier if Jeremy had OpenAI-level access to compute and distribution. At heart, Jeremy has always been “GPU poor”:“I've always been somebody who does not want to build stuff on lots of big computers because most people don't have lots of big computers and I hate creating stuff that most people can't use.”This story is a good reminder of how some of the best ideas are hiding in plain sight; we recently covered RWKV and will continue to highlight the most interesting research that isn't being done in the large labs. Replacing fine-tuning with continued pre-trainingEven though fine-tuning is now mainstream, we still have a lot to learn. The issue of “catastrophic forgetting” and potential solutions have been brought up in many papers: at the fine-tuning stage, the model can forget tasks it previously knew how to solve in favor of new ones. The other issue is apparent memorization of the dataset even after a single epoch, which Jeremy covered Can LLMs learn from a single example? but we still don't have the answer to. Despite being the creator of ULMFiT, Jeremy still professes that there are a lot of open questions on finetuning:“So I still don't know how to fine tune language models properly and I haven't found anybody who feels like they do.”He now advocates for "continued pre-training" - maintaining a diversity of data throughout the training process rather than separate pre-training and fine-tuning stages. Mixing instructional data, exercises, code, and other modalities while gradually curating higher quality data can avoid catastrophic forgetting and lead to more robust capabilities (something we covered in Datasets 101).“Even though I originally created three-step approach that everybody now does, my view is it's actually wrong and we shouldn't use it… the right way to do this is to fine-tune language models, is to actually throw away the idea of fine-tuning. There's no such thing. There's only continued pre-training. And pre-training is something where from the very start, you try to include all the kinds of data that you care about, all the kinds of problems that you care about, instructions, exercises, code, general purpose document completion, whatever. And then as you train, you gradually curate that, you know, you gradually make that higher and higher quality and more and more specific to the kinds of tasks you want it to do. But you never throw away any data….So yeah, that's now my view, is I think ULMFiT is the wrong approach. And that's why we're seeing a lot of these so-called alignment tax… I think it's actually because people are training them wrong.An example of this phenomena is CodeLlama, a LLaMA2 model finetuned on 500B tokens of code: while the model is much better at code, it's worse on generic tasks that LLaMA2 knew how to solve well before the fine-tuning. In the episode we also dive into all the places where open source model development and research is happening (academia vs Discords - tracked on our Communities list and on our survey), and how Jeremy recommends getting the most out of these diffuse, pseudonymous communities (similar to the Eleuther AI Mafia).Show Notes* Jeremy's Background* FastMail* Optimal Decisions* Kaggle* Enlitic* fast.ai* Rachel Thomas* Practical Deep Learning* fastai for PyTorch* nbdev* fastec2 (the underrated library we describe)* Can LLMs learn from a single example?* the Kaggle LLM Science Exam competition, which “challenges participants to answer difficult science-based questions written by a Large Language Model”.* Sebastian Ruder* Alec Radford* Sylvain Gugger* Stephen Merity* Chris Lattner* Modular.ai / Mojo* Jono Whittaker* Zeiler and Fergus paper* ULM Fit* DAWNBench* Phi-1* Code Llama* AlexNetTimestamps* [00:00:00] Intros and Jeremy's background* [00:05:28] Creating ULM Fit - a breakthrough in NLP using transfer learning* [00:06:32] The rise of GPT and the appeal of few-shot learning over fine-tuning* [00:10:00] Starting Fast.ai to distribute AI capabilities beyond elite academics* [00:14:30] How modern LMs like ChatGPT still follow the ULM Fit 3-step approach* [00:17:23] Meeting with Chris Lattner on Swift for TensorFlow at Google* [00:20:00] Continued pre-training as a fine-tuning alternative* [00:22:16] Fast.ai and looking for impact vs profit maximization* [00:26:39] Using Fast.ai to create an "army" of AI experts to improve their domains* [00:29:32] Fast.ai's 3 focus areas - research, software, and courses* [00:38:42] Fine-tuning memorization and training curve "clunks" before each epoch* [00:46:47] Poor training and fine-tuning practices may be causing alignment failures* [00:48:38] Academia vs Discords* [00:53:41] Jeremy's high hopes for Chris Lattner's Mojo and its potential* [01:05:00] Adding capabilities like SQL generation through quick fine-tuning* [01:10:12] Rethinking Fast.ai courses for the AI-assisted coding era* [01:14:53] Rapid model development has created major technical debt* [01:17:08] Lightning RoundAI Summary (beta)This is the first episode we're trying this. Here's an overview of the main topics before you dive in the transcript. * Jeremy's background and philosophies on AI* Studied philosophy and cognitive science in college* Focused on ethics and thinking about AI even 30 years ago* Believes AI should be accessible to more people, not just elite academics/programmers* Created fast.ai to make deep learning more accessible* Development of transfer learning and ULMFit* Idea of transfer learning critical for making deep learning accessible* ULMFit pioneered transfer learning for NLP* Proposed training general language models on large corpora then fine-tuning - this became standard practice* Faced skepticism that this approach would work from NLP community* Showed state-of-the-art results on text classification soon after trying it* Current open questions around fine-tuning LLMs* Models appear to memorize training data extremely quickly (after 1 epoch)* This may hurt training dynamics and cause catastrophic forgetting* Unclear how best to fine-tune models to incorporate new information/capabilities* Need more research on model training dynamics and ideal data mixing* Exciting new developments* Mojo and new programming languages like Swift could enable faster model innovation* Still lots of room for improvements in computer vision-like innovations in transformers* Small models with fine-tuning may be surprisingly capable for many real-world tasks* Prompting strategies enable models like GPT-3 to achieve new skills like playing chess at superhuman levels* LLMs are like computer vision in 2013 - on the cusp of huge new breakthroughs in capabilities* Access to AI research* Many key convos happen in private Discord channels and forums* Becoming part of these communities can provide great learning opportunities* Being willing to do real work, not just talk about ideas, is key to gaining access* The future of practical AI* Coding becoming more accessible to non-programmers through AI assistance* Pre-requisite programming experience for learning AI may no longer be needed* Huge open questions remain about how to best train, fine-tune, and prompt LLMsTranscriptAlessio: Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol AI. [00:00:21]Swyx: Hey, and today we have in the remote studio, Jeremy Howard all the way from Australia. Good morning. [00:00:27]Jeremy: The remote studio, also known as my house. Good morning. Nice to see you. [00:00:32]Swyx: Nice to see you too. I'm actually very used to seeing you in your mask as a message to people, but today we're mostly audio. But thank you for doing the very important public service of COVID awareness. It was a pleasure. [00:00:46]Jeremy: It was all very annoying and frustrating and tedious, but somebody had to do it. [00:00:52]Swyx: Somebody had to do it, especially somebody with your profile. I think it really drives home the message. So we tend to introduce people for them and then ask people to fill in the blanks on the personal side. Something I did not know about you was that you graduated with a BA in philosophy from the University of Melbourne. I assumed you had a PhD. [00:01:14]Jeremy: No, I mean, I barely got through my BA because I was working 80 to 100 hour weeks at McKinsey and Company from 19 years old onwards. So I actually didn't attend any lectures in second and third year university. [00:01:35]Swyx: Well, I guess you didn't need it or you're very sort of self-driven and self-motivated. [00:01:39]Jeremy: I took two weeks off before each exam period when I was working at McKinsey. And then, I mean, I can't believe I got away with this in hindsight, I would go to all my professors and say, oh, I was meant to be in your class this semester and I didn't quite turn up. Were there any assignments I was meant to have done, whatever. I can't believe all of them let me basically have it. They basically always would say like, okay, well, if you can have this written by tomorrow, I'll accept it. So yeah, stressful way to get through university, but. [00:02:12]Swyx: Well, it shows that, I guess, you min-maxed the opportunities. That definitely was a precursor. [00:02:18]Jeremy: I mean, funnily, like in as much as I, you know, in philosophy, the things I found interesting and focused on in the little bit of time I did spend on it was ethics and cognitive science. And it's kind of really amazing that it's now come back around and those are actually genuinely useful things to know about, which I never thought would happen. [00:02:38]Swyx: A lot of, yeah, a lot of relevant conversations there. So you were a consultant for a while and then in the magical month of June 1989, you founded both Optimal Decisions and Fastmeal, which I also briefly used. So thank you for that. [00:02:53]Jeremy: Oh, good for you. Yeah. Cause I had read the statistics, which is that like 90% or something of small businesses fail. So I thought if I start two businesses, I have a higher chance. In hindsight, I was thinking of it as some kind of stochastic thing I didn't have control over, but it's a bit odd, but anyway. [00:03:10]Swyx: And then you were president and chief scientist at Kaggle, which obviously is the sort of composition platform of machine learning. And then Enlitic, where you were working on using deep learning to improve medical diagnostics and clinical decisions. Yeah. [00:03:28]Jeremy: I was actually the first company to use deep learning in medicine, so I kind of founded the field. [00:03:33]Swyx: And even now that's still like a pretty early phase. And I actually heard you on your new podcast with Tanish, where you went very, very deep into the stuff, the kind of work that he's doing, such a young prodigy at his age. [00:03:47]Jeremy: Maybe he's too old to be called a prodigy now, ex-prodigy. No, no. [00:03:51]Swyx: I think he still counts. And anyway, just to round out the bio, you have a lot more other credentials, obviously, but most recently you started Fast.ai, which is still, I guess, your primary identity with Rachel Thomas. So welcome. [00:04:05]Jeremy: Yep. [00:04:06]Swyx: Thanks to my wife. Thank you. Yeah. Doing a lot of public service there with getting people involved in AI, and I can't imagine a better way to describe it than fast, fast.ai. You teach people from nothing to stable diffusion in seven weeks or something, and that's amazing. Yeah, yeah. [00:04:22]Jeremy: I mean, it's funny, you know, when we started that, what was that, like 2016 or something, the idea that deep learning was something that you could make more accessible was generally considered stupid. Everybody knew that deep learning was a thing that you got a math or a computer science PhD, you know, there was one of five labs that could give you the appropriate skills and that you would join, yeah, basically from one of those labs, you might be able to write some papers. So yeah, the idea that normal people could use that technology to do good work was considered kind of ridiculous when we started it. And we weren't sure if it was possible either, but we kind of felt like we had to give it a go because the alternative was we were pretty sure that deep learning was on its way to becoming, you know, the most or one of the most, you know, important technologies in human history. And if the only people that could use it were a handful of computer science PhDs, that seemed like A, a big waste and B, kind of dangerous. [00:05:28]Swyx: Yeah. [00:05:29]Alessio: And, you know, well, I just wanted to know one thing on your bio that at Kaggle, you were also the top rank participant in both 2010 and 2011. So sometimes you see a lot of founders running companies that are not really in touch with the problem, but you were clearly building something that you knew a lot about, which is awesome. Talking about deep learning, you created, published a paper on ULM fit, which was kind of the predecessor to multitask learning and a lot of the groundwork that then went to into Transformers. I've read back on the paper and you turned this model, AWD LSTM, which I did the math and it was like 24 to 33 million parameters, depending on what training data set you use today. That's kind of like not even small, it's like super small. What were some of the kind of like contrarian takes that you had at the time and maybe set the stage a little bit for the rest of the audience on what was kind of like the state of the art, so to speak, at the time and what people were working towards? [00:06:32]Jeremy: Yeah, the whole thing was a contrarian take, you know. So okay, so we started Fast.ai, my wife and I, and we thought, yeah, so we're trying to think, okay, how do we make it more accessible? So when we started thinking about it, it was probably 2015 and then 2016, we started doing something about it. Why is it inaccessible? Okay, well, A, no one knows how to do it other than a few number of people. And then when we asked those few number of people, well, how do you actually get good results? They would say like, oh, it's like, you know, a box of tricks that aren't published. So you have to join one of the labs and learn the tricks. So a bunch of unpublished tricks, not much software around, but thankfully there was Theano and rappers and particularly Lasagna, the rapper, but yeah, not much software around, not much in the way of data sets, you know, very hard to get started in terms of the compute. Like how do you get that set up? So yeah, no, everything was kind of inaccessible. And you know, as we started looking into it, we had a key insight, which was like, you know what, most of the compute and data for image recognition, for example, we don't need to do it. You know, there's this thing which nobody knows about, nobody talks about called transfer learning, where you take somebody else's model, where they already figured out like how to detect edges and gradients and corners and text and whatever else, and then you can fine tune it to do the thing you want to do. And we thought that's the key. That's the key to becoming more accessible in terms of compute and data requirements. So when we started Fast.ai, we focused from day one on transfer learning. Lesson one, in fact, was transfer learning, literally lesson one, something not normally even mentioned in, I mean, there wasn't much in the way of courses, you know, the courses out there were PhD programs that had happened to have recorded their lessons and they would rarely mention it at all. We wanted to show how to do four things that seemed really useful. You know, work with vision, work with tables of data, work with kind of recommendation systems and collaborative filtering and work with text, because we felt like those four kind of modalities covered a lot of the stuff that, you know, are useful in real life. And no one was doing anything much useful with text. Everybody was talking about word2vec, you know, like king plus queen minus woman and blah, blah, blah. It was like cool experiments, but nobody's doing anything like useful with it. NLP was all like lemmatization and stop words and topic models and bigrams and SPMs. And it was really academic and not practical. But I mean, to be honest, I've been thinking about this crazy idea for nearly 30 years since I had done cognitive science at university, where we talked a lot about the CELS Chinese room experiment. This idea of like, what if there was somebody that could kind of like, knew all of the symbolic manipulations required to answer questions in Chinese, but they didn't speak Chinese and they were kind of inside a room with no other way to talk to the outside world other than taking in slips of paper with Chinese written on them and then they do all their rules and then they pass back a piece of paper with Chinese back. And this room with a person in is actually fantastically good at answering any question you give them written in Chinese. You know, do they understand Chinese? And is this, you know, something that's intelligently working with Chinese? Ever since that time, I'd say the most thought, to me, the most thoughtful and compelling philosophical response is yes. You know, intuitively it feels like no, because that's just because we can't imagine such a large kind of system. But you know, if it looks like a duck and acts like a duck, it's a duck, you know, or to all intents and purposes. And so I always kind of thought, you know, so this is basically a kind of analysis of the limits of text. And I kind of felt like, yeah, if something could ingest enough text and could use the patterns it saw to then generate text in response to text, it could appear to be intelligent, you know. And whether that means it is intelligent or not is a different discussion and not one I find very interesting. Yeah. And then when I came across neural nets when I was about 20, you know, what I learned about the universal approximation theorem and stuff, and I started thinking like, oh, I wonder if like a neural net could ever get big enough and take in enough data to be a Chinese room experiment. You know, with that background and this kind of like interest in transfer learning, you know, I'd been thinking about this thing for kind of 30 years and I thought like, oh, I wonder if we're there yet, you know, because we have a lot of text. Like I can literally download Wikipedia, which is a lot of text. And I thought, you know, how would something learn to kind of answer questions or, you know, respond to text? And I thought, well, what if we used a language model? So language models are already a thing, you know, they were not a popular or well-known thing, but they were a thing. But language models exist to this idea that you could train a model to fill in the gaps. Or actually in those days it wasn't fill in the gaps, it was finish a string. And in fact, Andrej Karpathy did his fantastic RNN demonstration from this at a similar time where he showed like you can have it ingest Shakespeare and it will generate something that looks a bit like Shakespeare. I thought, okay, so if I do this at a much bigger scale, using all of Wikipedia, what would it need to be able to do to finish a sentence in Wikipedia effectively, to do it quite accurately quite often? I thought, geez, it would actually have to know a lot about the world, you know, it'd have to know that there is a world and that there are objects and that objects relate to each other through time and cause each other to react in ways and that causes proceed effects and that, you know, when there are animals and there are people and that people can be in certain positions during certain timeframes and then you could, you know, all that together, you can then finish a sentence like this was signed into law in 2016 by US President X and it would fill in the gap, you know. So that's why I tried to create what in those days was considered a big language model trained on the entirety on Wikipedia, which is that was, you know, a bit unheard of. And my interest was not in, you know, just having a language model. My interest was in like, what latent capabilities would such a system have that would allow it to finish those kind of sentences? Because I was pretty sure, based on our work with transfer learning and vision, that I could then suck out those latent capabilities by transfer learning, you know, by fine-tuning it on a task data set or whatever. So we generated this three-step system. So step one was train a language model on a big corpus. Step two was fine-tune a language model on a more curated corpus. And step three was further fine-tune that model on a task. And of course, that's what everybody still does today, right? That's what ChatGPT is. And so the first time I tried it within hours, I had a new state-of-the-art academic result on IMDB. And I was like, holy s**t, it does work. And so you asked, to what degree was this kind of like pushing against the established wisdom? You know, every way. Like the reason it took me so long to try it was because I asked all my friends in NLP if this could work. And everybody said, no, it definitely won't work. It wasn't like, oh, maybe. Everybody was like, it definitely won't work. NLP is much more complicated than vision. Language is a much more vastly complicated domain. You know, and you've got problems like the grounding problem. We know from like philosophy and theory of mind that it's actually impossible for it to work. So yeah, so don't waste your time. [00:15:10]Alessio: Jeremy, had people not tried because it was like too complicated to actually get the data and like set up the training? Or like, were people just lazy and kind of like, hey, this is just not going to work? [00:15:20]Jeremy: No, everybody wasn't lazy. So like, so the person I thought at that time who, you know, there were two people I thought at that time, actually, who were the strongest at language models were Stephen Merity and Alec Radford. And at the time I didn't know Alec, but I, after we had both, after I'd released ULM Fit and he had released GPT, I organized a chat for both of us with Kate Metz in the New York Times. And Kate Metz answered, sorry, and Alec answered this question for Kate. And Kate was like, so how did, you know, GPT come about? And he said, well, I was pretty sure that pre-training on a general large corpus wouldn't work. So I hadn't tried it. And then I read ULM Fit and turns out it did work. And so I did it, you know, bigger and it worked even better. And similar with, with Stephen, you know, I asked Stephen Merity, like, why don't we just find, you know, take your AWD-ASTLM and like train it on all of Wikipedia and fine tune it? And he's kind of like, well, I don't think that's going to really lie. Like two years before I did a very popular talk at KDD, the conference where everybody in NLP was in the audience. I recognized half the faces, you know, and I told them all this, I'm sure transfer learning is the key. I'm sure ImageNet, you know, is going to be an NLP thing as well. And, you know, everybody was interested and people asked me questions afterwards and, but not just, yeah, nobody followed up because everybody knew that it didn't work. I mean, even like, so we were scooped a little bit by Dai and Lee, Kwok Lee at Google. They had, they had, I already, I didn't even realize this, which is a bit embarrassing. They had already done a large language model and fine tuned it. But again, they didn't create a general purpose, large language model on a general purpose corpus. They only ever tested a domain specific corpus. And I haven't spoken to Kwok actually about that, but I assume that the reason was the same. It probably just didn't occur to them that the general approach could work. So maybe it was that kind of 30 years of mulling over the, the cell Chinese room experiment that had convinced me that it probably would work. I don't know. Yeah. [00:17:48]Alessio: Interesting. I just dug up Alec announcement tweet from 2018. He said, inspired by Cobe, Elmo, and Yola, I'm fit. We should have a single transformer language model can be fine tuned to a wide variety. It's interesting because, you know, today people think of AI as the leader, kind of kind of like the research lab pushing forward the field. What was that at the time? You know, like kind of like going back five years, people think of it as an overnight success, but obviously it took a while. [00:18:16]Swyx: Yeah. Yeah. [00:18:17]Jeremy: No, I mean, absolutely. And I'll say like, you know, it's interesting that it mentioned Elmo because in some ways that was kind of diametrically opposed to, to ULM fit. You know, there was these kind of like, so there was a lot of, there was a lot of activity at the same time as ULM fits released. So there was, um, so before it, as Brian McCann, I think at Salesforce had come out with this neat model that did a kind of multitask learning, but again, they didn't create a general fine tune language model first. There was Elmo, um, which I think was a lip, you know, actually quite a few months after the first ULM fit example, I think. Um, but yeah, there was a bit of this stuff going on. And the problem was everybody was doing, and particularly after GPT came out, then everybody wanted to focus on zero shot and few shot learning. You know, everybody hated fine tuning. Everybody hated transfer learning. And like, I literally did tours trying to get people to start doing transfer learning and people, you know, nobody was interested, particularly after GPT showed such good results with zero shot and few shot learning. And so I actually feel like we kind of went backwards for years and, and not to be honest, I mean, I'm a bit sad about this now, but I kind of got so disappointed and dissuaded by like, it felt like these bigger lab, much bigger labs, you know, like fast AI had only ever been just me and Rachel were getting all of this attention for an approach I thought was the wrong way to do it. You know, I was convinced was the wrong way to do it. And so, yeah, for years people were really focused on getting better at zero shot and few shots and it wasn't until, you know, this key idea of like, well, let's take the ULM fit approach, but for step two, rather than fine tuning on a kind of a domain corpus, let's fine tune on an instruction corpus. And then in step three, rather than fine tuning on a reasonably specific task classification, let's fine tune on a, on a RLHF task classification. And so that was really, that was really key, you know, so I was kind of like out of the NLP field for a few years there because yeah, it just felt like, I don't know, pushing uphill against this vast tide, which I was convinced was not the right direction, but who's going to listen to me, you know, cause I, as you said, I don't have a PhD, not at a university, or at least I wasn't then. I don't have a big set of computers to fine tune huge transformer models. So yeah, it was definitely difficult. It's always been hard. You know, it's always been hard. Like I've always been somebody who does not want to build stuff on lots of big computers because most people don't have lots of big computers and I hate creating stuff that most people can't use, you know, and also stuff that's created on lots of big computers has always been like much more media friendly. So like, it might seem like a recent thing, but actually throughout my 30 years in data science, the attention's always been on, you know, the big iron results. So when I first started, everybody was talking about data warehouses and it was all about Teradata and it'd be like, oh, this big bank has this huge room full of computers and they have like terabytes of data available, you know, at the press of a button. And yeah, that's always what people want to talk about, what people want to write about. And then of course, students coming out of their PhDs and stuff, that's where they want to go work because that's where they read about. And to me, it's a huge distraction, you know, because like I say, most people don't have unlimited compute and I want to help most people, not the small subset of the most well-off people. [00:22:16]Alessio: That's awesome. And it's great to hear, you do such a great job educating that a lot of times you're not telling your own story, you know? So I love this conversation. And the other thing before we jump into Fast.AI, actually, a lot of people that I know, they run across a new architecture and whatnot, they're like, I got to start a company and raise a bunch of money and do all of this stuff. And say, you were like, I want everybody to have access to this. Why was that the case for you? Was it because you already had a successful venture in like FastMail and you were more interested in that? What was the reasoning? [00:22:52]Jeremy: It's a really good question. So I guess the answer is yes, that's the reason why. So when I was a teenager, I thought it would be really cool to like have my own company. You know, I didn't know the word startup. I didn't know the word entrepreneur. I didn't know the word VC. And I didn't really know what any of those things were really until after we started Kaggle, to be honest. Even the way it started to what we now call startups. I just thought they were just small businesses. You know, they were just companies. So yeah, so those two companies were FastMail and Optimal Decisions. FastMail was the first kind of synchronized email provider for non-businesses. So something you can get your same email at home, on your laptop, at work, on your phone, whatever. And then Optimal Decisions invented a new approach to insurance pricing. Something called profit-optimized insurance pricing. So I saw both of those companies, you know, after 10 years. And at that point, I had achieved the thing that as a teenager I had wanted to do. You know, it took a lot longer than it should have because I spent way longer in management consulting than I should have because I got caught up in that stupid rat race. But, you know, eventually I got there and I remember my mom saying to me, you must be so proud. You know, because she remembered my dream. She's like, you've done it. And I kind of reflected and I was like, I'm not proud at all. You know, like people quite liked FastMail. You know, it's quite nice to have synchronized email. It probably would have happened anyway. Yeah, I'm certainly not proud that I've helped some insurance companies suck more money out of their customers. Yeah, no, I'm not proud. You know, it's actually, I haven't really helped the world very much. You know, maybe in the insurance case I've made it a little bit worse. I don't know. So, yeah, I was determined to not waste more years of my life doing things, working hard to do things which I could not be reasonably sure would have a lot of value. So, you know, I took some time off. I wasn't sure if I'd ever work again, actually. I didn't particularly want to, because it felt like, yeah, it felt like such a disappointment. And, but, you know, and I didn't need to. I had enough money. Like, I wasn't super rich, but I had enough money. I didn't need to work. And I certainly recognized that amongst the other people I knew who had enough money that they didn't need to work, they all worked ridiculously hard, you know, and constantly put themselves in extremely stressful situations. And I thought, I don't want to be one of those idiots who's tied to, you know, buying a bigger plane than the next guy or whatever. You know, Kaggle came along and I mainly kind of did that just because it was fun and interesting to hang out with interesting people. But, you know, with Fast.ai in particular, you know, Rachel and I had a very explicit, you know, long series of conversations over a long period of time about like, well, how can we be the most helpful to society as a whole, and particularly to those people who maybe need more help, you know? And so we definitely saw the world going in a potentially pretty dystopian direction if the world's most powerful technology was controlled by a small group of elites. So we thought, yeah, we should focus on trying to help that not happen. You know, sadly, it looks like it still is likely to happen. But I mean, I feel like we've helped make it a little bit less likely. So we've done our bit. [00:26:39]Swyx: You've shown that it's possible. And I think your constant advocacy, your courses, your research that you publish, you know, just the other day you published a finding on, you know, learning that I think is still something that people are still talking about quite a lot. I think that that is the origin story of a lot of people who are going to be, you know, little Jeremy Howards, furthering your mission with, you know, you don't have to do everything by yourself is what I'm saying. No, definitely. Definitely. [00:27:10]Jeremy: You know, that was a big takeaway from like, analytic was analytic. It definitely felt like we had to do everything ourselves. And I kind of, I wanted to solve medicine. I'll say, yeah, okay, solving medicine is actually quite difficult. And I can't do it on my own. And there's a lot of other things I'd like to solve, and I can't do those either. So that was definitely the other piece was like, yeah, you know, can we create an army of passionate domain experts who can change their little part of the world? And that's definitely happened. Like I find nowadays, at least half the time, probably quite a bit more that I get in contact with somebody who's done really interesting work in some domain. Most of the time I'd say, they say, yeah, I got my start with fast.ai. So it's definitely, I can see that. And I also know from talking to folks at places like Amazon and Adobe and stuff, which, you know, there's lots of alumni there. And they say, oh my God, I got here. And like half of the people are fast.ai alumni. So it's fantastic. [00:28:13]Swyx: Yeah. [00:28:14]Jeremy: Actually, Andre Kapathy grabbed me when I saw him at NeurIPS a few years ago. And he was like, I have to tell you, thanks for the fast.ai courses. When people come to Tesla and they need to know more about deep learning, we always send them to your course. And the OpenAI Scholars Program was doing the same thing. So it's kind of like, yeah, it's had a surprising impact, you know, that's just one of like three things we do is the course, you know. [00:28:40]Swyx: Yes. [00:28:40]Jeremy: And it's only ever been at most two people, either me and Rachel or me and Sylvia nowadays, it's just me. So yeah, I think it shows you don't necessarily need a huge amount of money and a huge team of people to make an impact. [00:28:56]Swyx: Yeah. So just to reintroduce fast.ai for people who may not have dived into it much, there is the courses that you do. There is the library that is very well loved. And I kind of think of it as a nicer layer on top of PyTorch that people should start with by default and use it as the basis for a lot of your courses. And then you have like NBDev, which I don't know, is that the third one? [00:29:27]Jeremy: Oh, so the three areas were research, software, and courses. [00:29:32]Swyx: Oh, sorry. [00:29:32]Jeremy: So then in software, you know, fast.ai is the main thing, but NBDev is not far behind. But then there's also things like FastCore, GHAPI, I mean, dozens of open source projects that I've created and some of them have been pretty popular and some of them are still a little bit hidden, actually. Some of them I should try to do a better job of telling people about. [00:30:01]Swyx: What are you thinking about? Yeah, what's on the course of my way? Oh, I don't know, just like little things. [00:30:04]Jeremy: Like, for example, for working with EC2 and AWS, I created a FastEC2 library, which I think is like way more convenient and nice to use than anything else out there. And it's literally got a whole autocomplete, dynamic autocomplete that works both on the command line and in notebooks that'll like auto-complete your instance names and everything like that. You know, just little things like that. I try to make like, when I work with some domain, I try to make it like, I want to make it as enjoyable as possible for me to do that. So I always try to kind of like, like with GHAPI, for example, I think that GitHub API is incredibly powerful, but I didn't find it good to work with because I didn't particularly like the libraries that are out there. So like GHAPI, like FastEC2, it like autocompletes both at the command line or in a notebook or whatever, like literally the entire GitHub API. The entire thing is like, I think it's like less than 100K of code because it actually, as far as I know, the only one that grabs it directly from the official open API spec that GitHub produces. And like if you're in GitHub and you just type an API, you know, autocomplete API method and hit enter, it prints out the docs with brief docs and then gives you a link to the actual documentation page. You know, GitHub Actions, I can write now in Python, which is just so much easier than writing them in TypeScript and stuff. So, you know, just little things like that. [00:31:40]Swyx: I think that's an approach which more developers took to publish some of their work along the way. You described the third arm of FastAI as research. It's not something I see often. Obviously, you do do some research. And how do you run your research? What are your research interests? [00:31:59]Jeremy: Yeah, so research is what I spend the vast majority of my time on. And the artifacts that come out of that are largely software and courses. You know, so to me, the main artifact shouldn't be papers because papers are things read by a small exclusive group of people. You know, to me, the main artifacts should be like something teaching people, here's how to use this insight and here's software you can use that builds it in. So I think I've only ever done three first-person papers in my life, you know, and none of those are ones I wanted to do. You know, they were all ones that, like, so one was ULM Fit, where Sebastian Ruder reached out to me after seeing the course and said, like, you have to publish this as a paper, you know. And he said, I'll write it. He said, I want to write it because if I do, I can put it on my PhD and that would be great. And it's like, okay, well, I want to help you with your PhD. And that sounds great. So like, you know, one was the masks paper, which just had to exist and nobody else was writing it. And then the third was the Fast.ai library paper, which again, somebody reached out and said, please, please write this. We will waive the fee for the journal and everything and actually help you get it through publishing and stuff. So yeah, so I don't, other than that, I've never written a first author paper. So the research is like, well, so for example, you know, Dawn Bench was a competition, which Stanford ran a few years ago. It was kind of the first big competition of like, who can train neural nets the fastest rather than the most accurate. And specifically it was who can train ImageNet the fastest. And again, this was like one of these things where it was created by necessity. So Google had just released their TPUs. And so I heard from my friends at Google that they had put together this big team to smash Dawn Bench so that they could prove to people that they had to use Google Cloud and use their TPUs and show how good their TPUs were. And we kind of thought, oh s**t, this would be a disaster if they do that, because then everybody's going to be like, oh, deep learning is not accessible. [00:34:20]Swyx: You know, to actually be good at it, [00:34:21]Jeremy: you have to be Google and you have to use special silicon. And so, you know, we only found out about this 10 days before the competition finished. But, you know, we basically got together an emergency bunch of our students and Rachel and I and sat for the next 10 days and just tried to crunch through and try to use all of our best ideas that had come from our research. And so particularly progressive resizing, just basically train mainly on small things, train on non-square things, you know, stuff like that. And so, yeah, we ended up winning, thank God. And so, you know, we turned it around from being like, like, oh s**t, you know, this is going to show that you have to be Google and have TPUs to being like, oh my God, even the little guy can do deep learning. So that's an example of the kind of like research artifacts we do. And yeah, so all of my research is always, how do we do more with less, you know? So how do we get better results with less data, with less compute, with less complexity, with less education, you know, stuff like that. So ULM fits obviously a good example of that. [00:35:37]Swyx: And most recently you published, can LLMs learn from a single example? Maybe could you tell the story a little bit behind that? And maybe that goes a little bit too far into the learning of very low resource, the literature. [00:35:52]Jeremy: Yeah, yeah. So me and my friend, Jono Whittaker, basically had been playing around with this fun Kaggle competition, which is actually still running as we speak, which is, can you create a model which can answer multiple choice questions about anything that's in Wikipedia? And the thing that makes it interesting is that your model has to run on Kaggle within nine hours. And Kaggle's very, very limited. So you've only got 14 gig RAM, only two CPUs, and a small, very old GPU. So this is cool, you know, if you can do well at this, then this is a good example of like, oh, you can do more with less. So yeah, Jono and I were playing around with fine tuning, of course, transfer learning, pre-trained language models. And we saw this, like, so we always, you know, plot our losses as we go. So here's another thing we created. Actually, Sylvain Guuger, when he worked with us, created called fast progress, which is kind of like TQEDM, but we think a lot better. So we look at our fast progress curves, and they kind of go down, down, down, down, down, down, down, a little bit, little bit, little bit. And then suddenly go clunk, and they drop. And then down, down, down, down, down a little bit, and then suddenly clunk, they drop. We're like, what the hell? These clunks are occurring at the end of each epoch. So normally in deep learning, this would be, this is, you know, I've seen this before. It's always been a bug. It's always turned out that like, oh, we accidentally forgot to turn on eval mode during the validation set. So I was actually learning then, or, oh, we accidentally were calculating moving average statistics throughout the epoch. So, you know, so it's recently moving average or whatever. And so we were using Hugging Face Trainer. So, you know, I did not give my friends at Hugging Face the benefit of the doubt. I thought, oh, they've fucked up Hugging Face Trainer, you know, idiots. Well, you'll use the Fast AI Trainer instead. So we switched over to Learner. We still saw the clunks and, you know, that's, yeah, it shouldn't really happen because semantically speaking in the epoch, isn't like, it's not a thing, you know, like nothing happens. Well, nothing's meant to happen when you go from ending one epoch to starting the next one. So there shouldn't be a clunk, you know. So I kind of asked around on the open source discords. That's like, what's going on here? And everybody was just like, oh, that's just what, that's just what these training curves look like. Those all look like that. Don't worry about it. And I was like, oh, are you all using Trainer? Yes. Oh, well, there must be some bug with Trainer. And I was like, well, we also saw it in Learner [00:38:42]Swyx: and somebody else is like, [00:38:42]Jeremy: no, we've got our own Trainer. We get it as well. They're just like, don't worry about it. It's just something we see. It's just normal. [00:38:48]Swyx: I can't do that. [00:38:49]Jeremy: I can't just be like, here's something that's like in the previous 30 years of neural networks, nobody ever saw it. And now suddenly we see it. [00:38:57]Swyx: So don't worry about it. [00:38:59]Jeremy: I just, I have to know why. [00:39:01]Swyx: Can I clarify? This is, was everyone that you're talking to, were they all seeing it for the same dataset or in different datasets? [00:39:08]Jeremy: Different datasets, different Trainers. They're just like, no, this is just, this is just what it looks like when you fine tune language models. Don't worry about it. You know, I hadn't seen it before, but I'd been kind of like, as I say, I, you know, I kept working on them for a couple of years after ULM fit. And then I kind of moved on to other things, partly out of frustration. So I hadn't been fine tuning, you know, I mean, Lama's only been out for a few months, right? But I wasn't one of those people who jumped straight into it, you know? So I was relatively new to the kind of Lama fine tuning world, where else these guys had been, you know, doing it since day one. [00:39:49]Swyx: It was only a few months ago, [00:39:51]Jeremy: but it's still quite a bit of time. So, so yeah, they're just like, no, this is all what we see. [00:39:56]Swyx: Don't worry about it. [00:39:56]Jeremy: So yeah, I, I've got a very kind of like, I don't know, I've just got this brain where I have to know why things are. And so I kind of, I ask people like, well, why, why do you think it's happening? And they'd be like, oh, it would pretty obviously, cause it's like memorize the data set. It's just like, that can't be right. It's only seen it once. Like, look at this, the loss has dropped by 0.3, 0.3, which is like, basically it knows the answer. And like, no, no, it's just, it is, it's just memorize the data set. So yeah. So look, Jono and I did not discover this and Jono and I did not come up with a hypothesis. You know, I guess we were just the ones, I guess, who had been around for long enough to recognize that like, this, this isn't how it's meant to work. And so we, we, you know, and so we went back and like, okay, let's just run some experiments, you know, cause nobody seems to have actually published anything about this. [00:40:51]Well, not quite true.Some people had published things, but nobody ever actually stepped back and said like, what the hell, you know, how can this be possible? Is it possible? Is this what's happening? And so, yeah, we created a bunch of experiments where we basically predicted ahead of time. It's like, okay, if this hypothesis is correct, that it's memorized in the training set, then we ought to see blah, under conditions, blah, but not under these conditions. And so we ran a bunch of experiments and all of them supported the hypothesis that it was memorizing the data set in a single thing at once. And it's a pretty big data set, you know, which in hindsight, it's not totally surprising because the theory, remember, of the ULMFiT theory was like, well, it's kind of creating all these latent capabilities to make it easier for it to predict the next token. So if it's got all this kind of latent capability, it ought to also be really good at compressing new tokens because it can immediately recognize it as like, oh, that's just a version of this. So it's not so crazy, you know, but it is, it requires us to rethink everything because like, and nobody knows like, okay, so how do we fine tune these things? Because like, it doesn't even matter. Like maybe it's fine. Like maybe it's fine that it's memorized the data set after one go and you do a second go and okay, the validation loss is terrible because it's now really overconfident. [00:42:20]Swyx: That's fine. [00:42:22]Jeremy: Don't, you know, don't, I keep telling people, don't track validation loss, track validation accuracy because at least that will still be useful. Just another thing that's got lost since ULMFiT, nobody tracks accuracy of language models anymore. But you know, it'll still keep learning and it does, it does keep improving. But is it worse? You know, like, is it like, now that it's kind of memorized it, it's probably getting a less strong signal, you know, I don't know. So I still don't know how to fine tune language models properly and I haven't found anybody who feels like they do, like nobody really knows whether this memorization thing is, it's probably a feature in some ways. It's probably some things that you can do usefully with it. It's probably, yeah, I have a feeling it's messing up training dynamics as well. [00:43:13]Swyx: And does it come at the cost of catastrophic forgetting as well, right? Like, which is the other side of the coin. [00:43:18]Jeremy: It does to some extent, like we know it does, like look at Code Llama, for example. So Code Llama was a, I think it was like a 500 billion token fine tuning of Llama 2 using code. And also pros about code that Meta did. And honestly, they kind of blew it because Code Llama is good at coding, but it's bad at everything else, you know, and it used to be good. Yeah, I was pretty sure it was like, before they released it, me and lots of people in the open source discords were like, oh my God, you know, we know this is coming, Jan Lukinsk saying it's coming. I hope they kept at least like 50% non-code data because otherwise it's going to forget everything else. And they didn't, only like 0.3% of their epochs were non-code data. So it did, it forgot everything else. So now it's good at code and it's bad at everything else. So we definitely have catastrophic forgetting. It's fixable, just somebody has to do, you know, somebody has to spend their time training a model on a good mix of data. Like, so, okay, so here's the thing. Even though I originally created three-step approach that everybody now does, my view is it's actually wrong and we shouldn't use it. [00:44:36]Jeremy: And that's because people are using it in a way different to why I created it. You know, I created it thinking the task-specific models would be more specific. You know, it's like, oh, this is like a sentiment classifier as an example of a task, you know, but the tasks now are like a, you know, RLHF, which is basically like answer questions that make people feel happy about your answer. So that's a much more general task and it's a really cool approach. And so we see, for example, RLHF also breaks models like, you know, like GPT-4, RLHDEFT, we know from kind of the work that Microsoft did, you know, the pre, the earlier, less aligned version was better. And these are all kind of examples of catastrophic forgetting. And so to me, the right way to do this is to fine-tune language models, is to actually throw away the idea of fine-tuning. There's no such thing. There's only continued pre-training. And pre-training is something where from the very start, you try to include all the kinds of data that you care about, all the kinds of problems that you care about, instructions, exercises, code, general purpose document completion, whatever. And then as you train, you gradually curate that, you know, you gradually make that higher and higher quality and more and more specific to the kinds of tasks you want it to do. But you never throw away any data. You always keep all of the data types there in reasonably high quantities. You know, maybe the quality filter, you stop training on low quality data, because that's probably fine to forget how to write badly, maybe. So yeah, that's now my view, is I think ULM fit is the wrong approach. And that's why we're seeing a lot of these, you know, so-called alignment tacks and this view of like, oh, a model can't both code and do other things. And, you know, I think it's actually because people are training them wrong. [00:46:47]Swyx: Yeah, well, I think you have a clear [00:46:51]Alessio: anti-laziness approach. I think other people are not as good hearted, you know, they're like, [00:46:57]Swyx: hey, they told me this thing works. [00:46:59]Alessio: And if I release a model this way, people will appreciate it, I'll get promoted and I'll kind of make more money. [00:47:06]Jeremy: Yeah, and it's not just money. It's like, this is how citations work most badly, you know, so if you want to get cited, you need to write a paper that people in your field recognize as an advancement on things that we know are good. And so we've seen this happen again and again. So like I say, like zero shot and few shot learning, everybody was writing about that. Or, you know, with image generation, everybody just was writing about GANs, you know, and I was trying to say like, no, GANs are not the right approach. You know, and I showed again through research that we demonstrated in our videos that you can do better than GANs, much faster and with much less data. And nobody cared because again, like if you want to get published, you write a GAN paper that slightly improves this part of GANs and this tiny field, you'll get published, you know. So it's, yeah, it's not set up for real innovation. It's, you know, again, it's really helpful for me, you know, I have my own research lab with nobody telling me what to do and I don't even publish. So it doesn't matter if I get citations. And so I just write what I think actually matters. I wish there was, and, you know, and actually places like OpenAI, you know, the researchers there can do that as well. It's a shame, you know, I wish there was more academic, open venues in which people can focus on like genuine innovation. [00:48:38]Swyx: Twitter, which is unironically has become a little bit of that forum. I wanted to follow up on one thing that you mentioned, which is that you checked around the open source discords. I don't know if it's too, I don't know if it's a pusher to ask like what discords are lively or useful right now. I think that something I definitely felt like I missed out on was the early days of Luther AI, which is a very hard bit. And, you know, like what is the new Luther? And you actually shouted out the alignment lab AI discord in your blog post. And that was the first time I even knew, like I saw them on Twitter, never knew they had a discord, never knew that there was actually substantive discussions going on in there and that you were an active member of it. Okay, yeah. [00:49:23]Jeremy: And then even then, if you do know about that and you go there, it'll look like it's totally dead. And that's because unfortunately, nearly all the discords, nearly all of the conversation happens in private channels. You know, and that's, I guess. [00:49:35]Swyx: How does someone get into that world? Because it's obviously very, very instructive, right? [00:49:42]Jeremy: You could just come to the first AI discord, which I'll be honest with you, it's less bustling than some of the others, but it's not terrible. And so like, at least, to be fair, one of Emma's bustling channels is private. [00:49:57]Swyx: I guess. [00:49:59]Jeremy: So I'm just thinking. [00:50:01]Swyx: It's just the nature of quality discussion, right? Yeah, I guess when I think about it, [00:50:05]Jeremy: I didn't have any private discussions on our discord for years, but there was a lot of people who came in with like, oh, I just had this amazing idea for AGI. If you just thought about like, if you imagine that AI is a brain, then we, you know, this just, I don't want to talk about it. You know, I don't want to like, you don't want to be dismissive or whatever. And it's like, oh, well, that's an interesting comment, but maybe you should like, try training some models first to see if that aligns with your intuition. Like, oh, but how could I possibly learn? It's like, well, we have a course, just actually spend time learning. Like, you know, anyway. And there's like, okay, I know the people who always have good answers there. And so I created a private channel and put them all in it. And I got to admit, that's where I post more often because there's much less, you know, flight of fancy views about how we could solve AGI, blah, blah, blah. So there is a bit of that. But having said that, like, I think the bar is pretty low. Like if you join a Discord and you can hit the like participants or community or whatever button, you can see who's in it. And then you'll see at the top, who the admins or moderators or people in the dev role are. And just DM one of them and say like, oh, here's my GitHub. Well, here's some blog posts I wrote. You know, I'm interested in talking about this, you know, can I join the private channels? And I've never heard of anybody saying no. I will say, you know, Alutha's all pretty open. So you can do the Alutha Discord still. You know, one problem with the Alutha Discord is it's been going on for so long that it's like, it's very inside baseball. It's quite hard to get started. Yeah. Carpa AI looks, I think it's all open. That's just less stability. That's more accessible. [00:52:03]Swyx: Yeah. [00:52:04]Jeremy: There's also just recently, now it's research that does like the Hermes models and data set just opened. They've got some private channels, but it's pretty open, I think. You mentioned Alignment Lab, that one it's all the interesting stuff is on private channels. So just ask. If you know me, ask me, cause I've got admin on that one. There's also, yeah, OS Skunkworks, OS Skunkworks AI is a good Discord, which I think it's open. So yeah, they're all pretty good. [00:52:40]Swyx: I don't want you to leak any, you know, Discords that don't want any publicity, but this is all helpful. [00:52:46]Jeremy: We all want people, like we all want people. [00:52:49]Swyx: We just want people who like, [00:52:51]Jeremy: want to build stuff, rather than people who, and like, it's fine to not know anything as well, but if you don't know anything, but you want to tell everybody else what to do and how to do it, that's annoying. If you don't know anything and want to be told like, here's a really small kind of task that as somebody who doesn't know anything is going to take you a really long time to do, but it would still be helpful. Then, and then you go and do it. That would be great. The truth is, yeah, [00:53:19]Swyx: like, I don't know, [00:53:20]Jeremy: maybe 5% of people who come in with great enthusiasm and saying that they want to learn and they'll do anything. [00:53:25]Swyx: And then somebody says like, [00:53:25]Jeremy: okay, here's some work you can do. Almost nobody does that work. So if you're somebody who actually does the work and follows up, you will massively stand out. That's an extreme rarity. And everybody will then want to help you do more work. [00:53:41]Swyx: So yeah. [00:53:41]Jeremy: So just, yeah, just do work and people will want to support you. [00:53:47]Alessio: Our Discord used to be referral only for a long time. We didn't have a public invite and then we opened it and they're kind of like channel gating. Yeah. A lot of people just want to do, I remember it used to be like, you know, a forum moderator. [00:54:00]Swyx: It's like people just want to do [00:54:01]Alessio: like drive-by posting, [00:54:03]Swyx: you know, and like, [00:54:03]Alessio: they don't want to help the community. They just want to get their question answered. [00:54:07]Jeremy: I mean, the funny thing is our forum community does not have any of that garbage. You know, there's something specific about the low latency thing where people like expect an instant answer. And yeah, we're all somehow in a forum thread where they know it's like there forever. People are a bit more thoughtful, but then the forums are less active than they used to be because Discord has got more popular, you know? So it's all a bit of a compromise, you know, running a healthy community is, yeah, it's always a bit of a challenge. All right, we got so many more things [00:54:47]Alessio: we want to dive in, but I don't want to keep you here for hours. [00:54:50]Swyx: This is not the Lex Friedman podcast [00:54:52]Alessio: we always like to say. One topic I would love to maybe chat a bit about is Mojo, modular, you know, CrystalLiner, not many of you on the podcast. So we want to spend a little time there. You recently did a hacker's guide to language models and you ran through everything from quantized model to like smaller models, larger models, and all of that. But obviously modular is taking its own approach. Yeah, what got you excited? I know you and Chris have been talking about this for like years and a lot of the ideas you had, so. [00:55:23]Jeremy: Yeah, yeah, yeah, yeah, no, absolutely. So I met Chris, I think it was at the first TensorFlow Dev Summit. And I don't think he had even like, I'm not sure if he'd even officially started his employment with Google at that point. So I don't know, you know, certainly nothing had been mentioned. So I, you know, I admired him from afar with LLVM and Swift and whatever. And so I saw him walk into the courtyard at Google. It's just like, oh s**t, man, that's Chris Latner. I wonder if he would lower his standards enough to talk to me. Well, worth a try. So I caught up my courage because like nobody was talking to him. He looked a bit lost and I wandered over and it's like, oh, you're Chris Latner, right? It's like, what are you doing here? What are you doing here? And I was like, yeah, yeah, yeah. It's like, oh, I'm Jeremy Howard. It's like, oh, do you do some of this AI stuff? And I was like, yeah, yeah, I like this AI stuff. Are you doing AI stuff? It's like, well, I'm thinking about starting to do some AI stuff. Yeah, I think it's going to be cool. And it's like, wow. So like, I spent the next half hour just basically brain dumping all the ways in which AI was stupid to him. And he listened patiently. And I thought he probably wasn't even remember or care or whatever. But yeah, then I kind of like, I guess I re-caught up with him a few months later. And it's like, I've been thinking about everything you said in that conversation. And he like narrated back his response to every part of it, projects he was planning to do. And it's just like, oh, this dude follows up. Holy s**t. And I was like, wow, okay. And he was like, yeah, so we're going to create this new thing called Swift for TensorFlow. And it's going to be like, it's going to be a compiler with auto differentiation built in. And blah, blah, blah. And I was like, why would that help? [00:57:10]Swyx: You know, why would you? [00:57:10]Jeremy: And he was like, okay, with a compiler during the forward pass, you don't have to worry about saving context, you know, because a lot will be optimized in the backward. But I was like, oh my God. Because I didn't really know much about compilers. You know, I spent enough to kind of like, understand the ideas, but it hadn't occurred to me that a compiler basically solves a lot of the problems we have as end users. I was like, wow, that's amazing. Okay, you do know, right, that nobody's going to use this unless it's like usable. It's like, yeah, I know, right. So I was thinking you should create like a fast AI for this. So, okay, but I don't even know Swift. And he was like, well, why don't you start learning it? And if you have any questions, ask me. It's just like, holy s**t. Like, not only has Chris Latner lowered his standards enough to talk to me, but he's offering me personal tutoring on the programming language that he made. So I was just like, I'm not g

Main Topics:Google Meet using Chrome on MacOS and facing Mac screen sharing issues.Transition from Zoho to FastMail, focusing on privacy and benefits.FastMail's pseudo email features and its cooperation with One Password.Differences and similarities between FastMail and Google Workspace.Deep analysis of Tesla's self-driving technology and neural network models.Evaluation of Tesla's autonomous driving performance.Technological understanding behind self-driving cars, discussing self-parking, sensors, cameras, and radar.Relationship of radar, LiDAR, and Microsoft Kinect in autonomous cars.Discussion on crash rates of autonomous cars vs human-driven cars.Comparing Waymo, Cruise, and Chevy Bolt EV costs and experiences.Introduction of Waymo's partnership with a Chinese manufacturer for a purpose-built car.Speculating the future of self-driving cars.Electric vehicle industry: challenges, GM's Ultium platform benefits, and cost savings.Review and analysis of films including Barbie, No Hard Feelings, Palm Springs, Deadpool, The Proposal, Zoolander, and Naked Gun.References:Google MeetChrome (macOS)ZohoFastMailiCloud+One PasswordGoogle WorkspaceTeslaROS (Robot Operating System)Microsoft KinectWaymoCruiseChevy Bolt EVGM's Ultium platformVarious films: Barbie, No Hard Feelings, Palm Springs, Deadpool, The Proposal, Zoolander, Naked Gun, Hotshot, Lethal Weapon, American Pie, Free guy.Reach Out: Engage with us and the community on our Discord Channel.

Apple releases iOS 17.0.3 to fix overheating issues, your hosts discuss iPhone 15 Pro Max 5x camera and Apple Watch Ultra 2, Google's adding 'Bard' to their Assistant, and remembering Steve Jobs 12 years later.Contact our hosts@stephenrobles on Threads@stephenrobles on TwitterStephen on Mastodon@williamgallagher_ on Threads@WGallagher on TwitterWilliam on MastodonSponsored by:Magic Lasso Adblock: Block ads, trackers and annoyances on your iPhone, iPad and Mac. Get 1 month free access to Pro features today. Visit: magiclasso.co/appleinsider to learn more.Fastmail: To learn more about Fastmail and receive 10% off your first year, visit: fastmail.com/appleinsiderBabbel: Get 55% off your subscription and start learning a new language with Babbel when you visit: babbel.com/appleinsiderLinks from the showNational emergency alert test scheduled for October 4Apple releases iOS 17.0.3 and iPadOS 17.0.3 to fix overheating bugBuy Razer Phone CoolerActions App on the App StoreAction Button Settings ShortcutHow to use spirit level in iOS 17 Camera appSDW's iPhone 15 Pro Max Camera ReviewWhat's new in iOS 17.1 beta 2: StandBy, ringtones, and moreHow to use Double Tap on Apple WatchHow to rearrange your iMessage apps in iOS 17Google's Pixel 8 series promises AI camera featuresGoogle teases Assistant with Bard for iOS and AndroidSupport the showSupport the show on Patreon or Apple Podcasts to get ad-free episodes every week, access to our private Discord channel, and early release of the show! We would also appreciate a 5-star rating and review in Apple PodcastsMore AppleInsider podcastsTune in to our HomeKit Insider podcast covering the latest news, products, apps and everything HomeKit related. Subscribe in Apple Podcasts, Overcast, or just search for HomeKit Insider wherever you get your podcasts.Subscribe and listen to our AppleInsider Daily podcast for the latest Apple news Monday through Friday. You can find it on Apple Podcasts, Overcast, or anywhere you listen to podcasts.Podcast artwork from Basic Apple Guy. Download the free wallpaper pack here.Those interested in sponsoring the show can reach out to us at: steve@appleinsider.com (00:00) - Intro (02:28) - Mountain Workouts (05:36) - U.S. Emergency Alert (09:16) - iOS 17.0.3 Overheating (12:55) - Action Button (19:20) - Sponsor: Magic Lasso Adblock (21:29) - External SSD Recording (25:13) - iPhone 15 Pro Max 5x (34:50) - Watch Ultra 2 Hike (37:33) - Sponsor: Fastmail (39:27) - Sponsor: Babbel (41:06) - iOS 17.1 Beta (47:40) - Messages Apps (50:12) - Pixel 8 (54:05) - Remembering Steve ★ Support this podcast on Patreon ★

We're trying out Rhino Linux—a unique take on rolling Ubuntu with AUR-like powers and other surprises.

Last week I covered the importance of audience testimonials. Now I want to share what our fans say about our network. "I have purchased several items advertised on the TWiT network. I buy them because I trust Leo Laporte and his team's knowledge of the latest in technology. If Leo supports it, I know I can trust it." "I have bought and/or subscribed to a number of products and services over the years as a result of ads on TWiT network shows. The ads are thoughtfully done, and the trust the network has built with its community lends credibility and authenticity to the host-read ads. Some of the ad-hoc commentary during the reads is great too." "'Podcasts you love from people you trust.' Sponsors I've purchased from: Casper, Squarespace, LastPass, Bitwarden, Express VPN, Hover, Harry's, Fastmail, Eight Sleep, iTVPro, and Mint Mobile. I'm sure there are more, but I remember those off the top of my head." "I have several times purchased items that I have heard advertised on podcasts from TWiT.tv. Have listened to these wonderful podcasts for years, and the advertisements presented by the hosts of these shows seem like products I can trust, because TWiT trusts them." "I don't know about others. But when one of your hosts recommends something they are catering for clear concise solution to a problem, pair that with conviction and trust you have powerful pitch for purchasing." If you are interested in advertising with TWiT.tv because you want the gold standard in podcast advertising, then please email advertise@twit.tv, and we will get back to you ASAP. Host: Lisa Laporte Download or subscribe to this show at https://twit.tv/shows/host-read-ads

This just in, egg corner, you won't believe what happens next! Scammers are everywhere, watch out! It's my way or the highway. Wait, who's way? Andrew tells us about a brand new app called Calendar.app and 15 alternatives. And don't forget, ~~One Prime Plus Dot Com~~ Canion Dot Blog Slash Save! Damn Flies 00:00:00 Gnat (https://en.wikipedia.org/wiki/Gnat) :bug Fly (https://en.wikipedia.org/wiki/Fly)

Twitter cut off its API access, and developers of apps like Tweetbot and Twitterrific are offering specific options to help them out. A new iMac is coming out soon? Why can't I access my files from the Google Takeout service? Is there another service I can use to access my Google data? Is there an iOS shortcut for this? The Father of Modern Genomics, Dr. George Church. Lucid dreaming & headphones you can wear to sleep. Is there an easy way to move contacts over to Fastmail? Ant Pruitt & moving from Windows to Mac. Is it worth setting up IoT devices on its own network? Does Apple send out a notification to your old iPhone when your device reaches its end of life? Hosts: Leo Laporte and Mikah Sargent Guest: Ant Pruitt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/1964 Download or subscribe to this show at: https://twit.tv/shows/ask-the-tech-guys Sponsors: lectricebikes.com fortra.com

Twitter cut off its API access, and developers of apps like Tweetbot and Twitterrific are offering specific options to help them out. A new iMac is coming out soon? Why can't I access my files from the Google Takeout service? Is there another service I can use to access my Google data? Is there an iOS shortcut for this? The Father of Modern Genomics, Dr. George Church. Lucid dreaming & headphones you can wear to sleep. Is there an easy way to move contacts over to Fastmail? Ant Pruitt & moving from Windows to Mac. Is it worth setting up IoT devices on its own network? Does Apple send out a notification to your old iPhone when your device reaches its end of life? Hosts: Leo Laporte and Mikah Sargent Guest: Ant Pruitt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/1964 Download or subscribe to this show at: https://twit.tv/shows/all-twittv-shows Sponsors: lectricebikes.com fortra.com

Twitter cut off its API access, and developers of apps like Tweetbot and Twitterrific are offering specific options to help them out. A new iMac is coming out soon? Why can't I access my files from the Google Takeout service? Is there another service I can use to access my Google data? Is there an iOS shortcut for this? The Father of Modern Genomics, Dr. George Church. Lucid dreaming & headphones you can wear to sleep. Is there an easy way to move contacts over to Fastmail? Ant Pruitt & moving from Windows to Mac. Is it worth setting up IoT devices on its own network? Does Apple send out a notification to your old iPhone when your device reaches its end of life? Hosts: Leo Laporte and Mikah Sargent Guest: Ant Pruitt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/1964 Download or subscribe to this show at: https://twit.tv/shows/total-leo Sponsors: lectricebikes.com fortra.com

Twitter cut off its API access, and developers of apps like Tweetbot and Twitterrific are offering specific options to help them out. A new iMac is coming out soon? Why can't I access my files from the Google Takeout service? Is there another service I can use to access my Google data? Is there an iOS shortcut for this? The Father of Modern Genomics, Dr. George Church. Lucid dreaming & headphones you can wear to sleep. Is there an easy way to move contacts over to Fastmail? Ant Pruitt & moving from Windows to Mac. Is it worth setting up IoT devices on its own network? Does Apple send out a notification to your old iPhone when your device reaches its end of life? Hosts: Leo Laporte and Mikah Sargent Guest: Ant Pruitt Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/ask-the-tech-guys/episodes/1964 Download or subscribe to this show at: https://twit.tv/shows/ask-the-tech-guys Sponsors: lectricebikes.com fortra.com

Apparently basketball is very important. Are you drinking the Hemispheric Views Kool-Aid? Martin is! Jason goes digging for rocks. Oh, and, of course, Andrew wants you to "save" some money. Quick Show Because Basketball 00:00:00 NBL (https://nbl.com.au/)

​Should you upgrade to the new iPads? What Mac mini should I get? Do I need a new Google account to set up a Chromebook? Setting up an email server, the right resolution to scan photos, drawing tablets for beginners, troubleshooting slow startups on Linux Mint. Round-up of this week's new iPad announcements What Mac-Mini should I get? Restoring iPad OS Setting up a Chromebook Setting up an email server Convert Flac to an Apple format The right resolution to scan paper photos Easier ways to search for a movie across different streaming services Finding a drawing tablet for a beginner Resources to learn about building a PC Playing YouTube on Dish's Hopper Troubleshooting slow startup on Linux Mint Make a Rukus - It's Musical Fun With The New RUKUSfx Host: Mikah Sargent Guests: Scott Wilkinson, Johnny Jet, and Dick DeBartolo Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/the-tech-guy/episodes/1937 Download or subscribe to this show at: https://twit.tv/shows/the-tech-guy Sponsors: nureva.com/twit cachefly.com

​Should you upgrade to the new iPads? What Mac mini should I get? Do I need a new Google account to set up a Chromebook? Setting up an email server, the right resolution to scan photos, drawing tablets for beginners, troubleshooting slow startups on Linux Mint. Round-up of this week's new iPad announcements What Mac-Mini should I get? Restoring iPad OS Setting up a Chromebook Setting up an email server Convert Flac to an Apple format The right resolution to scan paper photos Easier ways to search for a movie across different streaming services Finding a drawing tablet for a beginner Resources to learn about building a PC Playing YouTube on Dish's Hopper Troubleshooting slow startup on Linux Mint Make a Rukus - It's Musical Fun With The New RUKUSfx Host: Mikah Sargent Guests: Scott Wilkinson, Johnny Jet, and Dick DeBartolo Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/the-tech-guy/episodes/1937 Download or subscribe to this show at: https://twit.tv/shows/all-twittv-shows Sponsors: nureva.com/twit cachefly.com

​Should you upgrade to the new iPads? What Mac mini should I get? Do I need a new Google account to set up a Chromebook? Setting up an email server, the right resolution to scan photos, drawing tablets for beginners, troubleshooting slow startups on Linux Mint. Round-up of this week's new iPad announcements What Mac-Mini should I get? Restoring iPad OS Setting up a Chromebook Setting up an email server Convert Flac to an Apple format The right resolution to scan paper photos Easier ways to search for a movie across different streaming services Finding a drawing tablet for a beginner Resources to learn about building a PC Playing YouTube on Dish's Hopper Troubleshooting slow startup on Linux Mint Make a Rukus - It's Musical Fun With The New RUKUSfx Host: Mikah Sargent Guests: Scott Wilkinson, Johnny Jet, and Dick DeBartolo Get episodes ad-free with Club TWiT at https://twit.tv/clubtwit Show notes and links for this episode are available at: https://twit.tv/shows/the-tech-guy/episodes/1937 Download or subscribe to this show at: https://twit.tv/shows/the-tech-guy Sponsors: nureva.com/twit cachefly.com

Pre-show: Geometry Dash Number Munchers Number Munchers on Classic Reload Number Munchers on the App Store Unity Unity CEO’s colorful opinions The Talk Show #353: Shop Different Playdate Pulp VVVVVV FromSoftawre Follow-up: John on M1 MacBook Air vs. M2 MacBook Air speakers You can set up Fastmail to have a catch-all Casey’s referral code Find My on vacation Steps via Steve J. Stutz Open Find My Tap Devices Tap on the device you want to silence Scroll down and under Notifications toggle off Notify When Left Behind Paul Squires understood the assignment HomeKit automation & locations Derek Martin has tips regarding how to transfer to a new home address Jim Bruhn provides instructions for how to disable hub features: Settings Airplay and HomeKit Home Hub → My Home Switch to Disconnected Are there any decent AppleTV photo-viewer apps? (via Torb Lunde Mirada) Contact address ordering Reorder It! from @takaaki024app (via philuop) Robb Woodring saves the day with changing default messaging address/number Go to an existing thread Tap the name of the person at the top Tap Info Tap and hold the Message/bubble icon Select the appropriate number Unlocking automotive features Blaine Toups’ Jaguar F-Type’s two extra cylinders Car & Driver write-up Tesla’s Acceleration Boost (via Josh Rapoport) Sean Harding makes a good point about API deprecation John’s New TV Amazon link Previously on ATP… Dirac Live FILMMAKER MODE 4K HDR Demo Video Mac OS X Unix-style crash Post-show: Is it okay to copy a competitor’s API? S3 Cloudflare R2 Linode Object Storage Service Backblaze B2 Ceph Transmit Sponsored by: Linode: Instantly deploy and manage an SSD server in the Linode Cloud. New accounts get a $100 credit. Squarespace: Make your next move. Use code atp for 10% off your first order. Instabug: The visibility you need to build top-quality apps. Become a member for ad-free episodes and our early-release, unedited “bootleg” feed! Become a member!

Pre-show: Air conditioners, memories, and house veins Mini-split Follow-up: M2 MacBook Air [also] thermal throttles Jason’s review You should probably subscribe to 6C, too The Verge Gruber’s review Ars Technica’s review John’s Mac Studio safety straps M2 has Ultra-Wideband Chip? Default destination for Messages Apple Jann Gobble IPSec is not mandatory for IPv6 (via Brian Petersen) Wikipedia Squozen AirPlay & Touch Bar network problems Mnpn Writeup Apple Patents Dual “Pro Stand” Casey’s stand EU may mandate USB-C on all devices, including iPhones Go Pack mophie 3-in-1 travel charger with MagSafe Proposal Q&A #askatp How do we manage mail notifications on our devices? (via Michael Chan) Mimestream Swift Mail (for Fastmail/JMAP) How do we backup our photos/videos? (via Don Clark) Do we think macOS will bloat enough to make Apple Silicon seem slow? (Nitesh Singh) Why does John say “program” from time to time? (via William Reynish) SwitchGlass How does a current-whippersnapper pick a decent custom domain for email? (via Fletcher O’Connor) Casey’s Fastmail referral code How does John prepare the way for a Mac to enter cold storage? (via Dave Braasch) Post-show: Sponsored by: Trade Coffee: Incredible coffee delivered fresh from the best roasters in the nation. Get $30 off your first order. RevenueCat: In-app subscriptions made easy. Squarespace: Make your next move. Use code atp for 10% off your first order. Become a member for ad-free episodes and our early-release, unedited “bootleg” feed! Become a member!

Pre-show: Two is one, one is none. Zoom F3 USBPre2 MixPre-3 Follow-up: Cleaning apps and “launch services” launchd Launch Control A hint more on Casey’s frivoLiss ethernet project Many recommendations for Monoprice slim 6A patch cables Jonathan Litt reminds us about fire stops PoE “medusa” cables exist, for USB-C and barrel plugs Why bother with ethernet at all? (via Andrew Larson) Millimeter-wave IPv6 Pi-Hole NAT On USB-C KVMs and the Studio Display (via Alex) Magic cable Demonstration (YouTube timestamp link) GSuite is now free for “personal use” Casey’s Fastmail referral link Apple Fitness+ “Studio Tour” iJustine Etalk KCAU-TV John on “filling your TV” Quinn Nelson’s Rivian and bad controls Quinn’s Instagram video (seek to 1h10m47s) 2001 Buick Apple Pre-Announces Accessibility Enhancements Shelly Brisbin’s take TextSniper ADB #askatp: Is there any bespoke music app for listening to concerts? (via Bill Steinbach) Live Phish Barrowclift’s iOS Music Player Showcase What Safari extensions are we running and recommend? (via Robert Bateau) John Safari Reload Button Safari Keyword Search Instapaper Downie Hush Fixarriffic NetNewsWire Marco 1Password 1Blocker Casey Noir (App Store link) StopTheMadness Super Agent Vinegar (App Store link) How does one remember how one enrolled in a service? (via Brian Hamilton) Ashley Bischoff’s idea Post-show: Casey’s Ethernet adventure continues Sponsored by: Squarespace: Make your next move. Use code atp for 10% off your first order. Trade Coffee: Incredible coffee delivered fresh from the best roasters in the nation. Get $30 off your first order. Linode: Instantly deploy and manage an SSD server in the Linode Cloud. New accounts get a $100 credit. Become a member for ad-free episodes and our early-release, unedited “bootleg” feed! Become a member!