Podcast appearances and mentions of Ben Silverman

Get the full inside scoop from YouTube's biggest event of the year! This week, Joshua Cohen reports directly from the Brandcast frontline, revealing Lady Gaga's show-stopping performance, game-changing ad updates for creators, and YouTube's bold vision for the future. Plus, major news on Mr. Beast, the NFL on YouTube, and industry-shaking acquisitions!On this episode of Creator Upload, we unpack all the critical announcements from YouTube Brandcast 2025. From the star-studded red carpet featuring MrBeast to new advertising tools like Peak Points and Cultural Moment Sponsorships, we cover what creators and marketers NEED to know. We also dive into the redesign of YouTube on Connected TVs, the launch of the official YouTube Podcast Charts, and what "There's Only One YouTube" really means for the ecosystem.Beyond Brandcast, we're talking about Mr. Beast's "Beast Games" getting renewed for two more seasons by Amazon AND his major expansion into the Middle East. Get the details on the upcoming NFL game (Chargers vs. Chiefs in Brazil) that will be live-streamed FOR FREE on YouTube. And finally, we break down significant moves in the creator management world as Ben Silverman's Propagate acquires Select Management & Parker Management, and Skybound Entertainment snaps up Nine Four Entertainment.Creator Upload is your creator economy podcast, hosted by Lauren Schnipper and Joshua Cohen.Follow Lauren: https://www.linkedin.com/in/schnipper/Follow Josh: https://www.linkedin.com/in/joshuajcohen/Original music by London Bridge: https://www.instagram.com/londonbridgemusic/Edited and produced by Adam Conner: https://www.linkedin.com/in/adamonbrand

Malcolm Ethridge of Capital Area Planning Group and Barbara Doran of BD8 Capital Partners break down the shortened-week's market action. Mark Mahaney of Evercore ISI offers a Wall Street rapid reaction perspective on Netflix, followed by broader entertainment industry insights from Ben Silverman, Propagate Content Chairman and former NBC Entertainment Co-Chairman. Former acting White House Chief of Staff Mick Mulvaney talks the President's relationship with Fed Chair Jerome Powell. Huntington Bancshares CEO Stephen Steinour joins to discuss earnings, the consumer, and more. Mizuho's Jared Holz breaks down major moves in the health care sector. 

Inflation, tariffs, recession fears, and major policy shifts. Today's market is clouded with uncertainty. But when good information is hard to come by, insider moves can provide clarity. Join Ben Silverman and Verity Senior Analyst Max Magee as they uncover how insider buying, selling, corporate buybacks, and executive compensation can help guide investors through volatile times.Tickers discussed: ORCL, JBHT, MPC, BAHIn this episode, Ben and Max explore:Oracle's unexpected insider purchase amid speculation of a TikTok deal and historical ties to government contracts.JB Hunt's logistics executive placing a substantial $10 million bet despite tariff uncertainties.Marathon Petroleum's insider action signaling optimism amid energy sector turbulence.Defense contractors like Booz Allen strategically responding to market headlines through insider buying.Edited, mixed, and scored by Calvin Marty.

Happy Christmas Eve, everybody! Can you believe tomorrow's the big day and that Santa's already set out on his annual trip!? We sure can't! To celebrate the occasion, please enjoy our annual Christmas Eve story, in which the elves read Brian Baumgartner's and Ben Silverman's "The Night Before Christmas at Dunder Mifflin"! Illustrated by Maël Gourmelen, this humorous retelling of "A Visit from St. Nicholas" is an elf-sized treat everyone can enjoy - not just fans of "The Office", though they will especially appreciate all of the callbacks and references to the iconic show!  So take a break from the wrapping, settle back with your favorite holiday drink beside your Christmas tree in order to bask in its glow, and enjoy!  As always, thanks for your love and support, y'all! Have a very Merry Christmas! We hope you have an amazing one, filled with food, drinks, laughter, and loved ones, and we hope that the big man in red brings you everything you asked for! We're looking forward to counting down to Christmas 2025 with all of y'all!

Each weekend on Best Of The Gist, we listen back to an archival Gist segment from the past, then we replay something from the past week. This weekend, we listen back to Mike's recent interview with Brian Baumgartner (Kevin from NBC's The Office) and Ben Silverman, who discuss their new book, The Night Before Christmas at Dunder Mifflin. Then, a special Christmas song.    SUBSCRIBE  We offer premium subscriptions, including an AD-FREE version of the show and options for bonus content.    The Gist is produced by Joel Patterson and Corey Wara    Email us at thegist@mikepesca.com  Subscribe to The Gist's YouTube Page  Follow Mike's Substack > Pesca Profundities  To advertise on the show, click here  Learn more about your ad choices. Visit podcastchoices.com/adchoices

The markets rallied after the 2024 election. CEOs made optimistic public statements. But behind the scenes, corporate insiders were selling at record rates. Why sell when stocks are soaring? What did these insiders see that others missed? And why such aggressive selling from companies poised to benefit from new policies? Join us for a close look at one of the market's most reliable signals: what insiders do with their own money.Ben breaks down four telling examples:Griffin Corp's CEO started with 80,000 shares at $70. Then he unloaded 300,000 more as the stock topped $78.GM's Cruise president waited until shares hit $55 - a three-year high - then sold.CoreCivic saw eleven insiders sell. Even a longtime buyer switched to selling.Wolverine Worldwide's HR chief broke her trading pattern just as policy impacts loomed.Edited, mixed, and scored by Calvin Marty.

In this episode, Nick and Steve are joined by Ben Silverman. Ben is a Program Director at E-Source, who is one of the founding members of the International Wildfire Risk Mitigation Consortium (IWRMC). They begin by discussing the mission of the IWRMC and the history behind its creation. Ben shares the structure they have established to ensure that their monthly conversations are collaborative and productive while navigating complex regulatory requirements. After sharing an overview of the four areas the IWRMC focuses on, Ben then gives a more in-depth look at how the group is looking at Vegetation Management. The conversation wraps up with a look forward to what is in store for the IWRMC and a look back at some of the group's accomplishments thus far. If you are interested in joining IWRMC, you can contact Ben at ben_silverman@esource.com To find out more about IWRMC, visit https://www.esource.com/public/international-wildfire-risk-mitigation-consortium 

From Content London 2024, Propagate Content's Ben Silverman talks to Irish comedian and TV host Patrick Kielty about his career and the present state of the international TV market [02:45]; and Amplify Pictures' Rachel Eggebeen, EndemolShine Australia's Sara Richardson and Blink Films' Justine Kershaw reflect on adapting to survive and thrive in The New Content Economy [34:07].

Ben Silverman was the Producer of 'The Office" and Brian Baumgartner played 'Kevin Malone." The two have teamed up once again, this time for a book. The book is called "The Night Before Christmas at Dunder Mifflin" and it's already a NY Times Bestseller. Lou & Dave spoke to the Ben and Brian about the book and all things 'The Office." Lou asked for a final ruling on Hillary Swank, Hot or Not.

Kyle Dunnigan returns to the show and they open by discussing Adam's ongoing battle against pepperoni pizza and why Laken Riley's accused killer should get the death penalty. Then, they test Byron's knowledge of American history, “Donald Trump” reacts to some critical clips of him, and “Sly Stallone” auditions for a commercial. Next, Jason “Mayhem” Miller joins to read the news including stories about Costco recalling 80k pounds of butter because the label didn't mention that it contained milk, the Washington Post releasing an article about how to immigrate to other countries in response to the election results, Dwayne Johnson admitting to peeing in bottles on set and being late to filming, and a Catholic woman who was fired for refusing the COVID vaccine winning a $12M settlement. Finally, from The Office, Brian Baumgartner (Kevin) and Ben Silverman (executive producer) stop by to talk about the ever-changing audience of The Office, how writing & performing comedy is much harder than drama, and their new holiday book “The Night Before Christmas at Dunder Mifflin.” For more with Kyle Dunnigan: ● YOUTUBE: youtube.com/@KyleDunniganComedy ● INSTAGRAM: @kyledunnigan1 ● TWITTER/X: @kyledunnigan For more with Brian Baumgartner: ● NEW BOOK: “The Night Before Christmas at Dunder Mifflin” - available now. ● INSTAGRAM, TIKTOK & X: @bbbaumgartner For more with Ben Silverman: ● NEW BOOK: “The Night Before Christmas at Dunder Mifflin” - available now. ● TWITTER/X: @notbensilverman Thank you for supporting our sponsors: ● http://SimpliSafe.com/Adam ● http://Meater.com ● http://OReillyAuto.com/Adam

John & Heidi share funny stories of people doing weird things... plus we continue our segment #AsSeenOnTV as John chats with Brian Baumgartner and Ben SilvermanLearn more about our radio program, podcast & blog at www.JohnAndHeidiShow.com

Yesterday we talked about our comfort TV shows and today we had one of the most loveable characters on TV on the phone - Brian Baumgartner a.k.a. Kevin from The Office! He's got a new book out 'The Night Before Christmas at Dunder Mifflin' along with producer Ben Silverman. The most revealing part of the interview - Brian lives here in San Diego! But where? We're going to figure it out!

Actor Brian Baumgartner and creator of The Office Ben Silverman joined Adam Schein to talk about their new book, "The Night Before Christmas At Dunder Mifflin," how long it took for "The Office" to catch on, meeting Aaron Rodgers, the Dodgers winning the World Series, and the Georgia Bulldogs. Adam & Bob Stew debate their top 5 for NFL League MVP after week 9.

Netflix shares rose in Overtime after reporting earnings. We have you covered from all the angles: closely-followed analyst Mark Mahaney on the stock move; Propagate Content Chairman and former NBC executive Ben Silverman breaks down the content strategy and an update on the broader streaming wars; DoubleVerify is one of Netflix's ad partners and its CEO discusses the streaming giant's ad strategy and how it plans to grow. Plus, CSX CEO Joe Hinrichs on the health of the economy from his perspective and Robinhood Chief Brokerage Officer Steve Quirk on the company's new Legend tool and the latest sentiment from the users. 

How Investors Outsmart Corporate Buybacks  Buybacks are bigger than ever. How can investors outsmart the pitfalls of corporate buybacks and find edge?   In this episode of Differentiated, host Ben Silverman dives deep into the nuances of corporate share buybacks with guest Michael Seigne from Candor Partners — whose insights have been featured in the Wall Street Journal, Financial Times, and more. Listen for essential insights into the execution of buybacks and the potential pitfalls that can arise, particularly around hidden costs, inefficiencies, and misaligned incentives. Through real-world examples, like those at General Motors, Microsoft, Apple, and more, discover the importance of smart, nuanced buyback strategies.  Tickers Discussed: GM, F, MSFT, AAPL, ASML, RTXEdited, mixed, and scored by Calvin Marty.

Summary:Ever wondered how some books skyrocket to national bestseller status while others fade into obscurity? Our latest episode has some answers!If you're aiming to take your book to the next level, this conversation holds a treasure trove of insights you won't want to miss. As our guest today, a fellow publishing expert with a track record of creating bestsellers, shares his invaluable expertise!In episode #164 of The Author's Corner, Kevin Anderson shares his journey from a small farm in northern Alberta to the heights of the publishing world. He offers invaluable advice on what it takes to create a bestseller, the importance of storytelling, and how to navigate the ever-evolving landscape of the publishing industry. Make sure to listen in for Kevin's expert tips on everything from manuscript preparation to marketing strategies that can propel your book to bestseller status.Key takeaways:Explore the intricate criteria determining various bestseller lists.Understand why some books make the cut while others don't.Discover the crucial role of sustained and consistent efforts in book promotion.Gain insights into the importance of having a unique angle and staying authentic to create stories that resonate widely.Learn about ethical versus unethical promotion strategies.And more!Resources mentioned in this episode:Kevin Anderson & Associates WebsiteThe Subtle Art of Not Giving a F*ck by Mark MansonAtomic Habits by James ClearPrinciples by Ray DalioAbout Kevin Anderson:Kevin Anderson is a #1 New York Times-bestselling editor, a #1 national-bestselling author, and an entrepreneur who has built multiple 7- and 8-figure editorial service businesses. His flagship company, Kevin Anderson & Associates, has helped produce more than 200 New York Times bestsellers and 600 national bestsellers, supporting a network of 500+ freelance writers and editors. KAA is responsible for launching the #1 New York Times bestselling YA series, Five Nights at Freddy's by Scott Cawthon, the New York Times bestseller, Welcome to Dunder Mifflin: The Ultimate Oral History of The Office by actor Brian Baumgartner and former NBC co-chairman Ben Silverman, and the WSJ bestseller Built, Not Born by multi-billionaire Tom Golisano, among many others.Spread the word:LinkedInTwitterInstagramFacebook

This week, we reviewed and reflected on one of the most infamous examples of fat representation in TV history - The Biggest Loser. What really went down during production? And what kind of an effect has the show had on our own communities? Who are we? We're James and Tim; two gainers who want to explore everything about gaining and feedism. New episodes will come out every Tuesday, so please subscribe! Rate us five stars, leave us a review, donate to support us and share this episode with your friends. You can find us on our socials below if you want to contact us, but until next time, bye fats! Review The Biggest Loser, created by Ben Silverman, Mark Koops, Dave Broome, and J.D. Roth James Instagram: s.t.a.n.n.u.m BlueSky: stannnum.bsky.social Tim Instagram: thickey_mouse Grommr: orpheus Twitter: thickey_mouse YouTube: thickey_mouse TikTok: thickey_mouse Special Guest | Sammy Instagram: substandardsammy2 Twitter: substandardsam Tumblr: substandardsammy Thicc Radio Instagram: thiccradio TikTok: thiccradio YouTube: thiccradio Website: ⁠⁠podpage.com/thiccradio/⁠⁠ Email: ⁠⁠thethiccradio@gmail.com --- Support this podcast: https://podcasters.spotify.com/pod/show/thiccradio/support

Discover insiders to watch this year. Learn the data points and analyses that flagged their activities as worthy of investor attention: wild sentiment reversal, high-IQ cluster buying, & more. Plus, Ben answers a smart question from a listener: Do insiders only buy when they think the stock is undervalued, or do they sometimes buy to try to get the stock moving? Tickers discussed: THR, SWKS, PLAY, APPN, LEVIEdited, mixed, and scored by Calvin Marty.

New SEC disclosures are giving investors edge. This episode, Ben shares how sweeping changes to rule 10b5-1 have played out in the last year, giving investors more data, context, and insights related to corporate insider activity. Learn material nuances of the rule change, what it means, and hear three real-world examples, including opportunistic activity from JPMorgan Chase (JPM) CEO Jaime Dimon, Rivian (RIVN) Founder RJ Scaringe, and Shockwave Medical (SWAV) CEO Doug Godshall.Edited, mixed, and scored by Calvin Marty.

Investors who can source conflict, mixed messages, or negative data points in their research process are at an advantage. In this episode, Ben shares a few hidden signals — revealed in insider selling activity, buyback programs, and earnings call sentiment — that can fuel short ideas. Examples include Snowflake (SNOW), Palo Alto Networks (PANW), PayPal (PYPL), and Skechers (SKX). Finally, Ben opens the mailbag and shares a recent question he got from a client regarding the opportunistic timing of one executive's resignation announcement.Edited, mixed, and scored by Calvin Marty.

How should investors interpret insider selling? There are a lot of data points: timing, size, past behavior, company culture, 10b5-1 context, and more. What's signal? What's noise?In this episode, Ben shares 6 truths about insider selling. He weaves in illustrative examples of recent insider selling from Meta Platforms (META) CEO Mark Zuckerberg, PlanetLabs (PL) CEO Ashley Johnson, Netflix (NFLX) CEO Spencer Newman, Build-a-Bear Workshop (BBW) CEO Sharon John, Former General Electric (GE) CEO Jeffrey Immelt, and more.Plus, Ben answers a great question he got recently from a VerityData hedge fund client: “If an insider targets a specific price, is that always bad?“Edited, mixed, and scored by Calvin Marty.

This month, Ben looks back at the year in insider buying and selling activity. From the long list of insiders active in 2023, Ben selects a shortlist of the best-performing execs and directors. These six insiders displayed well-timed and opportunistic behavior that should put them on the radar of investors wanting to generate ideas, manage risk, and have more edge in 2024.Edited, mixed, and scored by Calvin Marty.

What does a modern earnings workflow look like? This month, Ben and guests discuss the challenge of absorbing torrential amounts of information during earnings — from the press release to the 10-K/Q disclosure to the call transcript. Plus, go behind the scenes with the VerityData analyst developing genAI tools (including AI-generated transcript summaries) designed to help analysts stay on top of it all.Edited, mixed, and scored by Calvin Marty.

Montel talks with Kate Miller, co-founder and CEO of Miss Grass, on this episode of Let's be Blunt. Miss Grass is a brand on a mission to help the world get good at weed. Inspired by her stint working at a dispensary in college and powered by a decade-long career in entertainment where she worked alongside Ben Silverman and led brand partnerships for Lorne Micheals' Broadway Video, Kate is dedicated to rewriting the pervasive and shameful narrative around the plant and building a community of conscious cannabis consumers.

Ben and guests look back at a handful of this year's winning investment ideas and analyze intriguing ideas from the recent past. Plus, what does it mean when insiders at a company suddenly stop selling stock? Listen for the signals.Edited, mixed, and scored by Calvin Marty.

Act One Podcast - Episode 38 - Interview with Producers, Aaron Benward and Cliff Young.Aaron Benward comes to Watershed Motion Pictures by way of the music business where he started his career as one half of the award-winning father/son duo Aaron Jeoffrey. He followed that up as the founding member of the 3 time CMA and ACM nominated duo Blue County. Aaron's creative skills continue into his acting career where he can be seen recently in the Netflix Original series “The Ranch”, Sony's “The Song” and City on a Hill's “Acts of God.” He joined the Watershed team by packaging and negotiating a worldwide distribution deal for The Watershed Short Films Collection.Cliff Young began his career as a founding member of Caedmon's Call. The band made 16 albums, sold over two million records and toured 48 states over 15 years. Cliff also served on the board of the Dalit Freedom Network, which helps the “untouchables” of India. Cliff began working full time for Second Baptist Church in 2006 as the Media Director. Cliff oversees the worldwide broadcast The Winning Walk and also has produced commercials, documentaries, and short films for the past 11 years.GOD. FAMILY. FOOTBALL. features the rich, diverse personal stories of Evangel's players, coaching staff, and the broader Shreveport community, set against the dramatic backdrop of the 2022 Louisiana high school football season. With the perennial high school football powerhouse—14 state championships in the last 20 years—coming off their worst season in school history, redemption is everyone's goal. Pastor Denny Duron has returned to the head coaching position to lead this talented group of kids, with dreams of playing in college and the NFL, into prominence on the field, while molding them into future leaders off of it.  As the team faces struggle and triumph on and off the field, they are united by coach Duron's formula for success: “God first, family second, and football third.”GOD. FAMILY. FOOTBALL. hails from Propagate and is executive produced by Ben Silverman, Howard T. Owens, and Drew Buckley. The series was created and executive produced by Aaron Benward of Watershed. Jared Goetz of Ascending Media Group, NFL quarterback Russell Wilson in association with Why Not You Productions and Rob Gehring serve as executive producers. Cliff Young, Cody Bess, Scott Brignac, Chelsea Friedland and Matt Woolsey serve as co-executive producers.GOD. FAMILY. FOOTBALL. is available to stream beginning September 1st on Amazon's Freevee channel.Trailer: https://youtu.be/dqUkf2DikBA?si=uOHPX4bGr-2NlWZcThe Act One Podcast provides insight and inspiration on the business and craft of Hollywood from a Christian perspective.Support the show

John Swantek provides his take on the FedExCup Playoffs so far and gives his prediction of who will win the season-long race at this week's TOUR Championship.  Also, listen to Episode 2 of this season's Chasing TOURBound podcast with host James Nitties and Ben Silverman, winner of the 2023 Bahamas Great Abaco Classic at The Abaco Club.

Ben and guests dive into popular insider datasets — corporate buybacks, insider buying & selling, stock-based compensation — and share recent examples where multiple factors came together for a differentiated view. Plus, the errors of sell-side short research. Tickers mentioned: THC, MMS, IBP, CVGW, IMGNEdited, mixed, and scored by Calvin Marty.

Host James Nitties welcomes Ben Silverman as they discuss winning the pro-am portion with Aaron Rodgers at the 2023 AT&T Pebble Beach Pro-Am, his focus on the mental game, his goal of being the best Canadian player in golf history, and his excitement to make it to the PGA TOUR.

For the second time on the podcast, Brian gets to pick the brain of the brilliant Ben Silverman, visionary executive producer behind The Office, Ugly Betty, Jane the Virgin, The Biggest Loser, and many more. He talks about his childhood being raised by theater and television in Manhattan, the disappointment of attaining his childhood dream, breaking into new fields in media, and he even gives Brian some Japanese lessons.See omnystudio.com/listener for privacy information.

Ben and Senior Analyst Max Magee discuss benefits and drawbacks for investors who want to take advantage of recent changes to Form 144s. With the changes, investors are getting more data around insider intent to sell, but how should you include it in your mosaic?Edited, mixed, and scored by Calvin Marty.

Multihyphenate producer-director-cinematographer and all around talented guy Randall Einhorn is currently the executive producer and director of the award-winning ABC show, Abbott Elementary. Randall began his career in series television first as the DP of The Office, then became one of the most frequent directors of the series. He got to know the mockumentary style intimately, and carried it onto many other shows such as Parks and Recreation, The Muppets, and Modern Family. Quinta Brunson, show creator and star of Abbott Elementary, was a huge fan of The Office and pitched her idea to executive producers Randall Einhorn and Patrick Schumacker. Randall immediately knew that the mockumentary format would work well as they followed the everyday drama of teachers in an underprivileged elementary school in Philadelphia. They began shooting the pilot in August 2021, working with kids who were mostly non-actors and hadn't been inside a classroom for an entire year due to COVID. Working with kids made everything harder, but also made everything better, and Randall emphasized that they would have a good time every day. The children were so happy and excited to see each other and to be in a classroom, even if it was a set. On Abbott Elementary, Randall wanted the teachers to be treated like heroes, so they chose to use ARRI cameras and Angenieux Optimo Zoom lenses. The classrooms look inviting, with wood, warm earth tones and bright light coming in from the windows. By contrast, on The Office they would “dirty up” the frame to make it seem more spontaneous, as though something unexpected was actually caught. Randall would pan to someone, purposely defocus, then bring the actor into focus, to make it seem as though it was just caught. For Abbott Elementary, the camera crew keeps everything mostly in focus, but they will make a conscious effort to keep a piece of doorway in the shot, for example, to imply that people are having a private moment with the cameras hanging back. Randall feels that there's an honesty to using a long lens and backing up so it would look like the actors are having an intimate conversation. Randall naturally developed his mockumentary shooting style after working on reality and extreme sports shows. Executive producer Ben Silverman saw his work and thought his verite style would work well for The Office. Randall met with executive producer Greg Daniels, and they hit it off. Since he'd never worked on scripted shows before, Randall broke lots of rules that were considered “normal” for series television on The Office, such as operating himself and pulling his own focus. Blocking and planning the camera placement ahead of time was also essential- the camera crew would never put a camera where it couldn't or wouldn't be. He also figured out how to add to the improvisational comedy through the camera's movement and focus. Randall would keep one eye on the eyepiece and another on the actors to see who was going to improv. He'd lean in with the camera on an actor, stepping in closer to make a moment even more awkward. Unlike the British version of The Office, which was always carefully rehearsed, they would just shoot the scenes and reactions, in true documentary style. Randall's company, Sad Unicorn, has a multi-year first look deal at Warner Bros. and he will continue executive producing and directing Abbott Elementary. Abbot Elementary is in its second season on ABC and Hulu, and season three will likely be delayed due to the writers strike. Sponsored by Hot Rod Cameras: www.hotrodcameras.com Sponsored by Aputure: https://www.aputure.com/ The Cinematography Podcast website: www.camnoir.com YouTube: https://www.youtube.com/c/TheCinematographyPodcast Facebook: @cinepod Instagram: @thecinepod Twitter: @ShortEndz

Ben and Senior Analyst Ali Ragih, CFA, discuss how investors can take advantage of the new buyback data that will become available once SEC's new buyback disclosure requirements come into effect. Ben dives into a recent example where one company capitalized on its 10b5-1 plan, sending valuation signals to any investors who were paying attention.Edited, mixed, and scored by Calvin Marty.

Ben and Senior Analyst Ali Ragih talk about equity gifts by corporate insiders, why they are important transactions, and how to incorporate them into your analysis when screening for investment ideas. Plus, how banking insiders signaled with company stock purchases in the wake of the SVB Financial meltdown. Edited, mixed, and scored by Calvin Marty.

2023 is the year of Multimodal AI, and Latent Space is going multimodal too! * This podcast comes with a video demo at the 1hr mark and it's a good excuse to launch our YouTube - please subscribe! * We are also holding two events in San Francisco — the first AI | UX meetup next week (already full; we'll send a recap here on the newsletter) and Latent Space Liftoff Day on May 4th (signup here; but get in touch if you have a high profile launch you'd like to make). * We also joined the Chroma/OpenAI ChatGPT Plugins Hackathon last week where we won the Turing and Replit awards and met some of you in person!This post featured on Hacker News.Out of the five senses of the human body, I'd put sight at the very top. But weirdly when it comes to AI, Computer Vision has felt left out of the recent wave compared to image generation, text reasoning, and even audio transcription. We got our first taste of it with the OCR capabilities demo in the GPT-4 Developer Livestream, but to date GPT-4's vision capability has not yet been released. Meta AI leapfrogged OpenAI and everyone else by fully open sourcing their Segment Anything Model (SAM) last week, complete with paper, model, weights, data (6x more images and 400x more masks than OpenImages), and a very slick demo website. This is a marked change to their previous LLaMA release, which was not commercially licensed. The response has been ecstatic:SAM was the talk of the town at the ChatGPT Plugins Hackathon and I was fortunate enough to book Joseph Nelson who was frantically integrating SAM into Roboflow this past weekend. As a passionate instructor, hacker, and founder, Joseph is possibly the single best person in the world to bring the rest of us up to speed on the state of Computer Vision and the implications of SAM. I was already a fan of him from his previous pod with (hopefully future guest) Beyang Liu of Sourcegraph, so this served as a personal catchup as well. Enjoy! and let us know what other news/models/guests you'd like to have us discuss! - swyxRecorded in-person at the beautiful StudioPod studios in San Francisco.Full transcript is below the fold.Show Notes* Joseph's links: Twitter, Linkedin, Personal* Sourcegraph Podcast and Game Theory Story* Represently* Roboflow at Pioneer and YCombinator* Udacity Self Driving Car dataset story* Computer Vision Annotation Formats* SAM recap - top things to know for those living in a cave* https://segment-anything.com/* https://segment-anything.com/demo* https://arxiv.org/pdf/2304.02643.pdf * https://ai.facebook.com/blog/segment-anything-foundation-model-image-segmentation/* https://blog.roboflow.com/segment-anything-breakdown/* https://ai.facebook.com/datasets/segment-anything/* Ask Roboflow https://ask.roboflow.ai/* GPT-4 Multimodal https://blog.roboflow.com/gpt-4-impact-speculation/Cut for time:* WSJ mention* Des Moines Register story* All In Pod: timestamped mention* In Forbes: underrepresented investors in Series A* Roboflow greatest hits* https://blog.roboflow.com/mountain-dew-contest-computer-vision/* https://blog.roboflow.com/self-driving-car-dataset-missing-pedestrians/* https://blog.roboflow.com/nerualhash-collision/ and Apple CSAM issue * https://www.rf100.org/Timestamps* [00:00:19] Introducing Joseph* [00:02:28] Why Iowa* [00:05:52] Origin of Roboflow* [00:16:12] Why Computer Vision* [00:17:50] Computer Vision Use Cases* [00:26:15] The Economics of Annotation/Segmentation* [00:32:17] Computer Vision Annotation Formats* [00:36:41] Intro to Computer Vision & Segmentation* [00:39:08] YOLO* [00:44:44] World Knowledge of Foundation Models* [00:46:21] Segment Anything Model* [00:51:29] SAM: Zero Shot Transfer* [00:51:53] SAM: Promptability* [00:53:24] SAM: Model Assisted Labeling* [00:56:03] SAM doesn't have labels* [00:59:23] Labeling on the Browser* [01:00:28] Roboflow + SAM Video Demo * [01:07:27] Future Predictions* [01:08:04] GPT4 Multimodality* [01:09:27] Remaining Hard Problems* [01:13:57] Ask Roboflow (2019)* [01:15:26] How to keep up in AITranscripts[00:00:00] Hello everyone. It is me swyx and I'm here with Joseph Nelson. Hey, welcome to the studio. It's nice. Thanks so much having me. We, uh, have a professional setup in here.[00:00:19] Introducing Joseph[00:00:19] Joseph, you and I have known each other online for a little bit. I first heard about you on the Source Graph podcast with bian and I highly, highly recommend that there's a really good game theory story that is the best YC application story I've ever heard and I won't tease further cuz they should go listen to that.[00:00:36] What do you think? It's a good story. It's a good story. It's a good story. So you got your Bachelor of Economics from George Washington, by the way. Fun fact. I'm also an econ major as well. You are very politically active, I guess you, you did a lot of, um, interning in political offices and you were responding to, um, the, the, the sheer amount of load that the Congress people have in terms of the, the support.[00:01:00] So you built, representing, which is Zendesk for Congress. And, uh, I liked in your source guide podcast how you talked about how being more responsive to, to constituents is always a good thing no matter what side of the aisle you're on. You also had a sideline as a data science instructor at General Assembly.[00:01:18] As a consultant in your own consultancy, and you also did a bunch of hackathon stuff with Magic Sudoku, which is your transition from N L P into computer vision. And apparently at TechCrunch Disrupt, disrupt in 2019, you tried to add chess and that was your whole villain origin story for, Hey, computer vision's too hard.[00:01:36] That's full, the platform to do that. Uh, and now you're co-founder c e o of RoboFlow. So that's your bio. Um, what's not in there that[00:01:43] people should know about you? One key thing that people realize within maybe five minutes of meeting me, uh, I'm from Iowa. Yes. And it's like a funnily novel thing. I mean, you know, growing up in Iowa, it's like everyone you know is from Iowa.[00:01:56] But then when I left to go to school, there was not that many Iowans at gw and people were like, oh, like you're, you're Iowa Joe. Like, you know, how'd you find out about this school out here? I was like, oh, well the Pony Express was running that day, so I was able to send. So I really like to lean into it.[00:02:11] And so you kind of become a default ambassador for places that. People don't meet a lot of other people from, so I've kind of taken that upon myself to just make it be a, a part of my identity. So, you know, my handle everywhere Joseph of Iowa, like I I, you can probably find my social security number just from knowing that that's my handle.[00:02:25] Cuz I put it plastered everywhere. So that's, that's probably like one thing.[00:02:28] Why Iowa[00:02:28] What's your best pitch for Iowa? Like why is[00:02:30] Iowa awesome? The people Iowa's filled with people that genuinely care. You know, if you're waiting a long line, someone's gonna strike up a conversation, kinda ask how you were Devrel and it's just like a really genuine place.[00:02:40] It was a wonderful place to grow up too at the time, you know, I thought it was like, uh, yeah, I was kind of embarrassed and then be from there. And then I actually kinda looking back it's like, wow, you know, there's good schools, smart people friendly. The, uh, high school that I went to actually Ben Silverman, the CEO and, or I guess former CEO and co-founder of Pinterest and I have the same teachers in high school at different.[00:03:01] The co-founder, or excuse me, the creator of crispr, the gene editing technique, Dr. Jennifer. Doudna. Oh, so that's the patent debate. There's Doudna. Oh, and then there's Fang Zang. Uh, okay. Yeah. Yeah. So Dr. Fang Zang, who I think ultimately won the patent war, uh, but is also from the same high school.[00:03:18] Well, she won the patent, but Jennifer won the[00:03:20] prize.[00:03:21] I think that's probably, I think that's probably, I, I mean I looked into it a little closely. I think it was something like she won the patent for CRISPR first existing and then Feng got it for, uh, first use on humans, which I guess for commercial reasons is the, perhaps more, more interesting one. But I dunno, biolife Sciences, is that my area of expertise?[00:03:38] Yep. Knowing people that came from Iowa that do cool things, certainly is. Yes. So I'll claim it. Um, but yeah, I, I, we, um, at Roble actually, we're, we're bringing the full team to Iowa for the very first time this last week of, of April. And, well, folks from like Scotland all over, that's your company[00:03:54] retreat.[00:03:54] The Iowa,[00:03:55] yeah. Nice. Well, so we do two a year. You know, we've done Miami, we've done. Some of the smaller teams have done like Nashville or Austin or these sorts of places, but we said, you know, let's bring it back to kinda the origin and the roots. Uh, and we'll, we'll bring the full team to, to Des Moines, Iowa.[00:04:13] So, yeah, like I was mentioning, folks from California to Scotland and many places in between are all gonna descend upon Des Moines for a week of, uh, learning and working. So maybe you can check in with those folks. If, what do they, what do they decide and interpret about what's cool. Our state. Well, one thing, are you actually headquartered in Des Moines on paper?[00:04:30] Yes. Yeah.[00:04:30] Isn't that amazing? That's like everyone's Delaware and you're like,[00:04:33] so doing research. Well, we're, we're incorporated in Delaware. Okay. We we're Delaware Sea like, uh, most companies, but our headquarters Yeah. Is in Des Moines. And part of that's a few things. One, it's like, you know, there's this nice Iowa pride.[00:04:43] And second is, uh, Brad and I both grew up in Brad Mc, co-founder and I grew up in, in Des Moines. And we met each other in the year 2000. We looked it up for the, the YC app. So, you know, I think, I guess more of my life I've known Brad than not, uh, which is kind of crazy. Wow. And during yc, we did it during 2020, so it was like the height of Covid.[00:05:01] And so we actually got a house in Des Moines and lived, worked outta there. I mean, more credit to. So I moved back. I was living in DC at the time, I moved back to to Des Moines. Brad was living in Des Moines, but he moved out of a house with his. To move into what we called our hacker house. And then we had one, uh, member of the team as well, Jacob Sorowitz, who moved from Minneapolis down to Des Moines for the summer.[00:05:21] And frankly, uh, code was a great time to, to build a YC company cuz there wasn't much else to do. I mean, it's kinda like wash your groceries and code. It's sort of the, that was the routine[00:05:30] and you can use, uh, computer vision to help with your groceries as well.[00:05:33] That's exactly right. Tell me what to make.[00:05:35] What's in my fridge? What should I cook? Oh, we'll, we'll, we'll cover[00:05:37] that for with the G P T four, uh, stuff. Exactly. Okay. So you have been featured with in a lot of press events. Uh, but maybe we'll just cover the origin story a little bit in a little bit more detail. So we'll, we'll cover robo flow and then we'll cover, we'll go into segment anything.[00:05:52] Origin of Roboflow[00:05:52] But, uh, I think it's important for people to understand. Robo just because it gives people context for what you're about to show us at the end of the podcast. So Magic Sudoku tc, uh, techers Disrupt, and then you go, you join Pioneer, which is Dan Gross's, um, YC before yc.[00:06:07] Yeah. That's how I think about it.[00:06:08] Yeah, that's a good way. That's a good description of it. Yeah. So I mean, robo flow kind of starts as you mentioned with this magic Sudoku thing. So you mentioned one of my prior business was a company called Represent, and you nailed it. I mean, US Congress gets 80 million messages a year. We built tools that auto sorted them.[00:06:23] They didn't use any intelligent auto sorting. And this is somewhat a solved problem in natural language processing of doing topic modeling or grouping together similar sentiment and things like this. And as you mentioned, I'd like, I worked in DC for a bit and been exposed to some of these problems and when I was like, oh, you know, with programming you can build solutions.[00:06:40] And I think the US Congress is, you know, the US kind of United States is a support center, if you will, and the United States is sports center runs on pretty old software, so mm-hmm. We, um, we built a product for that. It was actually at the time when I was working on representing. Brad, his prior business, um, is a social games company called Hatchlings.[00:07:00] Uh, he phoned me in, in 2017, apple had released augmented reality kit AR kit. And Brad and I are both kind of serial hackers, like I like to go to hackathons, don't really understand new technology until he build something with them type folks. And when AR Kit came out, Brad decided he wanted to build a game with it that would solve Sudoku puzzles.[00:07:19] And the idea of the game would be you take your phone, you hover hold it over top of a Sudoku puzzle, it recognizes the state of the board where it is, and then it fills it all in just right before your eyes. And he phoned me and I was like, Brad, this sounds awesome and sounds like you kinda got it figured out.[00:07:34] What, what's, uh, what, what do you think I can do here? It's like, well, the machine learning piece of this is the part that I'm most uncertain about. Uh, doing the digit recognition and, um, filling in some of those results. I was like, well, I mean digit recognition's like the hell of world of, of computer vision.[00:07:48] That's Yeah, yeah, MNIST, right. So I was like, that that part should be the, the easy part. I was like, ah, I'm, he's like, I'm not so super sure, but. You know, the other parts, the mobile ar game mechanics, I've got pretty well figured out. I was like, I, I think you're wrong. I think you're thinking about the hard part is the easy part.[00:08:02] And he is like, no, you're wrong. The hard part is the easy part. And so long story short, we built this thing and released Magic Sudoku and it kind of caught the Internet's attention of what you could do with augmented reality and, and with computer vision. It, you know, made it to the front ofer and some subreddits it run Product Hunt Air app of the year.[00:08:20] And it was really a, a flash in the pan type app, right? Like we were both running separate companies at the time and mostly wanted to toy around with, with new technology. And, um, kind of a fun fact about Magic Sudoku winning product Hunt Air app of the year. That was the same year that I think the model three came out.[00:08:34] And so Elon Musk won a Golden Kitty who we joked that we share an award with, with Elon Musk. Um, the thinking there was that this is gonna set off a, a revolution of if two random engineers can put together something that makes something, makes a game programmable and at interactive, then surely lots of other engineers will.[00:08:53] Do similar of adding programmable layers on top of real world objects around us. Earlier we were joking about objects in your fridge, you know, and automatically generating recipes and these sorts of things. And like I said, that was 2017. Roboflow was actually co-found, or I guess like incorporated in, in 2019.[00:09:09] So we put this out there, nothing really happened. We went back to our day jobs of, of running our respective businesses, I sold Represently and then as you mentioned, kind of did like consulting stuff to figure out the next sort of thing to, to work on, to get exposed to various problems. Brad appointed a new CEO at his prior business and we got together that summer of 2019.[00:09:27] We said, Hey, you know, maybe we should return to that idea that caught a lot of people's attention and shows what's possible. And you know what, what kind of gives, like the future is here. And we have no one's done anything since. No one's done anything. So why is, why are there not these, these apps proliferated everywhere.[00:09:42] Yeah. And so we said, you know, what we'll do is, um, to add this software layer to the real world. Will build, um, kinda like a super app where if you pointed it at anything, it will recognize it and then you can interact with it. We'll release a developer platform and allow people to make their own interfaces, interactivity for whatever object they're looking at.[00:10:04] And we decided to start with board games because one, we had a little bit of history there with, with Sudoku two, there's social by default. So if one person, you know finds it, then they'd probably share it among their friend. Group three. There's actually relatively few barriers to entry aside from like, you know, using someone else's brand name in your, your marketing materials.[00:10:19] Yeah. But other than that, there's no real, uh, inhibitors to getting things going and, and four, it's, it's just fun. It would be something that'd be bring us enjoyment to work on. So we spent that summer making, uh, boggle the four by four word game provable, where, you know, unlike Magic Sudoku, which to be clear, totally ruins the game, uh, you, you have to solve Sudoku puzzle.[00:10:40] You don't need to do anything else. But with Boggle, if you and I are playing, we might not find all of the words that adjacent letter tiles. Unveil. So if we have a, an AI tell us, Hey, here's like the best combination of letters that make high scoring words. And so we, we made boggle and released it and that, and that did okay.[00:10:56] I mean maybe the most interesting story was there's a English as a second language program in, in Canada that picked it up and used it as a part of their curriculum to like build vocabulary, which I thought was kind of inspiring. Example, and what happens just when you put things on the internet and then.[00:11:09] We wanted to build one for chess. So this is where you mentioned we went to 2019. TechCrunch Disrupt TechCrunch. Disrupt holds a Hackathon. And this is actually, you know, when Brad and I say we really became co-founders, because we fly out to San Francisco, we rent a hotel room in the Tenderloin. We, uh, we, we, uh, have one room and there's like one, there's room for one bed, and then we're like, oh, you said there was a cot, you know, on the, on the listing.[00:11:32] So they like give us a little, a little cot, the end of the cot, like bled and over into like the bathroom. So like there I am sleeping on the cot with like my head in the bathroom and the Tenderloin, you know, fortunately we're at a hackathon glamorous. Yeah. There wasn't, there wasn't a ton of sleep to be had.[00:11:46] There is, you know, we're, we're just like making and, and shipping these, these sorts of many[00:11:50] people with this hack. So I've never been to one of these things, but[00:11:52] they're huge. Right? Yeah. The Disrupt Hackathon, um, I don't, I don't know numbers, but few hundreds, you know, classically had been a place where it launched a lot of famous Yeah.[00:12:01] Sort of flare. Yeah. And I think it's, you know, kind of slowed down as a place for true company generation. But for us, Brad and I, who likes just doing hackathons, being, making things in compressed time skills, it seemed like a, a fun thing to do. And like I said, we'd been working on things, but it was only there that like, you're, you're stuck in a maybe not so great glamorous situation together and you're just there to make a, a program and you wanna make it be the best and compete against others.[00:12:26] And so we add support to the app that we were called was called Board Boss. We couldn't call it anything with Boggle cause of IP rights were called. So we called it Board Boss and it supported Boggle and then we were gonna support chess, which, you know, has no IP rights around it. Uh, it's an open game.[00:12:39] And we did so in 48 hours, we built an app that, or added fit capability to. Point your phone at a chess board. It understands the state of the chess board and converts it to um, a known notation. Then it passes that to stock fish, the open source chess engine for making move recommendations and it makes move recommendations to, to players.[00:13:00] So you could either play against like an ammunition to AI or improve your own game. We learn that one of the key ways users like to use this was just to record their games. Cuz it's almost like reviewing game film of what you should have done differently. Game. Yeah, yeah, exactly. And I guess the highlight of, uh, of chess Boss was, you know, we get to the first round of judging, we get to the second round of judging.[00:13:16] And during the second round of judging, that's when like, TechCrunch kind of brings around like some like celebs and stuff. They'll come by. Evan Spiegel drops by Ooh. Oh, and he uh, he comes up to our, our, our booth and um, he's like, oh, so what does, what does this all do? And you know, he takes an interest in it cuz the underpinnings of, of AR interacting with the.[00:13:33] And, uh, he is kinda like, you know, I could use this to like cheat on chess with my friends. And we're like, well, you know, that wasn't exactly the, the thesis of why we made it, but glad that, uh, at least you think it's kind of neat. Um, wait, but he already started Snapchat by then? Oh, yeah. Oh yeah. This, this is 2019, I think.[00:13:49] Oh, okay, okay. Yeah, he was kind of just checking out things that were new and, and judging didn't end up winning any, um, awards within Disrupt, but I think what we won was actually. Maybe more important maybe like the, the quote, like the co-founders medal along the way. Yep. The friends we made along the way there we go to, to play to the meme.[00:14:06] I would've preferred to win, to be clear. Yes. You played a win. So you did win, uh,[00:14:11] $15,000 from some Des Moines, uh, con[00:14:14] contest. Yeah. Yeah. The, uh, that was nice. Yeah. Slightly after that we did, we did win. Um, some, some grants and some other things for some of the work that we've been doing. John Papa John supporting the, uh, the local tech scene.[00:14:24] Yeah. Well, so there's not the one you're thinking of. Okay. Uh, there's a guy whose name is Papa John, like that's his, that's his, that's his last name. His first name is John. So it's not the Papa John's you're thinking of that has some problematic undertones. It's like this guy who's totally different. I feel bad for him.[00:14:38] His press must just be like, oh, uh, all over the place. But yeah, he's this figure in the Iowa entrepreneurial scene who, um, he actually was like doing SPACs before they were cool and these sorts of things, but yeah, he funds like grants that encourage entrepreneurship in the state. And since we'd done YC and in the state, we were eligible for some of the awards that they were providing.[00:14:56] But yeah, it was disrupt that we realized, you know, um, the tools that we made, you know, it took us better part of a summer to add Boggle support and it took us 48 hours to add chest support. So adding the ability for programmable interfaces for any object, we built a lot of those internal tools and our apps were kind of doing like the very famous shark fin where like it picks up really fast, then it kind of like slowly peters off.[00:15:20] Mm-hmm. And so we're like, okay, if we're getting these like shark fin graphs, we gotta try something different. Um, there's something different. I remember like the week before Thanksgiving 2019 sitting down and we wrote this Readme for, actually it's still the Readme at the base repo of Robo Flow today has spent relatively unedited of the manifesto.[00:15:36] Like, we're gonna build tools that enable people to make the world programmable. And there's like six phases and, you know, there's still, uh, many, many, many phases to go into what we wrote even at that time to, to present. But it's largely been, um, right in line with what we thought we would, we would do, which is give engineers the tools to add software to real world objects, which is largely predicated on computer vision. So finding the right images, getting the right sorts of video frames, maybe annotating them, uh, finding the right sort of models to use to do this, monitoring the performance, all these sorts of things. And that from, I mean, we released that in early 2020, and it's kind of, that's what's really started to click.[00:16:12] Why Computer Vision[00:16:12] Awesome. I think we should just kind[00:16:13] of[00:16:14] go right into where you are today and like the, the products that you offer, just just to give people an overview and then we can go into the, the SAM stuff. So what is the clear, concise elevator pitch? I think you mentioned a bunch of things like make the world programmable so you don't ha like computer vision is a means to an end.[00:16:30] Like there's, there's something beyond that. Yeah.[00:16:32] I mean, the, the big picture mission for the business and the company and what we're working on is, is making the world programmable, making it read and write and interactive, kind of more entertaining, more e. More fun and computer vision is the technology by which we can achieve that pretty quickly.[00:16:48] So like the one liner for the, the product in, in the company is providing engineers with the tools for data and models to build programmable interfaces. Um, and that can be workflows, that could be the, uh, data processing, it could be the actual model training. But yeah, Rob helps you use production ready computer vision workflows fast.[00:17:10] And I like that.[00:17:11] In part of your other pitch that I've heard, uh, is that you basically scale from the very smallest scales to the very largest scales, right? Like the sort of microbiology use case all the way to[00:17:20] astronomy. Yeah. Yeah. The, the joke that I like to make is like anything, um, underneath a microscope and, and through a telescope and everything in between needs to, needs to be seen.[00:17:27] I mean, we have people that run models in outer space, uh, underwater remote places under supervision and, and known places. The crazy thing is that like, All parts of, of not just the world, but the universe need to be observed and understood and acted upon. So vision is gonna be, I dunno, I feel like we're in the very, very, very beginnings of all the ways we're gonna see it.[00:17:50] Computer Vision Use Cases[00:17:50] Awesome. Let's go into a lo a few like top use cases, cuz I think that really helps to like highlight the big names that you've, big logos that you've already got. I've got Walmart and Cardinal Health, but I don't, I don't know if you wanna pull out any other names, like, just to illustrate, because the reason by the way, the reason I think that a lot of developers don't get into computer vision is because they think they don't need it.[00:18:11] Um, or they think like, oh, like when I do robotics, I'll do it. But I think if, if you see like the breadth of use cases, then you get a little bit more inspiration as to like, oh, I can use[00:18:19] CVS lfa. Yeah. It's kind of like, um, you know, by giving, by making it be so straightforward to use vision, it becomes almost like a given that it's a set of features that you could power on top of it.[00:18:32] And like you mentioned, there's, yeah, there's Fortune One there over half the Fortune 100. I've used the, the tools that Robel provides just as much as 250,000 developers. And so over a quarter million engineers finding and developing and creating various apps, and I mean, those apps are, are, are far and wide.[00:18:49] Just as you mentioned. I mean everything from say, like, one I like to talk about was like sushi detection of like finding the like right sorts of fish and ingredients that are in a given piece of, of sushi that you're looking at to say like roof estimation of like finding. If there's like, uh, hail damage on, on a given roof, of course, self-driving cars and understanding the scenes around us is sort of the, you know, very early computer vision everywhere.[00:19:13] Use case hardhat detection, like finding out if like a given workplace is, is, is safe, uh, disseminate, have the right p p p on or p p e on, are there the right distance from various machines? A huge place that vision has been used is environmental monitoring. Uh, what's the count of species? Can we verify that the environment's not changing in unexpected ways or like river banks are become, uh, becoming recessed in ways that we anticipate from satellite imagery, plant phenotyping.[00:19:37] I mean, people have used these apps for like understanding their plants and identifying them. And that dataset that's actually largely open, which is what's given a proliferation to the iNaturalist, is, is that whole, uh, hub of, of products. Lots of, um, people that do manufacturing. So, like Rivian for example, is a Rubal customer, and you know, they're trying to scale from 1000 cars to 25,000 cars to a hundred thousand cars in very short order.[00:20:00] And that relies on having the. Ability to visually ensure that every part that they're making is produced correctly and right in time. Medical use cases. You know, there's actually, this morning I was emailing with a user who's accelerating early cancer detection through breaking apart various parts of cells and doing counts of those cells.[00:20:23] And actually a lot of wet lab work that folks that are doing their PhDs or have done their PhDs are deeply familiar with that is often required to do very manually of, of counting, uh, micro plasms or, or things like this. There's. All sorts of, um, like traffic counting and smart cities use cases of understanding curb utilization to which sort of vehicles are, are present.[00:20:44] Uh, ooh. That can be[00:20:46] really good for city planning actually.[00:20:47] Yeah. I mean, one of our customers does exactly this. They, they measure and do they call it like smart curb utilization, where uhhuh, they wanna basically make a curb be almost like a dynamic space where like during these amounts of time, it's zoned for this during these amounts of times.[00:20:59] It's zoned for this based on the flows and e ebbs and flows of traffic throughout the day. So yeah, I mean the, the, the truth is that like, you're right, it's like a developer might be like, oh, how would I use vision? And then all of a sudden it's like, oh man, all these things are at my fingertips. Like I can just, everything you can see.[00:21:13] Yeah. Right. I can just, I can just add functionality for my app to understand and ingest the way, like, and usually the way that someone gets like almost nerd sniped into this is like, they have like a home automation project, so it's like send Yeah. Give us a few. Yeah. So send me a text when, um, a package shows up so I can like prevent package theft so I can like go down and grab it right away or.[00:21:29] We had a, uh, this one's pretty, pretty niche, but it's pretty funny. There was this guy who, during the pandemic wa, wanted to make sure his cat had like the proper, uh, workout. And so I've shared the story where he basically decided that. He'd make a cat workout machine with computer vision, you might be alone.[00:21:43] You're like, what does that look like? Well, what he decided was he would take a robotic arm strap, a laser pointer to it, and then train a machine to recognize his cat and his cat only, and point the laser pointer consistently 10 feet away from the cat. There's actually a video of you if you type an YouTube cat laser turret, you'll find Dave's video.[00:22:01] Uh, and hopefully Dave's cat has, has lost the weight that it needs to, cuz that's just the, that's an intense workout I have to say. But yeah, so like, that's like a, um, you know, these, uh, home automation projects are pretty common places for people to get into smart bird feeders. I've seen people that like are, are logging and understanding what sort of birds are, uh, in their background.[00:22:18] There's a member of our team that was working on actually this as, as a whole company and has open sourced a lot of the data for doing bird species identification. And now there's, I think there's even a company that's, uh, founded to create like a smart bird feeder, like captures photos and tells you which ones you've attracted to your yard.[00:22:32] I met that. Do, you know, get around the, uh, car sharing company that heard it? Them never used them. They did a SPAC last year and they had raised at like, They're unicorn. They raised at like 1.2 billion, I think in the, the prior round and inspected a similar price. I met the CTO of, of Getaround because he was, uh, using Rob Flow to hack into his Tesla cameras to identify other vehicles that are like often nearby him.[00:22:56] So he's basically building his own custom license plate recognition, and he just wanted like, keep, like, keep tabs of like, when he drives by his friends or when he sees like regular sorts of folks. And so he was doing like automated license plate recognition by tapping into his, uh, camera feeds. And by the way, Elliot's like one of the like OG hackers, he was, I think one of the very first people to like, um, she break iPhones and, and these sorts of things.[00:23:14] Mm-hmm. So yeah, the project that I want, uh, that I'm gonna work on right now for my new place in San Francisco is. There's two doors. There's like a gate and then the other door. And sometimes we like forget to close, close the gate. So like, basically if it sees that the gate is open, it'll like send us all a text or something like this to make sure that the gate is, is closed at the front of our house.[00:23:32] That's[00:23:32] really cool. And I'll, I'll call out one thing that readers and listeners can, uh, read out on, on your history. One of your most popular initial, um, viral blog post was about, um, autonomous vehicle data sets and how, uh, the one that Udacity was using was missing like one third of humans. And, uh, it's not, it's pretty problematic for cars to miss humans.[00:23:53] Yeah, yeah, actually, so yeah, the Udacity self-driving car data set, which look to their credit, it was just meant to be used for, for academic use. Um, and like as a part of courses on, on Udacity, right? Yeah. But the, the team that released it, kind of hastily labeled and let it go out there to just start to use and train some models.[00:24:11] I think that likely some, some, uh, maybe commercial use cases maybe may have come and, and used, uh, the dataset, who's to say? But Brad and I discovered this dataset. And when we were working on dataset improvement tools at Rob Flow, we ran through our tools and identified some like pretty, as you mentioned, key issues.[00:24:26] Like for example, a lot of strollers weren't labeled and I hope our self-driving cars do those, these sorts of things. And so we relabeled the whole dataset by hand. I have this very fond memory is February, 2020. Brad and I are in Taiwan. So like Covid is actually just, just getting going. And the reason we were there is we were like, Hey, we can work on this from anywhere for a little bit.[00:24:44] And so we spent like a, uh, let's go closer to Covid. Well, you know, I like to say we uh, we got early indicators of, uh, how bad it was gonna be. I bought a bunch of like N 90 fives before going o I remember going to the, the like buying a bunch of N 95 s and getting this craziest look like this like crazy tin hat guy.[00:25:04] Wow. What is he doing? And then here's how you knew. I, I also got got by how bad it was gonna be. I left all of them in Taiwan cuz it's like, oh, you all need these. We'll be fine over in the us. And then come to find out, of course that Taiwan was a lot better in terms of, um, I think, yeah. Safety. But anyway, we were in Taiwan because we had planned this trip and you know, at the time we weren't super sure about the, uh, covid, these sorts of things.[00:25:22] We always canceled it. We didn't, but I have this, this very specific time. Brad and I were riding on the train from Clay back to Taipei. It's like a four hour ride. And you mentioned Pioneer earlier, we were competing in Pioneer, which is almost like a gamified to-do list. Mm-hmm. Every week you say what you're gonna do and then other people evaluate.[00:25:37] Did you actually do the things you said you were going to do? One of the things we said we were gonna do was like this, I think re-release of this data set. And so it's like late, we'd had a whole week, like, you know, weekend behind us and, uh, we're on this train and it was very unpleasant situation, but we relabeled this, this data set, and one sitting got it submitted before like the Sunday, Sunday countdown clock starts voting for, for.[00:25:57] And, um, once that data got out back out there, just as you mentioned, it kind of picked up and Venture beat, um, noticed and wrote some stories about it. And we really rereleased of course, the data set that we did our best job of labeling. And now if anyone's listening, they can probably go out and like find some errors that we surely still have and maybe call us out and, you know, put us, put us on blast.[00:26:15] The Economics of Annotation (Segmentation)[00:26:15] But,[00:26:16] um, well, well the reason I like this story is because it, it draws attention to the idea that annotation is difficult and basically anyone looking to use computer vision in their business who may not have an off-the-shelf data set is going to have to get involved in annotation. And I don't know what it costs.[00:26:34] And that's probably one of the biggest hurdles for me to estimate how big a task this is. Right? So my question at a higher level is tell the customers, how do you tell customers to estimate the economics of annotation? Like how many images do, do we need? How much, how long is it gonna take? That, that kinda stuff.[00:26:50] How much money and then what are the nuances to doing it well, right? Like, cuz obviously Udacity had a poor quality job, you guys had proved it, and there's errors every everywhere. Like where do[00:26:59] these things go wrong? The really good news about annotation in general is that like annotation of course is a means to an end to have a model be able to recognize a thing.[00:27:08] Increasingly there's models that are coming out that can recognize things zero shot without any annotation, which we're gonna talk about. Yeah. Which, we'll, we'll talk more about that in a moment. But in general, the good news is that like the trend is that annotation is gonna become decreasingly a blocker to starting to use computer vision in meaningful ways.[00:27:24] Now that said, just as you mentioned, there's a lot of places where you still need to do. Annotation. I mean, even with these zero shot models, they might have of blind spots, or maybe you're a business, as you mentioned, that you know, it's proprietary data. Like only Rivian knows what a rivian is supposed to look like, right?[00:27:39] Uh, at the time of, at the time of it being produced, like underneath the hood and, and all these sorts of things. And so, yeah, that's gonna necessarily require annotation. So your question of how long is it gonna take, how do you estimate these sorts of things, it really comes down to the complexity of the problem that you're solving and the amount of variance in the scene.[00:27:57] So let's give some contextual examples. If you're trying to recognize, we'll say a scratch on one specific part and you have very strong lighting. You might need fewer images because you control the lighting, you know the exact part and maybe you're lucky in the scratch. Happens more often than not in similar parts or similar, uh, portions of the given part.[00:28:17] So in that context, you, you, the function of variance, the variance is, is, is lower. So the number of images you need is also lower to start getting up to work. Now the orders of magnitude we're talking about is that like you can have an initial like working model from like 30 to 50 images. Yeah. In this context, which is shockingly low.[00:28:32] Like I feel like there's kind of an open secret in computer vision now, the general heuristic that often. Users, is that like, you know, maybe 200 images per class is when you start to have a model that you can rely[00:28:45] on? Rely meaning like 90, 99, 90, 90%, um,[00:28:50] uh, like what's 85 plus 85? Okay. Um, that's good. Again, these are very, very finger in the wind estimates cuz the variance we're talking about.[00:28:59] But the real question is like, at what point, like the framing is not like at what point do it get to 99, right? The framing is at what point can I use this thing to be better than the alternative, which is humans, which maybe humans or maybe like this problem wasn't possible at all. And so usually the question isn't like, how do I get to 99?[00:29:15] A hundred percent? It's how do I ensure that like the value I am able to get from putting this thing in production is greater than the alternative? In fact, even if you have a model that's less accurate than humans, there might be some circumstances where you can tolerate, uh, a greater amount of inaccuracy.[00:29:32] And if you look at the accuracy relative to the cost, Using a model is extremely cheap. Using a human for the same sort of task can be very expensive. Now, in terms of the actual accuracy of of what you get, there's probably some point at which the cost, but relative accuracy exceeds of a model, exceeds the high cost and hopefully high accuracy of, of a human comparable, like for example, there's like cameras that will track soccer balls or track events happening during sporting matches.[00:30:02] And you can go through and you know, we actually have users that work in sports analytics. You can go through and have a human. Hours and hours of footage. Cuz not just watching their team, they're watching every other team, they're watching scouting teams, they're watching junior teams, they're watching competitors.[00:30:15] And you could have them like, you know, track and follow every single time the ball goes within blank region of the field or every time blank player goes into, uh, this portion of the field. And you could have, you know, exact, like a hundred percent accuracy if that person, maybe, maybe not a hundred, a human may be like 95, 90 7% accuracy of every single time the ball is in this region or this player is on the field.[00:30:36] Truthfully, maybe if you're scouting analytics, you actually don't need 97% accuracy of knowing that that player is on the field. And in fact, if you can just have a model run at a 1000th, a 10000th of the cost and goes through and finds all the times that Messi was present on the field mm-hmm. That the ball was in this region of the.[00:30:54] Then even if that model is slightly less accurate, the cost is just so orders of magnitude different. And the stakes like the stakes of this problem, of knowing like the total number of minutes that Messi played will say are such that we have a higher air tolerance, that it's a no-brainer to start to use Yeah, a computer vision model in this context.[00:31:12] So not every problem requires equivalent or greater human performance. Even when it does, you'd be surprised at how fast models get there. And in the times when you, uh, really look at a problem, the question is, how much accuracy do I need to start to get value from this? This thing, like the package example is a great one, right?[00:31:27] Like I could in theory set up a camera that's constantly watching in front of my porch and I could watch the camera whenever I have a package and then go down. But of course, I'm not gonna do that. I value my time to do other sorts of things instead. And so like there, there's this net new capability of, oh, great, I can have an always on thing that tells me when a package shows up, even if you know the, the thing that's gonna text me.[00:31:46] When a package shows up, let's say a flat pack shows up instead of a box and it doesn't know what a flat pack likes, looks like initially. Doesn't matter. It doesn't matter because I didn't have this capability at all before. And I think that's the true case where a lot of computer vision problems exist is like it.[00:32:00] It's like you didn't even have this capability, this superpower before at all, let alone assigning a given human to do the task. And that's where we see like this explosion of, of value.[00:32:10] Awesome. Awesome. That was a really good overview. I want to leave time for the others, but I, I really want to dive into a couple more things with regards to Robo Flow.[00:32:17] Computer Vision Annotation Formats[00:32:17] So one is, apparently your original pitch for Robo Flow was with regards to conversion tools for computer vision data sets. And I'm sure as, as a result of your job, you have a lot of rants. I've been digging for rants basically on like the best or the worst annotation formats. What do we know? Cause most of us, oh my gosh, we only know, like, you know, I like,[00:32:38] okay, so when we talk about computer vision annotation formats, what we're talking about is if you have an image and you, you picture a boing box around my face on that image.[00:32:46] Yeah. How do you describe where that Monty box is? X, Y, Z X Y coordinates. Okay. X, y coordinates. How, what do you mean from the top lefts.[00:32:52] Okay. You, you, you, you take X and Y and then, and then the. The length and, and the width of the, the[00:32:58] box. Okay. So you got like a top left coordinate and like the bottom right coordinate or like the, the center of the bottom.[00:33:02] Yeah. Yeah. Top, left, bottom right. Yeah. That's one type of format. Okay. But then, um, I come along and I'm like, you know what? I want to do a different format where I wanna just put the center of the box, right. And give the length and width. Right. And by the way, we didn't even talk about what X and Y we're talking about.[00:33:14] Is X a pixel count? Is a relative pixel count? Is it an absolute pixel count? So the point is, the number of ways to describe where a box lives in a freaking image is endless, uh, seemingly and. Everyone decided to kind of create their own different ways of describing the coordinates and positions of where in this context of bounding Box is present.[00:33:39] Uh, so there's some formats, for example, that like use re, so for the x and y, like Y is, uh, like the left, most part of the image is zero. And the right most part of the image is one. So the, the coordinate is like anywhere from zero to one. So 0.6 is, you know, 60% of your way right up the image to describe the coordinate.[00:33:53] I guess that was, that was X instead of Y. But the point is there, of the zero to one is the way that we determined where that was in the position, or we're gonna do an absolute pixel position anyway. We got sick, we got sick of all these different annotation formats. So why do you even have to convert between formats?[00:34:07] Is is another part of this, this story. So different training frameworks, like if you're using TensorFlow, you need like TF Records. If you're using PyTorch, it's probably gonna be, well it depends on like what model you're using, but someone might use Coco JSON with PyTorch. Someone else might use like a, just a YAML file and a text file.[00:34:21] And to describe the cor it's point is everyone that creates a model. Or creates a dataset rather, has created different ways of describing where and how a bounding box is present in the image. And we got sick of all these different formats and doing these in writing all these different converter scripts.[00:34:39] And so we made a tool that just converts from one script, one type of format to another. And the, the key thing is that like if you get that converter script wrong, your model doesn't not work. It just fails silently. Yeah. Because the bounding boxes are now all in the wrong places. And so you need a way to visualize and be sure that your converter script, blah, blah blah.[00:34:54] So that was the very first tool we released of robo. It was just a converter script, you know, like these, like these PDF to word converters that you find. It was basically that for computer vision, like dead simple, really annoying thing. And we put it out there and people found some, some value in, in that.[00:35:08] And you know, to this day that's still like a surprisingly painful[00:35:11] problem. Um, yeah, so you and I met at the Dall-E Hackathon at OpenAI, and we were, I was trying to implement this like face masking thing, and I immediately ran into that problem because, um, you know, the, the parameters that Dall-E expected were different from the one that I got from my face, uh, facial detection thing.[00:35:28] One day it'll go away, but that day is not today. Uh, the worst format that we work with is, is. The mart form, it just makes no sense. And it's like, I think, I think it's a one off annotation format that this university in China started to use to describe where annotations exist in a book mart. I, I don't know, I dunno why that So best[00:35:45] would be TF record or some something similar.[00:35:48] Yeah, I think like, here's your chance to like tell everybody to use one one standard and like, let's, let's, can[00:35:53] I just tell them to use, we have a package that does this for you. I'm just gonna tell you to use the row full package that converts them all, uh, for you. So you don't have to think about this. I mean, Coco JSON is pretty good.[00:36:04] It's like one of the larger industry norms and you know, it's in JS O compared to like V xml, which is an XML format and Coco json is pretty descriptive, but you know, it has, has its own sort of drawbacks and flaws and has random like, attribute, I dunno. Um, yeah, I think the best way to handle this problem is to not have to think about it, which is what we did.[00:36:21] We just created a, uh, library that, that converts and uses things. Uh, for us. We've double checked the heck out of it. There's been hundreds of thousands of people that have used the library and battle tested all these different formats to find those silent errors. So I feel pretty good about no longer having to have a favorite format and instead just rely on.[00:36:38] Dot load in the format that I need. Great[00:36:41] Intro to Computer Vision Segmentation[00:36:41] service to the community. Yeah. Let's go into segmentation because is at the top of everyone's minds, but before we get into segment, anything, I feel like we need a little bit of context on the state-of-the-art prior to Sam, which seems to be YOLO and uh, you are the leading expert as far as I know.[00:36:56] Yeah.[00:36:57] Computer vision, there's various task types. There's classification problems where we just like assign tags to images, like, you know, maybe safe work, not safe work, sort of tagging sort of stuff. Or we have object detection, which are the boing boxes that you see and all the formats I was mentioning in ranting about there's instant segmentation, which is the polygon shapes and produces really, really good looking demos.[00:37:19] So a lot of people like instant segmentation.[00:37:21] This would be like counting pills when you point 'em out on the, on the table. Yeah. So, or[00:37:25] soccer players on the field. So interestingly, um, counting you could do with bounding boxes. Okay. Cause you could just say, you know, a box around a person. Well, I could count, you know, 12 players on the field.[00:37:35] Masks are most useful. Polygons are most useful if you need very precise area measurements. So you have an aerial photo of a home and you want to know, and the home's not a perfect box, and you want to know the rough square footage of that home. Well, if you know the distance between like the drone and, and the ground.[00:37:53] And you have the precise polygon shape of the home, then you can calculate how big that home is from aerial photos. And then insurers can, you know, provide say accurate estimates and that's maybe why this is useful. So polygons and, and instant segmentation are, are those types of tasks? There's a key point detection task and key point is, you know, if you've seen those demos of like all the joints on like a hand kind of, kind of outlined, there's visual question answering tasks, visual q and a.[00:38:21] And that's like, you know, some of the stuff that multi-modality is absolutely crushing for, you know, here's an image, tell me what food is in this image. And then you can pass that and you can make a recipe out of it. But like, um, yeah, the visual question in answering task type is where multi-modality is gonna have and is already having an enormous impact.[00:38:40] So that's not a comprehensive survey, very problem type, but it's enough to, to go into why SAM is significant. So these various task types, you know, which model to use for which given circumstance. Most things is highly dependent on what you're ultimately aiming to do. Like if you need to run a model on the edge, you're gonna need a smaller model, cuz it is gonna run on edge, compute and process in, in, in real time.[00:39:01] If you're gonna run a model on the cloud, then of course you, uh, generally have more compute at your disposal Considerations like this now, uh,[00:39:08] YOLO[00:39:08] just to pause. Yeah. Do you have to explain YOLO first before you go to Sam, or[00:39:11] Yeah, yeah, sure. So, yeah. Yeah, we should. So object detection world. So for a while I talked about various different task types and you can kinda think about a slide scale of like classification, then obvious detection.[00:39:20] And on the right, at most point you have like segmentation tasks. Object detection. The bounding boxes is especially useful for a wide, like it's, it's surprisingly versatile. Whereas like classification is kind of brittle. Like you only have a tag for the whole image. Well, that doesn't, you can't count things with tags.[00:39:35] And on the other hand, like the mask side of things, like drawing masks is painstaking. And so like labeling is just a bit more difficult. Plus like the processing to produce masks requires more compute. And so usually a lot of folks kind of landed for a long time on obvious detection being a really happy medium of affording you with rich capabilities because you can do things like count, track, measure.[00:39:56] In some CAGR context with bounding boxes, you can see how many things are present. You can actually get a sense of how fast something's moving by tracking the object or bounding box across multiple frames and comparing the timestamp of where it was across those frames. So obviously detection is a very common task type that solves lots of things that you want do with a given model.[00:40:15] In obviously detection. There's been various model frameworks over time. So kind of really early on there's like R-CNN uh, then there's faster rc n n and these sorts of family models, which are based on like resnet kind of architectures. And then a big thing happens, and that is single shot detectors. So faster, rc n n despite its name is, is very slow cuz it takes two passes on the image.[00:40:37] Uh, the first pass is, it finds par pixels in the image that are most interesting to, uh, create a bounding box candidate out of. And then it passes that to a, a classifier that then does classification of the bounding box of interest. Right. Yeah. You can see, you can see why that would be slow. Yeah. Cause you have to do two passes.[00:40:53] You know, kind of actually led by, uh, like mobile net was I think the first large, uh, single shot detector. And as its name implies, it was meant to be run on edge devices and mobile devices and Google released mobile net. So it's a popular implementation that you find in TensorFlow. And what single shot detectors did is they said, Hey, instead of looking at the image twice, what if we just kind of have a, a backbone that finds candidate bounding boxes?[00:41:19] And then we, we set loss functions for objectness. We set loss function. That's a real thing. We set loss functions for objectness, like how much obj, how object do this part of the images. We send a loss function for classification, and then we run the image through the model on a single pass. And that saves lots of compute time and you know, it's not necessarily as accurate, but if you have lesser compute, it can be extremely useful.[00:41:42] And then the advances in both modeling techniques in compute and data quality, single shot detectors, SSDs has become, uh, really, really popular. One of the biggest SSDs that has become really popular is the YOLO family models, as you described. And so YOLO stands for you only look once. Yeah, right, of course.[00:42:02] Uh, Drake's, uh, other album, um, so Joseph Redman introduces YOLO at the University of Washington. And Joseph Redman is, uh, kind of a, a fun guy. So for listeners, for an Easter egg, I'm gonna tell you to Google Joseph Redman resume, and you'll find, you'll find My Little Pony. That's all I'll say. And so he introduces the very first YOLO architecture, which is a single shot detector, and he also does it in a framework called Darknet, which is like this, this own framework that compiles the Cs, frankly, kind of tough to work with, but allows you to benefit from the speedups that advance when you operate in a low level language like.[00:42:36] And then he releases, well, what colloquially is known as YOLO V two, but a paper's called YOLO 9,000 cuz Joseph Redmond thought it'd be funny to have something over 9,000. So get a sense for, yeah, some fun. And then he releases, uh, YOLO V three and YOLO V three is kind of like where things really start to click because it goes from being an SSD that's very limited to competitive and, and, and superior to actually mobile That and some of these other single shot detectors, which is awesome because you have this sort of solo, I mean, him and and his advisor, Ali, at University of Washington have these, uh, models that are becoming really, really powerful and capable and competitive with these large research organizations.[00:43:09] Joseph Edmond leaves Computer Vision Research, but there had been Alexia ab, one of the maintainers of Darknet released Yola VI four. And another, uh, researcher, Glenn Yer, uh, jocker had been working on YOLO V three, but in a PyTorch implementation, cuz remember YOLO is in a dark implementation. And so then, you know, YOLO V three and then Glenn continues to make additional improvements to YOLO V three and pretty soon his improvements on Yolov theory, he's like, oh, this is kind of its own things.[00:43:36] Then he releases YOLO V five[00:43:38] with some naming[00:43:39] controversy that we don't have Big naming controversy. The, the too long didn't read on the naming controversy is because Glen was not originally involved with Darknet. How is he allowed to use the YOLO moniker? Roe got in a lot of trouble cuz we wrote a bunch of content about YOLO V five and people were like, ah, why are you naming it that we're not?[00:43:55] Um, but you know,[00:43:56] cool. But anyway, so state-of-the-art goes to v8. Is what I gather.[00:44:00] Yeah, yeah. So yeah. Yeah. You're, you're just like, okay, I got V five. I'll skip to the end. Uh, unless, unless there's something, I mean, I don't want, well, so I mean, there's some interesting things. Um, in the yolo, there's like, there's like a bunch of YOLO variants.[00:44:10] So YOLOs become this, like this, this catchall for various single shot, yeah. For various single shot, basically like runs on the edge, it's quick detection framework. And so there's, um, like YOLO R, there's YOLO S, which is a transformer based, uh, yolo, yet look like you only look at one sequence is what s stands were.[00:44:27] Um, the pp yo, which, uh, is PAT Paddle implementation, which is by, which Chinese Google is, is their implementation of, of TensorFlow, if you will. So basically YOLO has like all these variants. And now, um, yo vii, which is Glen has been working on, is now I think kind of like, uh, one of the choice models to use for single shot detection.[00:44:44] World Knowledge of Foundation Models[00:44:44] Well, I think a lot of those models, you know, Asking the first principal's question, like let's say you wanna find like a bus detector. Do you need to like go find a bunch of photos of buses or maybe like a chair detector? Do you need to go find a bunch of photos of chairs? It's like, oh no. You know, actually those images are present not only in the cocoa data set, but those are objects that exist like kind of broadly on the internet.[00:45:02] And so computer visions kind of been like us included, have been like really pushing for and encouraging models that already possess a lot of context about the world. And so, you know, if GB T's idea and i's idea OpenAI was okay, models can only understand things that are in their corpus. What if we just make their corpus the size of everything on the internet?[00:45:20] The same thing that happened in imagery, what's happening now? And that's kinda what Sam represents, which is kind of a new evolution of, earlier on we were talking about the cost of annotation and I said, well, good news. Annotations then become decreasingly necessary to start to get to value. Now you gotta think about it more, kind of like, you'll probably need to do some annotation because you might want to find a custom object, or Sam might not be perfect, but what's about to happen is a big opportunity where you want the benefits of a yolo, right?[00:45:47] Where it can run really fast, it can run on the edge, it's very cheap. But you want the knowledge of a large foundation model that already knows everything about buses and knows everything about shoes, knows everything about real, if the name is true, anything segment, anything model. And so there's gonna be this novel opportunity to take what these large models know, and I guess it's kind of like a form of distilling, like distill them down into smaller architectures that you can use in versatile ways to run in real time to run on the edge.[00:46:13] And that's now happening. And what we're seeing in actually kind of like pulling that, that future forward with, with, with Robo Flow.[00:46:21] Segment Anything Model[00:46:21] So we could talk a bit about, um, about SAM and what it represents maybe into, in relation to like these, these YOLO models. So Sam is Facebook segment Everything Model. It came out last week, um, the first week of April.[00:46:34] It has 24,000 GitHub stars at the time of, of this recording within its first week. And why, what does it do? Segment? Everything is a zero shot segmentation model. And as we're describing, creating masks is a very arduous task. Creating masks of objects that are not already represented means you have to go label a bunch of masks and then train a model and then hope that it finds those masks in new images.[00:47:00] And the promise of Segment anything is that in fact you just pass at any image and it finds all of the masks of relevant things that you might be curious about finding in a given image. And it works remarkably. Segment anything in credit to Facebook and the fair Facebook research team, they not only released the model permissive license to move things forward, they released the full data set, all 11 million images and 1.1 billion segmentation masks and three model sizes.[00:47:29] The largest ones like 2.5 gigabytes, which is not enormous. Medium ones like 1.2 and the smallest one is like 400, 3 75 megabytes. And for context,[00:47:38] for, for people listening, that's six times more than the previous alternative, which, which is apparently open images, uh, in terms of number images, and then 400 times more masks than open[00:47:47] images as well.[00:47:48] Exactly, yeah. So huge, huge order magnitude gain in terms of dataset accessibility plus like the model and how it works. And so the question becomes, okay, so like segment. What, what do I do with this? Like, what does it allow me to do? And it didn't Rob float well. Yeah, you should. Yeah. Um, it's already there.[00:48:04] You um, that part's done. Uh, but the thing that you can do with segment anything is you can almost, like, I almost think about like this, kinda like this model arbitrage where you can basically like distill down a giant model. So let's say like, like let's return to the package example. Okay. The package problem of, I wanna get a text when a package appears on my front porch before segment anything.[00:48:25] The way that I would go solve this problem is I would go collect some images of packages on my porch and I would label them, uh, with bounding boxes or maybe masks in that part. As you mentioned, it can be a long process and I would train a model. And that model it actually probably worked pretty well cause it's purpose-built.[00:48:44] The camera position, my porch, the packages I'm receiving. But that's gonna take some time, like everything that I just mentioned the

In this episode of Studio 22, we sit down with acclaimed producer Ben Silverman to talk about his incredible career in television and film. As the mastermind behind hit shows like The Office, The Biggest Loser, and The Untold Series, as well as documentaries like Kurt and Courtney, Ben has made a name for himself as one of the most innovative and successful producers in the industry. We dive deep into Ben's role as producer on The Office, discussing the creative process behind the show and how it changed the face of comedy television. Ben shares his insights on the show's iconic characters and moments, and reflects on what made the show such a cultural phenomenon. But Ben's expertise goes beyond just comedy. We also talk about his work on The Biggest Loser, a groundbreaking reality show that transformed the lives of its contestants and inspired audiences around the world. Ben shares some of the challenges and rewards of producing a show that had such a profound impact on its participants and viewers. In addition to his work on The Biggest Loser, we discuss Ben's latest project, The Untold Series, a collection of documentary films that delve into some of the most compelling and little-known stories in sports history. Ben shares his experiences producing the series and the power of storytelling to illuminate the lives of ordinary people who achieve extraordinary things. Join us for a fascinating conversation with one of the most influential producers of our time. Listen in and gain insights into the creative process, the world of television and film production, and the secrets of telling stories that inspire and engage audiences. Check out  https://www.studio22podcast.com/ Follow us on social media! https://instagram.com/studio22podcast https://www.tiktok.com/@studioxxiipodcast https://instagram.com/brockohurn https://instagram.com/wmeldman33 https://www.tiktok.com/@brockohurn https://twitter.com/BrockohurnSee omnystudio.com/listener for privacy information.

Ben is joined by Analyst & 10b5-1 Expert C. Max Magee to discuss Rule 10b5-1 — what it is, how it provides institutional investors with unique insights, and which of the SEC's many changes you should pay attention to. Finally, Ben shares recent examples of corporate insider activity to highlight how the SEC may or may not actually curtail certain behaviors.Edited, mixed, and scored by Calvin Marty.

IN CELEBRATION (TRIO NO. 1)In Celebration (1989) was co-commissioned by the Krannert Center for the Performing Arts at the University of Illinois in honor of its 20th anniversary and by the Tisch Center for the Performing Arts at the 92nd Street Y in New York. The composition is dedicated to the Kalichstein- Laredo-Robinson Trio in response to their request to write something “jazzy.” The Trio gave its premiere on March 31, 1989, at the Krannert Center, and its New York premiere at the 92nd Street Y on April 11, 1989.REVEILLE (TRIO NO. 2) The piece was written in 2011 for the Kalichstein-Laredo-Robinson Trio. The work wascommissioned by Ben Silverman and dedicated to Herman Sandler. The Trio with guest artist Sting gave the premiere on September 14, 2011, at the 92nd Street Y in New York in observance of the tenth anniversary of 9/11.Help support our show by purchasing this album  at:Downloads (classicalmusicdiscoveries.store) Classical Music Discoveries is sponsored by Uber. @CMDHedgecock#ClassicalMusicDiscoveries #KeepClassicalMusicAlive#LaMusicaFestival #CMDGrandOperaCompanyofVenice #CMDParisPhilharmonicinOrléans#CMDGermanOperaCompanyofBerlin#CMDGrandOperaCompanyofBarcelonaSpain#ClassicalMusicLivesOn#Uber Please consider supporting our show, thank you!Donate (classicalmusicdiscoveries.store) staff@classicalmusicdiscoveries.comThis album is broadcasted with the permission of Crossover Media Music Promotion (Zachary Swanson and Amanda Bloom).

This week's guest on the Silver Club Podcast is Ben Silverman. Ben connected with NFL QB Aaron Rodgers to win the 2023 Pebble Beach Pro-Am. He also just won on the Korn Ferry Tour at the 2023 Bahamas Great Abaco Classic for his second victory on that tour. More of a late bloomer to the game, Ben gives us great insight to the life of a professional golfer a few drills that the weekend warrior can put into their repertoire to improve.

This month Ben talks about a truly strange insider trend that happened; discusses how investors can include company stock buybacks in fundamental analysis (with guest Ali Ragih, CFA), and covers the curious case of Coupa Software — where a huge payday for the CEO may have offered insight into an upcoming M&A deal.Edited, mixed, and scored by Calvin Marty.

Canadian Korn Ferry Tour pro Ben Silverman joins the show to discuss playing alongside Aaron Rodgers, winning the tournament, if Rodgers is really a 10 handicap, what it was like playing at Pebble Beach as a sponsor exemption and what his dream as a kid was.

Canadian Korn Ferry Tour pro Ben Silverman joins the show to discuss playing alongside Aaron Rodgers, winning the tournament, if Rodgers is really a 10 handicap, what it was like playing at Pebble Beach as a sponsor exemption and what his dream as a kid was.

Web3 Strategist Ben Silverman discusses his background as a filmmaker and consultant and how getting his start at Creative Artists Agency (CAA) paved the way for him to see trends within the entertainment industry, culminating with his understanding of how Web3 is poised to change content creation, change how we interact with content, and change how contracts are executed.Twitter: @bensilvermanInstagram: @benmarcsilvermanSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

This week on the Be Epic podcast Matt sits down with https://www.linkedin.com/in/drew-buckley-1bb182/ (Drew Buckley), Group President & Chief Operating Officer for https://propagatecontent.com/ (Propagate Content). They discuss Drew's journey in the entertainment and entrepreneurial space including roles at USA Today, AT&T, Yahoo and eventually founding his own company with Ben Silverman, former chairman of NBC. The independent content studio they founded together created CHOPPED, Running Wild with Bear Grylls, Jane the Virgin, and more. They go further into the episode discussing the dynamism of the digital revolution for entertainment content especially going from network television and prime time slots to Netflix and other streaming platforms where content is on demand. They finish by discussing how talent discovery has changed in the entertainment industry especially with the evolution of social media.

“I reached a point in my career where I really wanted to be able to make a change in the industry that would help other people come into the business, and find their place and be able to work and tell stories, and do all the things that will democratize that access.” Jayzen is excited to welcome H Schuster to the show. H has built her career producing some of the most iconic unscripted television shows that we all love, and now as Founder and CEO of HUSSLUP, is transforming the creative industry. As an opportunity-maker, she is constantly striving in her career to not only create incredible content, but open the door for others along the way. H has more than 15 years of experience producing and overseeing high profile hits, including the Kardashian franchises, "The Biggest Loser," "MasterChef," and others. During her career, she has worked closely with top producers like Mark Burnett, Ben Silverman, and Eli Holzman and A-list talent, as well as with numerous FORTUNE 100 companies to integrate their brands into TV and digital content. Guest Bio H Schuster Founder + CEO HUSSLUP HUSSLUP Founder and CEO H Schuster is a senior media executive with more than 20 years of experience producing TV and leading divisions at major media companies and startups. She founded HUSSLUP to connect millions of professional creatives around the globe to each other and to the companies that need them in order to democratize access to the industry. As CEO, she leads a diverse, senior-level team of industry veterans with the mission of “transforming Hollywood's old clique with one click.” Prior to HUSSLUP, H was Founder and CEO of Morocco Junction Entertainment, a production company financed by All3Media. She built a slate and oversaw creative development and production for projects at Discovery, TLC, Lifetime, History, Spike, MTV and Animal Planet, as well as executive producing “Abby's Studio Rescue” for Lifetime and creating “Tabatha's Life Takeover” for Bravo. She also provided strategic and content consulting to major production companies and producers. Earlier, Schuster headed unscripted television for Ryan Seacrest Productions, Reveille, and Shine America. She also served as the CCO who scaled Detour, a location-aware podcasting company that was one of Apple's top 10 apps of 2016 and sold to Bose. Schuster has more than 15 years of experience producing and overseeing high profile hits, including the Kardashian franchises, "The Biggest Loser," "MasterChef," and others. During her career, she has worked closely with top producers like Mark Burnett, Ben Silverman, and Eli Holzman and A-list talent, as well as with numerous FORTUNE 100 companies to integrate their brands into TV and digital content. At Detour, she also worked closely with the engineering team to iterate on and ship product. Schuster earned a PhD in cultural studies at NYU and attended Stanford Law School. She served multiple terms on the Outfest Board of Directors, is a founding member of Chief, and is a member of the Producers Guild of America, the Television Academy, and Women in Film. Links To learn more about Lead With Your Brand system, please visit: LeadWithyYourBrand.com To book Jayzen for a speaking engagement or workshop at your company, visit: JayzenPatria.com

"Big technology companies like Amazon (AMZN) and Alphabet (GOOGL) bought back the largest share of their own stock in their history in the second quarter. Meta Platforms (META), previously Facebook (FB) buybacks shrank to the lowest level since 2021. Apple (AAPL) led all companies in total dollar volume of buybacks. 1,317 U.S. companies reported buybacks recently," says Ben Silverman.

Meme stocks like AMC Entertainment (AMC) and GameStop (GME) have experienced high volume this week and the shares have rallied. Ben Silverman and David Trainer break down the recent market action and deliver their outlooks for the internet-driven space moving forward. Trainer says GameStop's valuation is completely disconnected from fundamentals and that there is no fundamental reason for someone to own shares of the company that has received so much attention from retail investors over the past couple of years.

Sure, they made it look easy, but executive producer Ben Silverman knows the true struggles behind bringing The Office to American television. Ben tells the tale of his decision to adapt the British sitcom and the meeting with Ricky Gervais that followed only 24 hours later, and the time he found out about the show's second season pickup … while watching Brad Pitt sleep on an airplane. Learn more about your ad-choices at https://www.iheartpodcastnetwork.com Learn more about your ad-choices at https://www.iheartpodcastnetwork.com

The Podcast is back BABY! With our new host, Charlie and now with VIDEO! This first episode is all about staying fit and healthy in Japan. Japanese food is so amazing! Gyms in Japan are so expensive! That's why after years of living here so many people give up on their fitness goals. Well Ben is here to set you straight, WITHOUT having to give up your favourite Japanese foods or joining office work parties. Find out more from Ben on Facebook: https://www.facebook.com/ben.silverman3 Want to Leave a Review? Thank you so much! This helps us a lot: iTunes This show is proudly sponsored by JobsinJapan.com! For your first job in Japan, your next job in Japan, your best job in Japan, go to JobsinJapan.com.

Brian is joined by The Office U.K. co-creator Stephen Merchant, all the way from London. Stephen talks about meeting the quintessential Hollywood producer, Ben Silverman, and finding just the right American counterparts to make The Office work on the other side of the Atlantic. Learn more about your ad-choices at https://www.iheartpodcastnetwork.com Learn more about your ad-choices at https://www.iheartpodcastnetwork.com