Podcasts about TGI

In this eye-opening episode of our Brand Series, restaurant industry experts Paul Barron and Paul Molinari dissect the current crisis in casual dining and reveal why some chains are thriving while others file for bankruptcy. Discover how Chili's achieved a remarkable 31% sales increase through strategic social media targeting and menu innovation, while TGI Fridays and Hooters struggle to connect with millennial consumers. The hosts analyze Red Lobster's repositioning strategy, debate the controversial Hooters rebranding plan, and explore how economic headwinds are creating recession indicators even for giants like McDonald's. Don't miss these critical insights on brand transformation, consumer behavior shifts, and the technological innovations poised to reshape restaurants in 2026-2027.~This episode is sponsored by: Gusto → https://gusto.pxf.io/PBN ~#1 rated HR platform for payroll, benefits, and moreWith Gusto's easy-to-use platform, you can empower your people and push your business forward. See why over 400,000 businesses choose Gusto.RestaurantBrandSeries #CasualDiningCrisis #FoodServiceFutureGet Your Podcast Now! Are you a hospitality or restaurant industry leader looking to amplify your voice and establish yourself as a thought leader? Look no further than SavorFM, the premier podcast platform designed exclusively for hospitality visionaries like you. Take the next step in your industry leadership journey – visit https://www.savor.fm/Capital & Advisory: Are you a fast-casual restaurant startup or a technology innovator in the food service industry? Don't miss out on the opportunity to tap into decades of expertise. Reach out to Savor Capital & Advisory now to explore how their seasoned professionals can propel your business forward. Discover if you're eligible to leverage our unparalleled knowledge in food service branding and technology and take your venture to new heights.Don't wait – amplify your voice or supercharge your startup's growth today with Savor's ecosystem of industry-leading platforms and advisory services. Visit https://www.savor.fm/capital-advisory

There' an exciting feeling around this franchise right now. We discuss the selection of Adbul Carter and the trade up for Jaxson Dart. We also look ahead to Day 2 and where the organization may go next. Sign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the appAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Jerry takes you through the first three picks of the draft which culminates in the Giants selecting perhaps the draft's most talented player in Penn State's Abdul Carter. Enjoy, folks.Sign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the appAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Tgi che va cun skis sto dar in'egliada sin il privel da lavinas. E per pudair valitar lez sto l'Institut per la perscrutaziun da naiv e lavinas (SLF) far differentas mesiraziuns e quai di per di. Il «Minisguard» ha accumpagnà Chasper Buchli dal SLF sin l'areal da perscrutaziun en las muntognas da Tavau. Ed il cool – i na va betg be per guardar quanta naiv ch'igl ha dà e sche quella è loma u bletscha, mabain anc per in'entira massa autras chaussas. La circulaziun da l'aua Sin noss mund datti radund 1,4 trilliardas liters aua, la gronda part en furma d'aua da sal en las mars. Tut quest'aua fa part da la circulaziun d'aua. Grazia a la chalira dal sulegl è ella permanentamain en moviment e sa mida adina puspè da la furma liquida en vapur ed enavos. Ma tge capita precis?

In this episode of the Becker's Healthcare Podcast, Erica Carbajal speaks with Jennifer Baron, Chief Experience Officer at NRC Health, and Kathryn Peisert, Editor in Chief & Senior Director at TGI, about the critical role of trust in shaping the future of patient experience. Drawing from NRC Health's 2025 Experience Perspective report, they explore how health systems can strategically embed trust into their culture, improve engagement, and stay ahead of shifting expectations.This episode is sponsored by NRC Health.

The Giants sent a fleet of folks to Colorado's Pro Day to watch Sheduer Sanders....or is it Travis Hunter? Or, both? Enjoy, folksSign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the appAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Dave Syvertsen of Ourlads returns to talk all things Giants draft. You don't want to miss this one, folks. Enjoy. Sign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the appAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

How long is Joe Schoen going to wait for Aaron Rodgers? Is it time to hand the keys to Russell Wilson or another QB (for now)? We discuss. Enjoy.Sign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in Bonus Credits and a freepick: underdogfantasy.com or download the appAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

We talk Giants draft with special guest Ric Serritella. Enjoy, folks.Sign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in Bonus Credits and a freepick: underdogfantasy.com or download the appAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Jorge Andrés Henao, gerente general de TGI by Diario La república

We return to the live stream to discuss the Joe Schoen presser, our defensive free agent wish list, and we read your comments. Best of luck, @Bret_Gibson. Your wife is an AMAZING person! Enjoy.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

On this live stream we review the free agency possibilities on the offensive side of the ball. At the end of the podcast, Jerry gives his review of "Becoming Led Zeppelin." Enjoy.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

This nightmarish season finally ends with our archrivals winning the Super Bowl (naturally!) and a Giants legend waving their flag. Make it stop!!! Will Papa joins us to talk about his new venture -- fourthandgoal.net Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Ourlads' Dave Syvertsen joins us to discuss the QBs at the top of the draft as well as some other first-round options for the GMEN. If the Chiefs beat the Eagles, it's 365 STRAIGHT DAYS OF TAYLOR SWIFT FOR JERRY. You don't want to miss this episode, folks. Enjoy. Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Tgi vul succeder a Viola Amherd? – 10 onns vischnanca fusiunada Scuol

The Giants hire a new position coach, we discuss some possibilities at No. 3, and we discuss the upcoming Conference Championship games. Enjoy. Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Reportascha dal Spital regiunal Surselva: prestaziuns da servetsch na "rendan" betg pli – Dumondas e respostas: Tgi decida tge prestaziuns ch'ils spitals ston u dastgan porscher? – Discurs cun Urs Cadruvi (directur da duas clinicas Hirslanden a Son Gagl) e Peder Plaz (president dal cussegl administrativ da la ÖKK): Stuessan las cassas da malsauns augmentar las tariffas? Èn ils spitals betg avunda effizients? – Intervista cun Peter Peyer (minister da sanadad): Tge schliaziuns vegnan en dumonda?

«Nagin commentari», «Igl è difficil» ubain «Jau ditg pli gugent nagut»–- quai èn las respostas ch'ins survegn sin las vias da Tavau, sch'ins dumonda las abitantas ed ils abitants davart il WEF. L'inscunter da l'elita politica, economica e sociala divida la populaziun da Tavau. Tgi fa ina pluna daners cun affittar butias u abitaziuns e tgi sa vilenta da la decadenza. La «Marella» sa fatschenta cun il WEF e sias consequenzas per las abitantas ed ils abitants da Tavau. Nus dain dentant er in sguard sin il mund dal WEF che para ester entamez las muntognas.

Jerome Henderson is out (hmm..), we review the Giants' unrestricted free agents and discuss who we should keep vs. who we should launch, and we make our picks for this playoff weekend. Enjoy.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

The NY Post's Ryan Dunleavy joins the podcast to discuss the John Mara, Joe Schoen, and Brian Daboll pressers. We enjoy Ryan's perspectives and appreciate him lending us his opinions as well as his time.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

The Giants lose to the Eagles and secure the third overall pick in the 2025 NFL Draft. We discuss the future of the Schoen/Daboll regime. This disaster of a season is finally over, folks.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

L'onn è anc frestg, tut è pussaivel. Per il Minisguard avain nus dumandà ils uffants sin tge ch'els sa legran, tge ch'els giavischan e sch'els han in u l'auter propiest per il 2025. Tgi vul far urden pli regular en chombra? Tgi less dar gas en scola? E tgi è cuntainta sco igl è e na less far nagut auter che fin uss?

Social media is on fire as the Giants win but drop from 1 to 4 in the 2025 draft order. We preview the matchup with Philly and discuss how long the Eagles will play their starters. We also preview the rest of Week 18. Hang in, folks. For YouTube and podcast:Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Happy holidays! We'll be sharing snippets from Latent Space LIVE! through the break bringing you the best of 2024! We want to express our deepest appreciation to event sponsors AWS, Daylight Computer, Thoth.ai, StrongCompute, Notable Capital, and most of all all our LS supporters who helped fund the gorgeous venue and A/V production!For NeurIPS last year we did our standard conference podcast coverage interviewing selected papers (that we have now also done for ICLR and ICML), however we felt that we could be doing more to help AI Engineers 1) get more industry-relevant content, and 2) recap 2024 year in review from experts. As a result, we organized the first Latent Space LIVE!, our first in person miniconference, at NeurIPS 2024 in Vancouver. Today, we're proud to share Loubna's highly anticipated talk (slides here)!Synthetic DataWe called out the Synthetic Data debate at last year's NeurIPS, and no surprise that 2024 was dominated by the rise of synthetic data everywhere:* Apple's Rephrasing the Web, Microsoft's Phi 2-4 and Orca/AgentInstruct, Tencent's Billion Persona dataset, DCLM, and HuggingFace's FineWeb-Edu, and Loubna's own Cosmopedia extended the ideas of synthetic textbook and agent generation to improve raw web scrape dataset quality* This year we also talked to the IDEFICS/OBELICS team at HuggingFace who released WebSight this year, the first work on code-vs-images synthetic data.* We called Llama 3.1 the Synthetic Data Model for its extensive use (and documentation!) of synthetic data in its pipeline, as well as its permissive license. * Nemotron CC and Nemotron-4-340B also made a big splash this year for how they used 20k items of human data to synthesize over 98% of the data used for SFT/PFT.* Cohere introduced Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progress observing gains of up to 56.5% improvement in win rates comparing multiple teachers vs the single best teacher model* In post training, AI2's Tülu3 (discussed by Luca in our Open Models talk) and Loubna's Smol Talk were also notable open releases this year.This comes in the face of a lot of scrutiny and criticism, with Scale AI as one of the leading voices publishing AI models collapse when trained on recursively generated data in Nature magazine bringing mainstream concerns to the potential downsides of poor quality syndata:Part of the concerns we highlighted last year on low-background tokens are coming to bear: ChatGPT contaminated data is spiking in every possible metric:But perhaps, if Sakana's AI Scientist pans out this year, we will have mostly-AI AI researchers publishing AI research anyway so do we really care as long as the ideas can be verified to be correct?Smol ModelsMeta surprised many folks this year by not just aggressively updating Llama 3 and adding multimodality, but also adding a new series of “small” 1B and 3B “on device” models this year, even working on quantized numerics collaborations with Qualcomm, Mediatek, and Arm. It is near unbelievable that a 1B model today can qualitatively match a 13B model of last year:and the minimum size to hit a given MMLU bar has come down roughly 10x in the last year. We have been tracking this proxied by Lmsys Elo and inference price:The key reads this year are:* MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases* Apple Intelligence Foundation Language Models* Hymba: A Hybrid-head Architecture for Small Language Models* Loubna's SmolLM and SmolLM2: a family of state-of-the-art small models with 135M, 360M, and 1.7B parameters on the pareto efficiency frontier.* and Moondream, which we already covered in the 2024 in Vision talkFull Talk on YouTubeplease like and subscribe!Timestamps* [00:00:05] Loubna Intro* [00:00:33] The Rise of Synthetic Data Everywhere* [00:02:57] Model Collapse* [00:05:14] Phi, FineWeb, Cosmopedia - Synthetic Textbooks* [00:12:36] DCLM, Nemotron-CC* [00:13:28] Post Training - AI2 Tulu, Smol Talk, Cohere Multilingual Arbitrage* [00:16:17] Smol Models* [00:18:24] On Device Models* [00:22:45] Smol Vision Models* [00:25:14] What's NextTranscript2024 in Synthetic Data and Smol Models[00:00:00] ​[00:00:05] Loubna Intro[00:00:05] Speaker: ​I'm very happy to be here. Thank you for the invitation. So I'm going to be talking about synthetic data in 2024. And then I'm going to be talking about small on device models. So I think the most interesting thing about synthetic data this year is that like now we have it everywhere in the large language models pipeline.[00:00:33] The Rise of Synthetic Data Everywhere[00:00:33] Speaker: I think initially, synthetic data was mainly used just for post training, because naturally that's the part where we needed human annotators. And then after that, we realized that we don't really have good benchmarks to [00:01:00] measure if models follow instructions well, if they are creative enough, or if they are chatty enough, so we also started using LLMs as judges.[00:01:08] Speaker: Thank you. And I think this year and towards the end of last year, we also went to the pre training parts and we started generating synthetic data for pre training to kind of replace some parts of the web. And the motivation behind that is that you have a lot of control over synthetic data. You can control your prompt and basically also the kind of data that you generate.[00:01:28] Speaker: So instead of just trying to filter the web, you could try to get the LLM to generate what you think the best web pages could look like and then train your models on that. So this is how we went from not having synthetic data at all in the LLM pipeline to having it everywhere. And so the cool thing is like today you can train an LLM with like an entirely synthetic pipeline.[00:01:49] Speaker: For example, you can use our Cosmopedia datasets and you can train a 1B model on like 150 billion tokens that are 100 percent synthetic. And those are also of good quality. And then you can [00:02:00] instruction tune the model on a synthetic SFT dataset. You can also do DPO on a synthetic dataset. And then to evaluate if the model is good, you can use.[00:02:07] Speaker: A benchmark that uses LLMs as a judge, for example, MTBench or AlpacaEvil. So I think this is like a really mind blowing because like just a few years ago, we wouldn't think this is possible. And I think there's a lot of concerns about model collapse, and I'm going to talk about that later. But we'll see that like, if we use synthetic data properly and we curate it carefully, that shouldn't happen.[00:02:29] Speaker: And the reason synthetic data is very popular right now is that we have really strong models, both open and closed. It is really cheap and fast to use compared to human annotations, which cost a lot and take a lot of time. And also for open models right now, we have some really good inference frameworks.[00:02:47] Speaker: So if you have enough GPUs, it's really easy to spawn these GPUs and generate like a lot of synthetic data. Some examples are VLM, TGI, and TensorRT.[00:02:57] Model Collapse[00:02:57] Speaker: Now let's talk about the elephant in the room, model [00:03:00] collapse. Is this the end? If you look at the media and all of like, for example, some papers in nature, it's really scary because there's a lot of synthetic data out there in the web.[00:03:09] Speaker: And naturally we train on the web. So we're going to be training a lot of synthetic data. And if model collapse is going to happen, we should really try to take that seriously. And the other issue is that, as I said, we think, a lot of people think the web is polluted because there's a lot of synthetic data.[00:03:24] Speaker: And for example, when we're building fine web datasets here at Guillerm and Hinek, we're interested in like, how much synthetic data is there in the web? So there isn't really a method to properly measure the amount of synthetic data or to save a webpage synthetic or not. But one thing we can do is to try to look for like proxy words, for example, expressions like as a large language model or words like delve that we know are actually generated by chat GPT.[00:03:49] Speaker: We could try to measure the amount of these words in our data system and compare them to the previous years. For example, here, we measured like a, these words ratio in different dumps of common crawl. [00:04:00] And we can see that like the ratio really increased after chat GPT's release. So if we were to say that synthetic data amount didn't change, you would expect this ratio to stay constant, which is not the case.[00:04:11] Speaker: So there's a lot of synthetic data probably on the web, but does this really make models worse? So what we did is we trained different models on these different dumps. And we then computed their performance on popular, like, NLP benchmarks, and then we computed the aggregated score. And surprisingly, you can see that the latest DOMs are actually even better than the DOMs that are before.[00:04:31] Speaker: So if there's some synthetic data there, at least it did not make the model's worse. Yeah, which is really encouraging. So personally, I wouldn't say the web is positive with Synthetic Data. Maybe it's even making it more rich. And the issue with like model collapse is that, for example, those studies, they were done at like a small scale, and you would ask the model to complete, for example, a Wikipedia paragraph, and then you would train it on these new generations, and you would do that every day.[00:04:56] Speaker: iteratively. I think if you do that approach, it's normal to [00:05:00] observe this kind of behavior because the quality is going to be worse because the model is already small. And then if you train it just on its generations, you shouldn't expect it to become better. But what we're really doing here is that we take a model that is very large and we try to distill its knowledge into a model that is smaller.[00:05:14] Phi, FineWeb, Cosmopedia - Synthetic Textbooks[00:05:14] Speaker: And in this way, you can expect to get like a better performance for your small model. And using synthetic data for pre-training has become really popular. After the textbooks are all you need papers where Microsoft basically trained a series of small models on textbooks that were using a large LLM.[00:05:32] Speaker: And then they found that these models were actually better than models that are much larger. So this was really interesting. It was like first of its time, but it was also met with a lot of skepticism, which is a good thing in research. It pushes you to question things because the dataset that they trained on was not public, so people were not really sure if these models are really good or maybe there's just some data contamination.[00:05:55] Speaker: So it was really hard to check if you just have the weights of the models. [00:06:00] And as Hugging Face, because we like open source, we tried to reproduce what they did. So this is our Cosmopedia dataset. We basically tried to follow a similar approach to what they documented in the paper. And we created a synthetic dataset of textbooks and blog posts and stories that had almost 30 billion tokens.[00:06:16] Speaker: And we tried to train some models on that. And we found that like the key ingredient to getting a good data set that is synthetic is trying as much as possible to keep it diverse. Because if you just throw the same prompts as your model, like generate like a textbook about linear algebra, and even if you change the temperature, the textbooks are going to look alike.[00:06:35] Speaker: So there's no way you could scale to like millions of samples. And the way you do that is by creating prompts that have some seeds that make them diverse. In our case, the prompt, we would ask the model to generate a textbook, but make it related to an extract from a webpage. And also we try to frame it within, to stay within topic.[00:06:55] Speaker: For example, here, we put like an extract about cardiovascular bioimaging, [00:07:00] and then we ask the model to generate a textbook related to medicine that is also related to this webpage. And this is a really nice approach because there's so many webpages out there. So you can. Be sure that your generation is not going to be diverse when you change the seed example.[00:07:16] Speaker: One thing that's challenging with this is that you want the seed samples to be related to your topics. So we use like a search tool to try to go all of fine web datasets. And then we also do a lot of experiments with the type of generations we want the model to generate. For example, we ask it for textbooks for middle school students or textbook for college.[00:07:40] Speaker: And we found that like some generation styles help on some specific benchmarks, while others help on other benchmarks. For example, college textbooks are really good for MMLU, while middle school textbooks are good for benchmarks like OpenBookQA and Pico. This is like a sample from like our search tool.[00:07:56] Speaker: For example, you have a top category, which is a topic, and then you have some [00:08:00] subtopics, and then you have the topic hits, which are basically the web pages in fine web does belong to these topics. And here you can see the comparison between Cosmopedia. We had two versions V1 and V2 in blue and red, and you can see the comparison to fine web, and as you can see throughout the training training on Cosmopedia was consistently better.[00:08:20] Speaker: So we managed to get a data set that was actually good to train these models on. It's of course so much smaller than FineWeb, it's only 30 billion tokens, but that's the scale that Microsoft data sets was, so we kind of managed to reproduce a bit what they did. And the data set is public, so everyone can go there, check if everything is all right.[00:08:38] Speaker: And now this is a recent paper from NVIDIA, Neumatron CC. They took things a bit further, and they generated not a few billion tokens, but 1. 9 trillion tokens, which is huge. And we can see later how they did that. It's more of, like, rephrasing the web. So we can see today that there's, like, some really huge synthetic datasets out there, and they're public, so, [00:09:00] like, you can try to filter them even further if you want to get, like, more high quality corpses.[00:09:04] Speaker: So for this, rephrasing the web this approach was suggested in this paper by Pratyush, where basically in this paper, they take some samples from C4 datasets, and then they use an LLM to rewrite these samples into a better format. For example, they ask an LLM to rewrite the sample into a Wikipedia passage or into a Q& A page.[00:09:25] Speaker: And the interesting thing in this approach is that you can use a model that is Small because it doesn't, rewriting doesn't require knowledge. It's just rewriting a page into a different style. So the model doesn't need to have like knowledge that is like extensive of what is rewriting compared to just asking a model to generate a new textbook and not giving it like ground truth.[00:09:45] Speaker: So here they rewrite some samples from C4 into Q& A, into Wikipedia, and they find that doing this works better than training just on C4. And so what they did in Nemo Trans CC is a similar approach. [00:10:00] They rewrite some pages from Common Crawl for two reasons. One is to, like improve Pages that are low quality, so they rewrite them into, for example, Wikipedia page, so they look better.[00:10:11] Speaker: And another reason is to create more diverse datasets. So they have a dataset that they already heavily filtered, and then they take these pages that are already high quality, and they ask the model to rewrite them in Question and Answer format. into like open ended questions or like multi choice questions.[00:10:27] Speaker: So this way they can reuse the same page multiple times without fearing like having multiple duplicates, because it's the same information, but it's going to be written differently. So I think that's also a really interesting approach for like generating synthetic data just by rephrasing the pages that you already have.[00:10:44] Speaker: There's also this approach called Prox where they try to start from a web page and then they generate a program which finds how to write that page to make it better and less noisy. For example, here you can see that there's some leftover metadata in the web page and you don't necessarily want to keep that for training [00:11:00] your model.[00:11:00] Speaker: So So they train a model that can generate programs that can like normalize and remove lines that are extra. So I think this approach is also interesting, but it's maybe less scalable than the approaches that I presented before. So that was it for like rephrasing and generating new textbooks.[00:11:17] Speaker: Another approach that I think is really good and becoming really popular for using synthetic data for pre training is basically building a better classifiers. For filtering the web for example, here we release the data sets called fine web edu. And the way we built it is by taking Llama3 and asking it to rate the educational content of web pages from zero to five.[00:11:39] Speaker: So for example, if a page is like a really good textbook that could be useful in a school setting, it would get a really high score. And if a page is just like an advertisement or promotional material, it would get a lower score. And then after that, we take these synthetic annotations and we train a classifier on them.[00:11:57] Speaker: It's a classifier like a BERT model. [00:12:00] And then we run this classifier on all of FineWeb, which is a 15 trillion tokens dataset. And then we only keep the pages that have like a score that's higher than 3. So for example, in our case, we went from 15 trillion tokens to 3. to just 1. 5 trillion tokens. Those are really highly educational.[00:12:16] Speaker: And as you can see here, a fine web EDU outperforms all the other public web datasets by a larger margin on a couple of benchmarks here, I show the aggregated score and you can see that this approach is really effective for filtering web datasets to get like better corpuses for training your LLMs.[00:12:36] DCLM, Nemotron-CC[00:12:36] Speaker: Others also try to do this approach. There's, for example, the DCLM datasets where they also train the classifier, but not to detect educational content. Instead, they trained it on OpenHermes dataset, which is a dataset for instruction tuning. And also they explain like IAM5 subreddits, and then they also get really high quality dataset which is like very information dense and can help [00:13:00] you train some really good LLMs.[00:13:01] Speaker: And then Nemotron Common Crawl, they also did this approach, but instead of using one classifier, they used an ensemble of classifiers. So they used, for example, the DCLM classifier, and also classifiers like the ones we used in FineWebEducational, and then they combined these two. Scores into a, with an ensemble method to only retain the best high quality pages, and they get a data set that works even better than the ones we develop.[00:13:25] Speaker: So that was it for like synthetic data for pre-training.[00:13:28] Post Training - AI2 Tulu, Smol Talk, Cohere Multilingual Arbitrage[00:13:28] Speaker: Now we can go back to post training. I think there's a lot of interesting post training data sets out there. One that was released recently, the agent instructs by Microsoft where they basically try to target some specific skills. And improve the performance of models on them.[00:13:43] Speaker: For example, here, you can see code, brain teasers, open domain QA, and they managed to get a dataset that outperforms that's when fine tuning Mistral 7b on it, it outperforms the original instruct model that was released by Mistral. And as I said, to get good synthetic data, you really [00:14:00] have to have a framework to make sure that your data is diverse.[00:14:03] Speaker: So for example, for them, they always. And then they see the generations on either source code or raw text documents, and then they rewrite them to make sure they're easier to generate instructions from, and then they use that for their like instruction data generation. There's also the Tool3SFT mixture, which was released recently by Allen AI.[00:14:23] Speaker: It's also really good quality and it covers a wide range of tasks. And the way they make sure that this dataset is diverse is by using personas from the persona hub datasets. Which is basically a data set of like I think over a million personas. And for example, in the tool mixture to generate like a new code snippet, they would give like the model persona, for example, a machine learning researcher interested in neural networks, and then ask it to generate like a coding problem.[00:14:49] Speaker: This way you make sure that your data set is really diverse, and then you can further filter the data sets, for example, using the reward models. We also released a dataset called Smalltalk, [00:15:00] and we also tried to cover the wide range of tasks, and as you can see here, for example, when fine tuning Mistral 7b on the dataset, we also outperformed the original Mistral instructs on a number of benchmarks, notably on mathematics and instruction following with ifevil.[00:15:18] Speaker: Another paper that's really interesting I wanted to mention is this one called Multilingual Data Arbitrage by Cohere. And basically they want to generate a data set for post training that is multilingual. And they have a really interesting problem. It's the fact that there isn't like one model that's really good at all the languages they wanted.[00:15:36] Speaker: So what they do is that like they use not just one teacher model, but multiple teachers. And then they have a router which basically sends the prompts they have to all these models. And then they get the completions and they have a reward model that traces all these generations and only keeps the best one.[00:15:52] Speaker: And this is like arbitrage and finance. So well, I think what's interesting in this, it shows that like synthetic data, it doesn't have to come from a single model. [00:16:00] And because we have so many good models now, you could like pull these models together and get like a dataset that's really high quality and that's diverse and that's covers all your needs.[00:16:12] Speaker: I was supposed to put a meme there, but. Yeah, so that was it for like a synthetic data.[00:16:17] Smol Models[00:16:17] Speaker: Now we can go to see what's happening in the small models field in 2024. I don't know if you know, but like now we have some really good small models. For example, Lama 3. 2 1B is. It matches Lama 2. 13b from, that was released last year on the LMSYS arena, which is basically the default go to leaderboard for evaluating models using human evaluation.[00:16:39] Speaker: And as you can see here, the scores of the models are really close. So I think we've made like hugely forward in terms of small models. Of course, that's one, just one data point, but there's more. For example, if you look at this chart from the Quint 2. 5 blog post, it shows that today we have some really good models that are only like 3 billion parameters [00:17:00] and 4 billion that score really high on MMLU.[00:17:03] Speaker: Which is a really popular benchmark for evaluating models. And you can see here that the red, the blue dots have more than 65 on MMLU. And the grey ones have less. And for example, Llama33b had less. So now we have a 3b model that outperforms a 33b model that was released earlier. So I think now people are starting to realize that like, we shouldn't just scale and scale models, but we should try to make them more efficient.[00:17:33] Speaker: I don't know if you knew, but you can also chat with a 3B plus model on your iPhone. For example, here, this is an app called PocketPal, where you can go and select a model from Hugging Face. It has a large choice. For example, here we loaded the 5. 3. 5, which is 3. 8 billion parameters on this iPhone. And we can chat with this and you can see that even the latency is also acceptable.[00:17:57] Speaker: For example, here, I asked it to give me a joke about [00:18:00] NeurIPS. So let's see what it has to say.[00:18:06] Speaker: Okay, why did the neural network attend NeurIPS? Because it heard there would be a lot of layers and fun and it wanted to train its sense of humor. So not very funny, but at least it can run on device. Yeah, so I think now we have good small models, but we also have like good frameworks and tools to use these small models.[00:18:24] On Device Models[00:18:24] Speaker: So I think we're really close to having like really on edge and on device models that are really good. And I think for a while we've had this narrative. But just training larger models is better. Of course, this is supported by science scaling laws. As you can see here, for example, when we scale the model size, the loss is lower and obviously you get a better model.[00:18:46] Speaker: But and we can see this, for example, in the GPT family of models, how we went from just a hundred million parameters to more than a trillion. parameters. And of course, we all observed the performance improvement when using the latest model. But [00:19:00] one thing that we shouldn't forget is that when we scale the model, we also scale the inference costs and time.[00:19:05] Speaker: And so the largest models were are going to cost so much more. So I think now instead of just building larger models, we should be focusing on building more efficient models. It's no longer a race for the largest models since these models are really expensive to run and they require like a really good infrastructure to do that and they cannot run on, for example, consumer hardware.[00:19:27] Speaker: And when you try to build more efficient models that match larger models, that's when you can really unlock some really interesting on device use cases. And I think a trend that we're noticing now is the trend of training smaller models longer. For example, if you compare how much, how long LLAMA was trained compared to LLAMA3, there is a huge increase in the pre training length.[00:19:50] Speaker: LLAMA was trained on 1 trillion tokens, but LLAMA3 8b was trained on 15 trillion tokens. So Meta managed to get a model that's the same size, but But it performs so much [00:20:00] better by choosing to like spend the sacrifice during training, because as we know, training is a one time cost, but inference is something that's ongoing.[00:20:08] Speaker: If we want to see what are like the small models reads in 2024, I think this mobile LLM paper by Meta is interesting. They try to study different models that are like have the less than 1 billion parameters and find which architecture makes most sense for these models. For example, they find that depth is more important than width.[00:20:29] Speaker: So it's more important to have models that have like more layers than just one. making them more wide. They also find that GQA helps, that tying the embedding helps. So I think it's a nice study overall for models that are just a few hundred million parameters. There's also the Apple intelligence tech report, which is interesting.[00:20:48] Speaker: So for Apple intelligence, they had two models, one that was like on server and another model that was on device. It had 3 billion parameters. And I think the interesting part is that they trained this model using [00:21:00] pruning. And then distillation. And for example, they have this table where they show that, like, using pruning and distillation works much better than training from scratch.[00:21:08] Speaker: And they also have some interesting insights about, like, how they specialize their models on specific tasks, like, for example, summarization and rewriting. There's also this paper by NVIDIA that was released recently. I think you've already had a talk about, like, hybrid models that was all interesting.[00:21:23] Speaker: And this model, they used, like, a hybrid architecture between state space models and transformers. And they managed to train a 1B model that's really performant without needing to train it on a lot of tokens. And regarding our work, we just recently released SmallM2, so it's a series of three models, which are the best in class in each model size.[00:21:46] Speaker: For example, our 1. 7b model outperforms Lama 1b and also Qt 2. 5. And how we managed to train this model is the following. That's where you spent a lot of time trying to curate the pre training datasets. We did a lot of [00:22:00] ablations, trying to find which datasets are good and also how to mix them. We also created some new math and code datasets that we're releasing soon.[00:22:08] Speaker: But you basically really spent a lot of time trying to find what's the best mixture that you can train these models on. And then we spent some time trying to like we also trained these models for very long. For example, small M1 was trained only on 1 trillion tokens, but this model is trained on 11 trillion tokens.[00:22:24] Speaker: And we saw that the performance kept improving. The models didn't really plateau mid training, which I think is really interesting. It shows that you can train such small models for very long and keep getting performance gains. What's interesting about SmallLM2 is that it's fully open. We also released, like the pre training code base, the fine tuning code, the datasets, and also evaluation in this repository.[00:22:45] Smol Vision Models[00:22:45] Speaker: Also there's, like, really interesting small models for text, but also for vision. For example, here you can see SmallVLM, which is a 2B model that's really efficient. It doesn't consume a lot of RAM, and it also has a good performance. There's also Moondream 0. [00:23:00] 5b, which was released recently. It's like the smallest visual language model.[00:23:04] Speaker: And as you can see, there isn't like a big trade off compared to Moondream 2b. So now I showed you that we have some really good small models. We also have the tools to use them, but why should you consider using small models and when? I think, like, small models are really interesting because of the on device feature.[00:23:23] Speaker: Because these models are small and they can run fast, you can basically run them on your laptop, but also on your mobile phone. And this means that your dataset stays locally. You don't have to send your queries to third parties. And this really enhances privacy. That was, for example, one of the big selling points for Apple Intelligence.[00:23:42] Speaker: Also, right now, we really have a lot of work to do. So many frameworks to do on device inference. For example, there's MLX, MLC, Llama, CPP, Transformers, JS. So we have a lot of options and each of them have like great features. So you have so many options for doing that. Small models are also really powerful if you choose to specialize them.[00:24:00][00:24:00] Speaker: For example, here there's a startup called Numind, which took small LM and then they fine tuned it on text extraction datasets. And they managed to get a model that's not very far from models that are much larger. So I think text extraction is like one use case where small models can be really performant and it makes sense to use them instead of just using larger models.[00:24:19] Speaker: You can also chat with these models in browser. For example, here, you can go there, you can load the model, you can even turn off your internet and just start chatting with the model locally. Speaking of text extraction, if you don't want to fine tune the models, there's a really good method of structure generation.[00:24:36] Speaker: We can basically force the models to follow a JSON schema that you defined. For example, here, we try to force the model to follow a schema for extracting key information from GitHub issues. So you can input free text, which is a complaint about a GitHub repository, something not working. And then you can run it there and the model can extract anything that is relevant for your GitHub issue creation.[00:24:58] Speaker: For example, the [00:25:00] priority, for example, here, priority is high, the type of the issue bug, and then a title and the estimation of how long this will take to fix. And you can just like do this in the browser, you can transform your text into a GitHub issue that's properly formatted.[00:25:14] What's Next[00:25:14] Speaker: So what's next for synthetic data and small models?[00:25:18] Speaker: I think that domain specific synthetic data is going to be, it's already important, it's going to be even more important. For example, generating synthetic data for math. I think this really would help improve the reasoning of a lot of models. And a lot of people are doing it, for example, Quint 2. 12 math, everyone's trying to reproduce a one.[00:25:37] Speaker: And so I think for synthetic data, trying to specialize it on some domains is going to be really important. And then for small models, I think specializing them through fine tuning, it's also going to be really important because I think a lot of companies are just trying to use these large models because they are better.[00:25:53] Speaker: But on some tasks, I think you can already get decent performance with small models. So you don't need to Pay like a [00:26:00] cost that's much larger just to make your model better at your task by a few percent. And this is not just for text. And I think it also applies for other modalities like vision and audio.[00:26:11] Speaker: And I think you should also watch out for on device frameworks and applications. For example, like the app I showed, or lama, all these frameworks are becoming really popular and I'm pretty sure that we're gonna get like more of them in 2025. And users really like that. Maybe for other, I should also say hot take.[00:26:28] Speaker: I think that like in AI, we just started like with fine tuning, for example, trying to make BERT work on some specific use cases, and really struggling to do that. And then we had some models that are much larger. So we just switched to like prompt engineering to get the models And I think we're going back to fine tuning where we realize these models are really costly.[00:26:47] Speaker: It's better to use just a small model or try to specialize it. So I think it's a little bit of a cycle and we're going to start to see like more fine tuning and less of just like a prompt engineering the models. So that was my talk. Thank you for following. And if you have [00:27:00] any questions, we can take them now. Get full access to Latent Space at www.latent.space/subscribe

We start off this podcast with Jerry's chance encounter with Bill Belichick on 6th Ave, we review the loss to the Saints, preview the Ravens matchup, and make our picks for Week 15. Hang in, folks.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Béatrice Brugère est notre invitée dans les podcasts de l'ISP. Béatrice Brugère, comme chacun le sait, vous êtes magistrate. Vous avez été substitut du Procureur à la Cour d'appel de Douai, Magistrat du siège au TGI Paris, au contentieux des JIRS, Vice-Procureur au TGI de Versailles et vous êtes désormais première vice-procureur au TJ de Paris. Vous avez été réélue récemment secrétaire générale du syndicat Unité-Magistrat. Votre parole compte et s'entend régulièrement dans les grands médias. Vous avez écrit en 2024, un ouvrage plébiscité « Justice : la colère qui monte. Plaidoyer pour une refondation », lequel a reçu le prix du livre politique du Barreau de Paris et le prix Edgar Faure. Béatrice Brugère, il est évident lorsqu'on lit votre livre que vous avez une vision pour la Justice. Dans ce podcast, vous allez nous expliquer quelle est cette vision ? Quelle est votre idée de la nécessaire refondation de la justice ?

The Giants lost yet another game to the Cowboys on Thanksgiving Day. We discuss that loss, preview the Saints game, and make our picks for Week 14. Hang in there, folks. Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Another week, another loss, Giants fans -- this time in embarrassing fashion at home to the Tampa Bay Buccaneers. Enjoy this therapy session where we pretty much cover everything regarding this franchise, preview the Cowboys matchup, and make our picks for Week 13. Hang in.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

We hand out our midseason grades for MVP and most disappointing, discuss the move from Danny Dimes to Tommy Cutlets, and we make our picks for Week 12.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

The Giants lose to the Panthers in Germany in absolutely horrific fashion. Just brutal honesty in this podcast, folks. Hang in.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

There's a lot to talk about in this one, folks. No moves at the trade deadline, we review the Commanders game, preview the Panthers game and make our picks for Week 10. Hang in.Download the Underdog fantasy app and sign up with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, Nebraska; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Welcome to ohmTown. The Non Sequitur News Show is held live via Twitch and Youtube every day. We, Mayor Watt and the AI that runs ohmTown, cover a selection of aggregated news articles and discuss them briefly with a perspective merging business, technology, and society. You can visit https://www.youtube.com/ohmtown for the complete history since 2022.Articles Discussed:No, You're a Disputanthttps://www.ohmtown.com/groups/roundersgear/f/d/arkansas-lottery-scratch-off-disputants-ordered-to-split-500k-prize/Steams Built in Recordinghttps://www.ohmtown.com/groups/nonsequiturnews/f/d/steams-built-in-game-recording-is-now-available-to-all/TGI isn't being spent.https://www.ohmtown.com/groups/mobble/f/d/bankrupt-tgi-fridays-has-50-million-in-unused-gift-cards/Oh, that Hertz.https://www.ohmtown.com/groups/four-wheel-tech/f/d/hertz-apologizes-for-threatening-to-have-customer-who-drove-25000-miles-in-rental-car-arrested/Just a Raccoon trying to catch a flight.https://www.ohmtown.com/groups/four-wheel-tech/f/d/raccoon-jumps-the-check-in-line-at-laguardia/Citizen Chemists, What could go Wronghttps://www.ohmtown.com/groups/greenagram/f/d/citizen-scientists-can-be-chemists-give-them-a-chance/Trucker Shortage in Japanhttps://www.ohmtown.com/groups/nonsequiturnews/f/d/japans-intense-trucker-shortage-may-inspire-a-drastic-solution-a-giant-conveyer-belt-between-cities/Nintendo says Switch 2 will be compatible.https://www.ohmtown.com/groups/nonsequiturnews/f/d/nintendo-says-its-switch-successor-will-be-backward-compatible-with-switch-games/ER Reboot is totally different.https://www.ohmtown.com/groups/the-continuity-report/f/d/warner-bros-fires-back-at-crichton-estate-over-claim-the-pitt-is-an-er-reboot-its-a-completely-different-show/Jarvis AI can take over computershttps://www.ohmtown.com/groups/technologytoday/f/d/google-accidentally-leaked-a-preview-of-its-jarvis-ai-that-can-take-over-computers/

We recap the disaster against Philly and preview the Monday Night matchup with the Steelers. We also take your questions and make our picks for Week 8.Sign up and deposit for Underdog HERE with promo code TGI to get up to $1,000 in bonus cash and a free pick: underdogfantasy.com or download the app.Must be 18+ (19+ AL, NE; 19+ in CO for some games, 21+MA & AZ) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369).Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

We preview the pivotal matchup against the Eagles and make our picks for Week 7. Chris discusses a couple conversations from the locker room while Jerry complains about stadium and airplane etiquette....Go to UnderdogFantasy.com, sign up with promo code TGI, and Underdog will give you a FREE PICK to use on your first cash Pick'em entry PLUS up to $1,000 in bonus cash when you deposit.Must be 18+ (21+MA & AZ, 19+ AL, NE) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369)Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

The Giants had a chance to keep pace in the NFC East but once again lost a heartbreaker in prime time. The defense did their part but the offense couldn't move the ball. Yes, we focus on the quarterback.Shout out to Big Blue BBQ for their incredible tailgate.Go to UnderdogFantasy.com, sign up with promo code TGI, and Underdog will give you a FREE PICK to use on your first cash Pick'em entry PLUS up to $1,000 in bonus cash when you deposit.Must be 18+ (21+MA & AZ, 19+ AL, NE) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369)Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

It's our first victory podcast of the year, folks. The Giants upset the Cleveland Browns by a score of 21 - 15. We recap the good (it's mostly good) as well as the bad. Enjoy!Go to UnderdogFantasy.com, sign up with promo code TGI, and Underdog will give you a FREE PICK to use on your first cash Pick'em entry PLUS up to $1,000 in bonus cash when you deposit.Must be 18+ (21+MA & AZ, 19+ AL, NE) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369)Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

We review the heartbreaker against Washington, we preview the matchup with Cleveland, and we make our picks for Week 3. Oh, and yes, some commentary on the NFC East. Enjoy!Go to UnderdogFantasy.com, sign up with promo code TGI, and Underdog will give you a FREE PICK to use on your first cash Pick'em entry PLUS up to $1,000 in bonus cash when you deposit.Must be 18+ (21+MA & AZ, 19+ AL, NE) and present in a state where Underdog Fantasy operates. Terms apply. Void in CO. Concerned with your play? Call 1-800-GAMBLER or visit www.ncpgambling.org; AZ: 1-800-NEXT-STEP (1-800-639-8783) or text NEXT-STEP to 53342; NY: Call the 24/7 HOPEline at 1-877-8-HOPENY or Text HOPENY (467369)Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

The incarceration of transgender, gender nonconforming, and intersex (TGI) people often leads to heightened discrimination and violence within prison walls. Miss Major Alexander L. Lee TGIJP Black Trans Cultural Center is standing up for these individuals, offering legal and emotional support while pushing for systemic change. Discover how they are working to defend dignity and transform the future for marginalized communities. Want to support Miss Major Alexander L. Lee TGIJP Black Trans Cultural Center? https://tgijp.org/ Find this episode at: https://great.com/great-talks-with/miss-major-alexander-l-lee-tgijp-black-trans-cultural-center/

O conflito entre Estados árabes e Israel que definiu boa parte dos problemas políticos no Oriente Médio. Foram seis dias mas pareceram pelo menos seis anos! Separe trinta minutos do seu dia e aprenda com o professor Vítor Soares (@profvitorsoares) sobre o que foi a Guerra dos Seis Dias. - Se você quiser ter acesso a episódios exclusivos e quiser ajudar o História em Meia Hora a continuar de pé, clique no link: www.apoia.se/historiaemmeiahora Compre o livro "História em Meia Hora - Grandes Civilizações"! https://www.loja.literatour.com.br/produto/pre-venda-livro-historia-em-meia-hora-grandes-civilizacoesversao-capa-dura/ Compre meu primeiro livro-jogo de história do Brasil "O Porão": https://amzn.to/4a4HCO8 Compre nossas camisas, moletons e muito mais coisas com temática História na Lolja! www.lolja.com.br/creators/historia-em-meia-hora/ PIX e contato: historiaemmeiahora@gmail.com Apresentação: Prof. Vítor Soares. Roteiro: Prof. Vítor Soares e Prof. Victor Alexandre (@profvictoralexandre) REFERÊNCIAS USADAS: - ARMSTRONG, Karen. Jerusalém: uma cidade, três religiões. São Paulo: Companhia das Letras, 2000. - CAMARGO, Cláudio. Guerras Árabe-israelenses. In.: MAGNOLI, Demétrio (org.). História das Guerras. São Paulo: Contexto, 2013. - NPR. Timeline: The Six Day War. Disponível em: https://www.npr.org/templates/story/story.php?storyId=10694216. - BBC Brasil. Os seis dias que já duram 50 anos: a guerra que mudou para sempre o Oriente Médio. Disponível em: https://www.bbc.com/portuguese/internacional-40200042. - TROY, G. (2018), The Zionist Ideas. Philadelphia: University of Nebraska Press - SANTOS, Claudio Roberto dos. Judeus contra Israel: uma análise crítica do sionismo. 2018. Trabalho de Conclusão de Curso (Graduação) – Faculdade de Filosofia, Letras e Ciências Humanas, Universidade de São Paulo, São Paulo, 2018. Disponível em: https://repositorio.usp.br/directbitstream/574a7296-fa43-40c3-a3a3-228543917353/2019_ClaudioRobertoDosSantos.TGI.pdf. Acesso em: 17 out. 2023. - SILVA, Daniel Neves. "Guerra dos Seis Dias"; Brasil Escola. Disponível em: https://brasilescola.uol.com.br/historiag/guerra-dos-seis-dias-poder-israelense.htm. Acesso em 25 de junho de 2024

2024 Legislative & Budget Priorities2024 #1 Priority Legislation AB 1955 (Ward, LGBTQ Caucus) – SAFETY ActThe Support Academic Futures & Educators for Today's Youth Act (SAFETY Act), would strengthen existing California protections against forced outings of LGBTQ+ students in schools; provide critical resources for parents and families of LGBTQ+ students to support them in working towards family acceptance on their own terms; and provide additional protections to educators who face retaliatory actions from administrators and school boards for seeking to create an inclusive and safe school environment. 2024 Priority “Sponsored” Legislation AB 1899 (Cervantes) – Gender-Inclusive Jury QuestionnairesThis bill requires Judicial Council to create a template juror questionnaire that is inclusive of gender expression and identity.AB 1979 (Ward) – Doxing Victims Recourse ActThis bill provides recourse for victims who have been harmed as a result of being doxed by allowing a victim to pursue civil action to receive restitution for the harms endured as a result of being doxed.AB 2258 (Zbur) – Protecting Access to Preventive ServicesThe bill codifies longstanding federal guidance that health plans and insurers must cover services that are integral to providing recommended preventive care – including anesthesia and polyp removal during a colonoscopy; placement, management, and removal of long-acting reversible contraceptives; and, ancillary and support services for PrEP including HIV and other STI screening – without cost sharing.AB 2442 (Zbur) – Expedited Medical Licensure for Gender-Affirming CareThis bill requires the expedited processing of licensure applications by the Medical Board of California, the Osteopathic Medical Board of California, the Board of Registered Nursing, the Physician Assistant Board, the Board of Behavioral Sciences, and the Board of Psychology for applicants demonstrating a commitment to providing gender-affirming health care or gender-affirming mental health care services within their licensed scope of practice.AB 2477 (Zbur) – Foster Care Cash SavingsThis bill permits youth transitioning to adulthood from foster care the chance to grow the best financial safety net possible by updating state law to clarify that young adults have the ability to accumulate cash savings while in foster care.AB 2498 (Zbur) – California Housing Security ActThis bill aims to prevent individuals from falling into homelessness by providing rent subsidies to a range of rent-burdened populations, including former foster youth, older adults, adults with disabilities, people experiencing unemployment or homelessness, and recently incarcerated people.AB 3031 (Lee and Low) – Statewide LGBTQ+ CommissionThis bill establishes a Statewide LGBTQ+ Commission to serve as a state-level focal point for identification of key issues for the Caucus to prioritize in the future.SB 11 (Menjivar) – California State University Mental Health [Two-Year Bill]This bill would require the CSU to decrease the ratio of students to mental health counselors to address increased student needs and work to create a pipeline for CSU students to become mental health professionals. Also, this bill would increase data collection on CSU's mental health services and student wellbeing.SB 729 (Menjivar) – Health Care Coverage for Infertility and Fertility Treatment [Two-Year Bill]This bill would expand access to fertility care for Californians, including coverage for in vitro fertilization (IVF). Also, this bill would revise the definition of infertility to ensure same-sex couples are covered by health care insurance and are treated without discrimination.SB 954 (Menjivar) – Youth Health Equity + Safety (YHES) Act This bill seeks to address the sexually transmitted infection (STI) epidemic among California youth and improve equitable public health outcomes statewide by expanding teen access to condoms in schools and communities.SB 957 (Wiener) – SOGI Data CollectionThis bill requires the California Department of Public Health (CDPH) to collect sexual orientation and gender identity (SOGI) data from third-party entities, including local health jurisdictions, on any forms or electronic data systems, unless prohibited by federal or state law. The bill also requires CDPH to provide an annual report to the public and to the Legislature on its efforts to collect, analyze, and report SOGI data.SB 959 (Menjivar) – TGI Resources WebsiteThis bill establishes an online resource for transgender, gender diverse, and intersex (TGI) people and their families to combat misinformation and provide accurate information about access to trans-inclusive health care, existing legal protections for patients and health care providers, and other available support services.SB 990 (Padilla) – LGBTQ+ Disaster Relief PlansThis bill requires Cal-OES to consult with LGBTQ+ organizations and advocates in the community when creating the State Disaster Plan.SB 1278 (Laird) – World AIDS DayThis bill enshrines December 1st as World AIDS Day, a day globally recognized in solidarity with people affected by HIV.SB 1333 (Eggman) – HIV Data SharingThis bill requires state and local health department employees and contractors to annually sign the agreement and would repeal the annual review of the agreements. Additionally, this bill authorizes disclosure to other local, state, or federal public health agencies or to medical researchers when confidential information is necessary for the coordination of, linkage to, or reengagement in care for the person.SB 1491 (Eggman) – LGBTQ+ Higher Education EquityThis bill, beginning with the 2026–27 school year, requires the Student Aid Commission to provide a written notice to students who receive state financial aid regarding whether their postsecondary educational institution has an exemption from either the Equity in Higher Education Act or Title IX on file with the commission.  2024 Endorsed “Supported” Legislation AB 1810 (Bryan) – Incarcerated Peoples' Menstrual ProductsCaucus Co-Author: Assemblymember Zbur This bill ensures that any incarcerated person and/or youth who menstruates or experiences uterine or vaginal bleeding has ready access to, is allowed to use, and continues to use materials necessary for personal hygiene without having to request them.AB 1825 (Muratsuchi) – The California Freedom to Read ActCaucus Principal Co-Author: Assemblymember Ward This bill prohibits public libraries from banning books based on partisan or political reasons, view point discrimination, gender, sexual identity, religion, disability, or on the basis that the books contain inclusive and diverse perspectives.AB 3161 (Bonta) – Equity in Health Care Act: Ensuring Safety and AccountabilityCaucus Co-Author: Assemblymember Jackson This bill requires hospitals to analyze patient safety events by sociodemographic factors, like race, ethnicity, language, sexual orientation, and disability status. This will allow us to see the disparities in health that communities of color and LGBTQ communities are facing. Additionally, AB 3161 requires hospital safety plans to include a process for addressing racism and discrimination and its impacts on patient health and safety.SB 1022 (Skinner) – Defending Housing, Employment, and Other Civil Rights ViolationsCaucus Co-Author: Senator Wiener This bill empowers the Civil Rights Department (CRD) to stop systemic workplace discrimination by doing the following: (1) Clarify that deadlines that apply to individual complaints do not apply to complaints initiated by CRD or to group/class claims being prosecuted by CRD; (2) Allow CRD to rectify longrunning civil rights violations for the benefit of all victims, not only recent victims; (3) Allow CRD to pause investigations when the parties agree; and, (4) Allow housing discrimination cases to be brought in any county where CRD has an office.  May Revise Budget Priorities Preserve all funding for the LBTQ Women's Health Equity Initiative Fund within CDPH Office of Health Equity's Gender Health Equity Section by authorizing existing funds to transfer from FY23/24 to FY24/25.Reject proposed cuts to the CYBHI – Public Education and Change Campaign funding within CDPH Office of Health Equity to ensure LGBTQ+ preventive mental health programs are prioritized including local LGBTQ organizations and the statewide LGBTQ campaign, and replace proposed cuts with a more equitable level of funding reduction.Reject proposed cuts for “The Future of Public Health” initiative at CDPH Office of Health Equity to ensure LGBTQ community services within local health departments are supported for sexual health and harm reduction programs.Support requested expenditure authority of $725,000 with Department of Health Care Services (DHCS) to support addition of intersexuality to voluntary self-identification information to be collected by state departments and entities, pursuant to the requirements of AB 1163 (Lesbian, Gay, Bisexual, and Transgender Disparities Reduction Act).Support requested expenditure authority of $710,000 with Department of Public Health (CDPH) to implement system changes to collect voluntary self identification information pertaining to intersexuality in the course of collecting demographic data, pursuant to the requirements requirements of AB 1163 (Lesbian, Gay, Bisexual, and Transgender Disparities Reduction Act).Support requested expenditure authority of $718,000 with Health Care Access and Information (HCAI) to to support implementation of required planning by hospitals for increasing the diversity of procured vendors, pursuant to the requirements of AB 1392 (Rodriguez), Chapter 840, Statutes of 2023.  Priority Budget Requests (In Alphabetical Order) ADAP Rebate Fund Loan Reduction & Modernizations – This budget request reduces the Governor's proposed $500 million loan from the AIDS Drug Assistance Program (ADAP) Rebate Fund to the General Fund (GF) to $250 million, of which $5 million of the loaned ADAP-to-GF must go towards SB 954 (Menjivar, 2024), the YHES Act. Additionally, this budget request seeks the following modernizations to ADAP: (1) ADAP and PrEP-AP eligibility increase from 500% Federal Poverty Level (FPL) to 600% FPL – $3.5 million (one-time); (2) Harm Reduction Clearinghouse Increase: $10 million (one-time); (3) Health Insurance Premium Payment Cap on Premium Payments Lift: $3.5 million (one-time) & $7 million (ongoing); (4) TGI Wellness and Equity Fund: $5 million (ongoing); and, (5) Needs assessments and analyses for both gap identification of client navigation and retention services, as well as PrEP Navigation Program: $400 thousand (onetime).California Coalition of Transgender Immigrants – This budget request seeks $250,000 in funding to be divided into three programs to help bring equity, justice, and inclusion for Transgender, Gender NonConforming, and Intersex (TGI) immigrants: (1) Trans Immigrant Asylee program – $150,000; (2) Trans Inter-Sectional Unity program – $50,000; and, (3) Trans Emerging Leadership and Artist program – $50,000.Raise-A-Child Foster Family Recruitment & Retention Expansion – This budget request seeks $1 million in funding to accelerate the expansion of Raise-A-Child services throughout California to go towards: (1) Recruitment Promotion Campaigns; (2) Community Events and Engagement; (3) Virtual Information and Orientation Sessions; and, (4) Technical Assistance and Support.Renewal of Preservation of LGBTQ+ History Program Historical Archives – This budget request seeks to renew previously allocated funding for the “Preservation and Accessibility of California's LGBTQ+ History Program,” which is a competitive grant program that is administered by the California State Library. This program supports LGBTQ+ archives of all sizes for projects that work to preserve and make publicly accessible collections relevant to the LGBTQ+ movement, culture, experience, and/or history in California, as well as provides vital information services, including research opportunities, youth engagement, and academic enrichment. Specifically, this San Francisco Harvey Milk Plaza ADA Updates – This budget request seeks to invest $5 million in funding to be used towards the installation of a new ADA-compliant main stair and a new escalator to access the entrance to the Castro Muni Station for Harvey Milk Plaza. AB 1955 (Ward, LGBTQ Caucus) – SAFETY Act 

Nadine Alameh, Executive Director Taylor Geospatial Institute joins Megan Lynch to preview TGI's Inaugural Town Hall Event on May 22

Ask anyone in business long enough to find success and they'll likely tell you that relationships got them there. Cosmo Tires, operated by Tire Group International, is no different and is continuing to lean on its uncanny ability to cultivate strong, family-first partnerships as a cornerstone for growth."With everything that we do as a brand, if it's not all authentic and we're not able to reach the consumers where they interact with brands in the marketplace, then we lose some of that connectivity. It can get filtered, and we want to make sure that we capture everything from the street, take it all the way into the organization, and feed it directly into product development," says Dominick Montouri, the chief strategy officer for Tire Group International. "We want to make sure we keep that chain very, very short, but also keep it authentic and you don't lose any of those key attributes about what really matters to consumers in our space."In this episode of What's Treading with Tire Review, Montouri shares insights into how Cosmo Tires plans to expand its footprint and its engagement with consumers and partners, ensuring that the brand's evolution is just as much about meaningful connections and solutions in the tire industry as it is growth.Want more What's Treading? Click here.Tire Review: www.tirereview.comAAPEX: www.aapexshow.com

Marge Cole, Director of Consortium Engagement at TGI joins Megan Lynch discussing how AI could boost the work being done on geospatial science in the St. Louis region. 

We discuss The Ghost Inside's brand new album “Searching for Solace” with vocalist Jonathan Vigil, TGI are back and better than ever with 11 monster new tracks. Jonathan lets us know about recording the album, finally heading back to Australia in September, life in Vegas his twitch stream, LA Sports and a whole bunch more!

TGI(good)F, bro! Let's get it!TODAY ON THE SHOW:EASTER.EGG.ROULETTE is back!Spring Break BUST!What movie did you WALK OUT of???Who is the SMARTEST in the ROOM?!?+ SOmuchMORE!!

Our first ever demo day aimed for 15-20 people and ended up ballooning to >200 and covered in the news. We are now running the 2024 edition in SF on Feb 23: Latent Space Final Frontiers, a startup and research competition in “The Autonomous Workforce”, ​”Beyond Transformers & GPUs”, and “​Embodied AI”. RSVP here! You can find all LS online/IRL events on our new calendar. Super Early Bird tickets have just gone on sale for AI Engineer World's Fair, June 25-27!Today we have the honor of hosting two of Together AI's co-founders: Ce Zhang (CTO) and Vipul Ved Prakash (CEO). This is a rare opportunity to recap the history of the company since our last check-in with Tri Dao (Chief Scientist), some of their big releases, and do a deep dive into the state of the AI inference market. Together has emerged as one of the most consequential new startups in the new AI summer, last announcing a ~$100m Series A raise in November (at a ~$360-565m valuation). But there are at least three Togethers - Together the Research Lab, Together the Fine Tuning & Inference platform, and Together the custom models service. As we clarify on the pod, the overarching philosophy of Together is the ability to improve on all these fronts simultaneously by being “full stack”, from the lowest level kernel and systems programming to the highest level mathematical abstractions driving new model architectures and inference algorithms.Bringing Research and Industry TogetherIn just one year, Together has been behind some of the most exciting research in AI:* RedPajama, a fully open source dataset for model pre-training which mirrored the Llama1 recipe. Then followed by RedPajama2, a 30T tokens dataset of filtered and de-duplicated tokens. * RedPajama-INCITE-3B and 7B, which were SOTA in a few benchmarks at the time of release. * FlashAttention-2, developed by Together's Chief Scientist Tri Dao. We covered FA-2 in a previous episode with him.* Mamba-3B, the most promising transformer-alternative model that they released in collaboration with Cartesia. * StripedHyena, a SOTA graft of Hyena state space models and transformer models together* Medusa, an alternative to speculative decoding that lets you use multiple decoding heads instead of a draft model. * MonarchMixer, which was one of the most popular orals at NeurIPS 2023. It's an approach to transformers that replaces many of its core parts with Monarch matrices for better computational efficiency. And I'm sure we missed something! As Vipul reveals, almost 50% of Together staff is researchers, and two of their co-founders (Chris Ré and Percy Liang) are professors at Stanford, so we can expect a lot more here.Bringing “Disaggregated” GPUs TogetherOn their cloud, they offer inference as a service, fine-tuning, pre-training, etc, but unlike other providers they think of themselves as a disaggregated cloud. Today, they have ~8,000 A100 and H100 GPUs on their platform (an exclusive revealed on the pod!) totaling over 20 exaflops of compute, but instead of just buying more and putting them in a cluster and then exposing a `us-east-1` option for customers, they are taking heterogenous compute sources and adding a unified layer on top of it for developers to consume. Building on Ce's research, Together's GPU Clusters are taking on comparable AWS and GCP offerings in both cost and speed:Take the Hessian AI center in Germany or the DoE's INCITE; they have GPUs that they want to share with researchers, but they lack the cloud layer over it. Similarly, there's starting to be more and more differentiation amongst types of GPUs: H100s, A100s, MI3000s, etc. Each of them has different availability and performance based on task, and the end user shouldn't have to be an hardware expert to run inference on a model, so Together abstracts a lot of that away.A big theme of the Together inference stack, a “bag of 50 tricks” that we discuss on the pod, is also “hardware-aware” algorithms like FlashAttention and Mamba, which further emphasize the benefits of co-developing everything together:Special Focus: Transformer AlternativesAs we mentioned above, they are also funding a lot of research in Transformer alternatives. To reiterate a few points on why they matter:* Longer context is not the motivation for sub-quadratic architectures: Transformers don't inherently have hard limitations on context size, but they just get extremely expensive. When developing sub-quadratic alternatives, you easily enable very long context, but that's now how you should compare them. Even at same context size, inference and training is much cheaper on sub-quadratic architectures like Hyena.* Emergence of hybrid architectures: a lot of early conversations have been around the “post-Transformers” era, but it might be more like “half-Transformers”. Hybrid architectures could have split layers with some transformer-based and some state-space ones. One of the challenges is that a lot of hardware kernels are optimized for transformer operations, so you'd lose a lot by moving away completely.* Higher speed = higher GPU throughput: if we could reach the same benchmark performance on subquadratic architectures, it'd solve a lot of the GPU crunch. Today we peak at ~170 tok/s on inference in some open models; if we could reach 5,000 tok/s on the same card, you'd be able to serve 30x more customers on the same hardware. As a cloud provider, you're obviously incentivized to get there.We had a lot of fun chatting with the Together guys and we covered a lot of ground, so enjoy the conversation!Note: This is the first episode of a “cloud providers mini-series”. We have Erik from Modal and Ben from Replicate coming up next!Video PodcastJoin us to watching the video version of this pod on our snazzy YouTube!Show Notes* Together AI* RedPajama Dataset v1 Announcement* RedPajama Models v1 Announcement* Together Embeddings* StripedHyena-7B* Mamba-3B-SlimPJ* Vipul's X thread on Anyscale* Vipul's Razor* SemiAnalysis' "Inference Race to the Bottom" post* Chris Ré* Mike Conover's episode* Slim Pajama by Cerebras* Dolma by AI2* Jina AI* Tengyu's Voyage AITimestamps* [00:00:00] Introductions* [00:00:43] Origin and current state of Together.ai* [00:02:15] Transition from Apple to Together and the vision for open AI* [00:04:54] How Chris Ré introduced Ce and Vipul* [00:08:43] How RedPajama came to be* [00:13:34] Model training and Transformer alternatives* [00:15:37] DSIR and the importance of data in LLMs* [00:21:19] Inference vs Fine-tuning vs Pre-training usage on Together* [00:23:20] Together's GPU stash* [00:27:02] Why standardization of inference metrics is important* [00:29:26] Building moats in AI inference* [00:31:49] Federated vs disaggregated cloud computing* [00:34:57] Opportunities for improvement in the inference stack* [00:36:13] Anyscale benchmarking drama* [00:41:27] Not just an inference platform* [00:43:50] Together Embeddings and the future of embedding models* [00:45:53] State space models and hybrid architectures* [00:53:52] The need for 5,000 tokens/s speed in AI inference* [01:00:23] What's the most interesting unsolved question in AI?TranscriptAlessio [00:00:00]: Hey, everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO in Residence at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol.ai.Swyx [00:00:14]: Hey, and today we're together with Together. Welcome to the studio, guys.Ce / Vipul [00:00:20]: Thank you.Swyx [00:00:21]: I don't know how you typically give self intros, but does anyone want to go first? How do we get our audience acquainted, especially to who's speaking, because it's unusual for us to do a four-person pod. Yeah.Ce [00:00:33]: Hi, everyone. I'm Ce. I'm one of the co-founders of Together and the CTO, working with the team on technical things.Vipul [00:00:40]: I'm Vipul Ved Prakash, co-founder and CEO of Together.Swyx [00:00:43]: I always consider you guys as one of the sort of all-in-one companies. I always want to say labs, but I feel like you're not a lab. What is the sort of origin of Together, and then what is it today? I feel like it used to be Together.xyz, and then now you're Together.ai.Vipul [00:01:00]: I think fundamentally, Together is about open and independent AI systems. We think this is one of the most consequential technologies of our time, and when we started the company in June 2022, our focus was to build a platform for open source, independent, user-owned AI systems. One way to think about it is big labs, frontier model labs, have built their own platforms for developer platforms for their models. We think of Together as a platform for everything else, whether these are open models, whether these are models being built by companies that are owned by them. Our sort of XYZ roots, we have a fairly deep decentralization and open ethos that kind of reflects in all our platform and strategy and business. And we also, the way we structure our cloud is by combining data centers around the world instead of, you know, we are today not located in hyperscalers, we have built a footprint of AI supercomputers in this sort of very disaggregated, decentralized manner.Alessio [00:02:15]: I know before Together, you were at Apple, so you go from like the most walled garden, private, we don't say anything company, to we want everything to be open and everybody to know somebody. What maybe did you learn from like the Apple way of being super close and polished and maybe what are you taking now to Together to make it open, but also a very nice developer experience?Vipul [00:02:37]: Yeah, I would say, you know, one sort of my, you know, background has been in open source for a long time. One of the first things I created was a collaborative spam filter, you know, this was back in the day. It's called Vipul's Razor. And it became quite popular. And the first company I founded called CloudMark was built around, you know, taking open source and building both an open side of it and a commercial product around it. I think Apple is sort of very focused on providing this amazing experience to its customers with, you know, most of the technology sort of hidden behind the product. And certainly the focus on fluidity and applying complex technology to make everyday things simple is something that Apple does really well. And, you know, that's been a sort of big part of how we think about our developer platforms. I think it informs it. The other thing is that during my years at Apple, we, you know, worked a lot on deep learning. And one of the things that was sort of very viscerally accessible to me was how well these systems worked. We, you know, we built an open domain Q&A system. This was based on Facebook's LSTM paper in 2016. And it was remarkable because we had a parallel system based on sort of information retrieval techniques, which is extremely complicated, didn't work that well. And you know, this thing we wrote in a week was just incredible performance. So I think some of those experiences, at least for me personally, sort of were creating this roadmap of how important and powerful this technology is. And you know, when the scaling loss paper was published, I was very clear, like it was in some ways something very profound. We've never had algorithms that improve in capabilities with scale out. So this is almost a new era of computing. So that's been, I think, the influence of Apple, my years at Apple, really for me, like crystallized the value of what we are doing together.Alessio [00:04:54]: And how did you decide to join forces? Because you did a postdoc with Chris Ré at Stanford. You know, we already had Tri Dao from Together and we talked about Hazy. What was like the meeting of the mind of, hey, I come from like the more technical postdoc assistant professor background and we've got yet a more product thing. What got you excited to like build this now?Ce [00:05:15]: So we have been working on this together, Chris, in the essentially last like 10 years, right? So it was like a machine learning system 10 years ago was like Power BI's graphic model, right? And then convolutional neural network and then all the foundation model that we see today. But if you look at this, I think that fundamentally the thing we are actually optimizing is actually not that different. It's always about data movement across essentially all the stacks, right? So when you do distributed like computing, it's about communication across different machines. When you do, for example, flash attention, it's about data movement at a different essentially memory hierarchy, right? So we have been doing this in the last 10 years and seeing the field start grow, grow, grow. So we kind of feel the current kind of this like wave of technology is actually the perfect time to actually bring all the research essentially into something real. And we are super lucky that we got introduced to Weibo, right? And then we hope to join forces and bring this to real world.Swyx [00:06:10]: It's an unusual team of like sort of research and industry. Like you've been like a third or fourth time founder now. Third time founder, yeah. And so like what is your first order of business when you like set up together? Like how do you sort of put something like this together? Oh my God, I'm going to use this word so much.Vipul [00:06:27]: I feel AI companies are really kind of driven by research. And Chris and I had been talking about how to reduce the cost of building models. We felt that there aren't really big data modes around foundation models. They are built from a subset of the web. What is difficult is the cost of capital to build these. And one of the ways in which you can reduce this cost is by making more efficient systems. With that, it was really about finding the right set of co-founders and team. In fact, when Chris introduced me to Ce, and I think within the first five minutes of talking to Ce, I was like, we are starting this company. And our early focus was thinking about this more sort of disparate set of resources, you know, GPUs around the internet. Can we use those to build? And we really have to compress communication for, you know, when we do gradient averaging, there's just a lot of traffic. And if you can reduce that somehow, you sort of open up the possibility of using cheaper compute, you know, across the network. And Ce's research for a decade has been in that subject. You know, and from there, finding, you know, other folks in the network, I think there is generally a lot of excitement and philosophical alignment around what we are doing, which, you know, we publish papers, we publish open source libraries and code, we build open models. And I think the people in academia in, you know, machine learning and NLP, that's really what they want to do. So I think that's been really a kind of kernel for, you know, composition of the company. And we're lucky to have, you know, at this point, attracted some of the best researchers in the field. So I think that's the most important thing. And, you know, the rest of it is sort of driven by us. A couple of these philosophies around independent systems and decentralization and good developer interfaces, you want to make it accessible. That's, you know, just as important. And the rest follows from there, I think.Alessio [00:08:43]: I want to try and fill in some of the blanks in the history of Together. I think people come on your website today and they say, you raised a hundred million dollars Series A. They're like, wow, these guys are like super legit company. But it feels like Red Pajama just came out a year ago. I remember we had Mike Conover in the studio, who had built Dolly at Databricks. And you announced it literally the morning we were recording. So we're like in the studio on our phones, looking at it. And it's like, wow, this is like the first time now there's like a good curated dataset to do open pre-training. So maybe let's start from there. Like, what was the motivation behind it? Why did you decide to do that? It's, datasets are one of the things that most people don't want to work on. They just want to do models, not datasets.Ce [00:09:27]: Yeah. So, yeah, first one is not the first, right? So I think it's actually built on a whole bunch of amazing effort the community already have. For example, Eleuther have the pile, right? There's a whole bunch of amazing datasets they have, like C4, right, from Google, right? So I think really get inspired by the impact those like datasets have on the community, right? So I think when we did Red Pajama, it was a time that people are really fascinated by Lama, the model, like Lama 1, right? Which I feel like decades ago, right? But it's kind of, people are really excited about the quality, right? So that's really like a big shift in people how to think about open model. People start to see hope, right? So, but the one problem of Lama is the data recipe is being described in a pretty detailed way in the paper, but the data is actually not there. So, and our original thinking is how about we take the recipe and we try to do our best effort reproduction and try to put it out, such that we can learn from our mistakes in the reproduction together, right? So that's essentially the original thinking behind Red Pajama. And we have been pretty happy and excited about what community have been kind of build on it. For example, there's a dataset called Slim Pajama, right? Which do deduplication over our data, right?Swyx [00:10:38]: From Cerebras, did they talk to you before?Ce [00:10:39]: Oh, yeah, yeah, yeah, yeah. So, yeah, so we are very good friends so we can discuss about technical perspective. We are pretty excited because I think it's kind of why we do Red Pajama in the first place is that people can actually build not only models, but also datasets essentially over that piece of artifact, right? So that's actually what inspired us to do the first version of Red Pajama dataset.Swyx [00:11:01]: Yeah, and then you released V2 maybe two months ago.Ce [00:11:04]: Yeah.Swyx [00:11:05]: 30 trillion tokens.Ce [00:11:06]: Yeah, 30 trillion tokens. So I think what's exciting about Red Pajama V2 is not only the number of tokens, but we start to kind of learn from Red Pajama V1. So one thing that we learned was that data quality is really the core, right? So you want to take this couple trillion token dataset and try to bring them down maybe to one trillion or two trillion, right? The way that you actually filter them, deduplicate them is not something that kind of pre-decided before you see the application, right? So you kind of want to have a modular framework to think about data quality, right? So like given application, let's automatically or maybe semi-automatically try to come up with a way to filter it down. So that's why in Red Pajama V2, we kind of overlay the dataset with like 40 different pre-computed quality signal, right? If you want to reproduce your best effort, like C4 filter, it's kind of like 20 lines of code, right? And this open up this opportunity you can actually put different filter together, learn the combination of filter. We are very excited to see what community actually come up with using Red Pajama V2.Swyx [00:12:11]: It was retrospectively so obvious that this is a good idea that I wonder how come more datasets don't do this. You release the dataset with all these toggles that you can turn on and off, right? And you can sort of tune up and down the quality in ways that you believe is important to you. Yeah, I just, it makes so much sense now in retrospect. Because everyone just publishes like their pipeline and then the end result. But what about all the intermediate stages? Yeah.Ce [00:12:35]: Yeah, so I think, so there are multiple things there. I don't think we are the only one like doing that. For example, like Doma from AI2, right? They have this very flexible format to actually put in those quality signals, right? Think like, we are actually calling them some, right? So you can actually load Red Pajama using their tool. That whole thing should work, right? So I think one fundamental thing that changed in the last year, essentially, in the beginning when people think about data, it's always like a byproduct of the model, right? You release the model, you also release the data, right? The data side is there essentially to show people, ah, if you train on this data, you'll get a good model. But I think what started to change is when people started building more and more of those models, people started to realize like different subset of data side is kind of valuable for different applications, right? The data becomes something to play with, right? So I think we are kind of lucky that we happen to release Red Pajama right at that point that we get this opportunity to actually learn from that.Alessio [00:13:34]: And you guys have a custom model training platform on Together 2. You have a bunch of stuff in there for data selection, like the DSIR and things like that. How did you decide to work on that versus, because you first started with like some of the fine tunes on LLAMA. Do you see a lot of interest there? And I know you've been doing a lot of research on state space models and other transformer alternatives. Like, do you also see that as something you'll keep working on this year and push more people towards?Vipul [00:14:02]: Yeah, I mean, we, you know, we think of how to make training more efficient and building models more efficient. Part of that is being able to select the right dataset. This is why you have signals, DSIR. You can start with a small dataset and find similar documents, build models with that. So we think it's an important part of the kind of model build tooling that, you know, sort of widely useful for people building different kinds of models. Similarly, you know, we are running into the limits of how fast you can make transformers. And we want inference at 5,000 tokens per second. I don't think we will get there with transformers and we need to learn longer sequences. Data, again, becomes very, very expensive with transformers. So I work on space state models and all the research that we are doing there. And hopefully other labs will pick up on this and make it a kind of important target for optimization. But we think that, you know, open source is a great place for this. We can provide these recipes for data and for training to our customers who are building, you know, custom models themselves. And, you know, we are quite excited about the sort of progress we are seeing there.Alessio [00:15:18]: Do you have some of these models available for inference on Together? Can people play around with a strictly, you know?Swyx [00:15:25]: Yeah.Vipul [00:15:25]: Yeah, they're available for inference on our serverless platform.Swyx [00:15:29]: I always try to be the person who asks about acronyms in case, you know, people want to understand. Should we explain importance resampling, you know, that kind of stuff?Ce [00:15:37]: Oh, yeah. So DSIR essentially, it's a fundamental idea. So it's one of the paper from Percy, right? So essentially, if you know what you are doing, you can actually use that as a very strong signal about what data to put in to insert training process, right? So that's essentially the fundamental idea, right? So, and then more concretely, right? So there are actually different versions of DSIR, right? So one version is like if you have a validation site, right? You can actually somehow measure the similarity between the validation site and also your pre-trained corpus and essentially subset, like the subset. And often there's actually like less targeted version of DSIR where you'll say, yeah, maybe Wikipedia is actually a very good corpus. Let's try to find more Wikipedia, right? And you can think about it in two ways, either as a way to come up with different weights for different data slices. Yeah, so as like filter type of step. Yeah, for a data set, or think about that as like data augmentation. So that's how, yeah, that's how we think about DSIR.Swyx [00:16:33]: That makes sense. I will have to read the paper to understand a little bit more. Because when you say things like, we have to know in advance what we were trying to do with the model, then we do importance resampling. That is against the principle of general intelligence, right? Like the point is to train AGI.Ce [00:16:48]: Yeah, so it depends on what do you mean by being general or generic, right? So I think, I mean, you can always take a meta-learning perspective that we know the distribution of tasks that we care about, right? So you can always go kind of up in the ladder of how general the whole thing is, right? But also for many of the customers that we are actually talking to, right, they have kind of very targeted application, right? The benefit you can get out of that is you could build a better open model, often smaller, often easier to do inference, if you know what you want, right? So I think the whole trade-off would be, and the x-axis would be how generic the whole thing will be. The y-axis would be not only the top accuracy, but also a whole bunch of the deployment cost, right? The size of the model, right? The robustness of the model. So I think different people will navigate the space in different way. And we want to be the platform, essentially, whatever point that you want, we have a solution for you.Swyx [00:17:43]: One more thing on data before we go deeper on state-space models. Are we running out of data? Can we go in order of magnitude? Can we go five orders of magnitude? How do both of you think about how much data we have and how much we need?Ce [00:17:55]: Yeah, so I think that's a very, very good question. So I don't think we are running out of data on Earth.Swyx [00:18:02]: Right, so think about it globally. Training data, training class data.Ce [00:18:05]: Yeah, yeah, so I think, I mean, some of them are not accessible, right? But I do think there are many organizations in the world have enough data to actually train very, very good models, right? So, I mean, they are not publicly available, right? But there are people who actually have access to those, right? So I think in general, right? So if you think about the data in the open space, right? So I guess that was specifically that you actually mean whether we are running out of data. I do think there need to be some way, right? That people who are training open models get connected with essentially data that's not internet data. So I think that channel need to be opened up for the open model to get more data, right? But I'm kind of on the optimistic side that the society will figure out a way that we can train open models that's beyond this internet data.Swyx [00:18:57]: Beyond internet, meaning books?Ce [00:19:00]: I mean, there are a lot of those, right?Swyx [00:19:02]: Books, right?Ce [00:19:02]: Transcripts, right? Videos, audios, right? So there are a whole bunch of data sources that we are not integrating into open data side, right? So, and maybe they shouldn't be open, right? So I think the community need to figure out a way, yeah, like the best balance, yeah? Such that we can have open models, but on the other hand, also have a reasonable collection of data that we can actually use.Swyx [00:19:29]: I think a lot of people think that, there's a theory that Whisper was released so that you could transcribe YouTube and then use that as a source of tokens. Then I talked to other researchers who are like, you know, YouTube has very low quality tokens. You know, do you want your model to talk like a live streamer from YouTube? Because that's what they're going to do. So it's not clear, like what the quality of this data could be.Ce [00:19:53]: Yeah, I guess that depends on your application, right? So I think as a platform, right? So our goal is whatever application that you have, yeah, so we have a platform that you can actually achieve your goal, right? So there are definitely applications that kind of make sense to speak like YouTube, right? So, but there are probably also other application that kind of more on the formal side, right? So I think there are going to be a diverse collection of models, both open and closed, right? So, and we kind of want to be the engine that powers that.Swyx [00:20:21]: There's a lot of people who own data sources who are doing the locally optimal thing and humanity as a whole is losing out. So like New York Times is swinging open AI, you know, Stack Overflow shut down their API, Reddit shut down their API, X, you know, made their own model, right? On Twitter data. We're just going to have all these like tiny little gardens of data that it would be useful in a general model, but everyone's just trying to make their own model. And it seems like globally suboptimal.Vipul [00:20:47]: I think you need to have some kind of a marketplace for figuring out how to get this, you know, data into models and have, I think we'll increasingly see more of that. You know, I think there's a positive aspect to it too. There is a incentive for creators to participate in a system, which is sort of more fair relative to, you know, the capture of value by an AI company that's taking their data. But I agree. I think this is a big open problem that needs to be solved. And I hope there will be, you know, serious efforts around it.Alessio [00:21:19]: Let's talk about the most precious resource on planet earth, GPUs. You have a lot of compute obviously, but you also have a lot of product pieces. You have inference, you have fine tuning, you have pre-training. What's the split in terms of usage? Do you see most people are just running inference on off the shelf models? Do you see maybe some last mile fine tuning?Vipul [00:21:40]: I would say right now, the top five models on our inference stack are probably all fine-tuned versions of open models. And we've seen- Who fine-tuned them?Swyx [00:21:51]: You fine-tuned them?Vipul [00:21:52]: They were fine-tuned by our customers.Swyx [00:21:54]: By your customers.Vipul [00:21:55]: You know, either on our platform or off our platform. And we are generally seeing that, you know, that is the sort of trend where you can get better quality on your task by sort of now easily adapting these models to your data. We also have, I would say, over 20 big model builds happening on the platform, which are customer. We see a lot of training and it's also somewhat surprisingly a more continuous kind of workload. We sort of imagine that this would be more episodic. You train a model and then you do inference. But what we find is, you know, we train a model and then they train the next version and then the next version, which sort of grows in scale. I would say training is still the bigger portion. Some ways inference is super linear to model quality. And as the models are getting better, there's more and more inference.Swyx [00:22:48]: Oh, because they're more useful. Yeah, they're more useful, yeah. So, okay, so training is bigger. This is actually consistent with what we've heard from Mosaic, that, you know, people think that training is sort of like a one-time deal. You do one big run and then you're done. It's never true. And so I'm interested in, like, putting some numbers and I don't know what you have disclosed or what you want to disclose, but, like, how many GPUs do you have? What is the equivalent amount of compute that you have? Because I understand that your GPU setup is different than what people typically think of, like, a giant data center somewhere, right?Vipul [00:23:20]: I don't think we have shared this number publicly. It's, you know, so this will be the first time, I guess. Like, we have close to 7,000 to 8,000 GPUs today. It's growing monthly.Swyx [00:23:31]: What class of GPU are they?Vipul [00:23:32]: They're mostly A100s and H100s.Swyx [00:23:35]: Okay.Vipul [00:23:36]: And probably more, I think, split towards H100s now. You know, we'll be sort of building this best-of-class hardware. So as there are other versions of these coming out later this year, we plan to have those in the fleet as well.Alessio [00:23:53]: I know when we talked last year, you were also using some of the supercomputers by the Department of Energy. There was kind of like a lot of random GPU compute in the world. Have you seen that kind of getting timed out? I think maybe a year ago, people were like, oh, yeah, you can use this GPU computer that is going to be end-of-life. Has the bar changed to give access to those resources?Ce [00:24:13]: From our perspective, it's actually getting better. Yeah, so from the community perspective, because many of the institutions in the world, they're actually investing in hardware, right? So for example, we are working with one of the institutes in Germany called Hessian AI, right, which gives us a lot of help on the compute side. So they start to have this very big GPU cluster, and they're actually sharing that with the community, right? And it's not super big, right, but also not a small one, right? So you start to see this, like, different lives that start to pop up, right? And because of the power of the community, they start to actually share that. So we actually find as a researcher today, it's probably easier for them to actually get a GPU than last year.Swyx [00:24:56]: Interesting.Alessio [00:24:56]: And then for you to buy them, what's the state of the market right now? Is it still extremely hard to get any? Do you have Jensen's phone number? Do you have like GM phone number? Do you guys get like the SDR because you're like under 10,000?Vipul [00:25:12]: NVIDIA is obviously motivated to help us, both as an investor and we are their customers. I would say the market is very tight still, and it's likely going to be this way for a while, is my sense that the demand for AI computing is just kind of ramped up very, very quickly, and it will take a while for supply to catch up.Swyx [00:25:37]: So how tight it is, and let's say compared to like a year ago, two years ago, what do you mean when you say tight? The things you want, you can't get?Vipul [00:25:42]: You can't get them immediately. They're sort of, you know, minimally like two to three months out. Any inventory that shows up tends to clear very, very rapidly. And, you know, we obviously sort of look at this in a very detailed and analytic. There is four to 5 million GPUs that will be sold this year from NVIDIA and others buying. And if you think about 512 to 1,000 GPU cluster for a company, that's 4,000 to 8,000 companies, right? So it's in some ways a very small number. In other ways, the cost of GPUs will be, you know, 80 to $100 billion, and then you layer servers and data center space and electricity on top of that, and that's, you know, close to $250 billion worth of kind of compute, which when you compare it to the cloud computing of today, you know, AWS's last year was $88 billion in revenue. So this is really kind of a build-out happening of AI hyperscalers. It is much more disaggregated, and it's very, very global. So, you know, we think that GPUs are going to be sort of a precious resource for a long time, and using them optimally is very valuable.Swyx [00:27:02]: Yeah.Alessio [00:27:02]: Our friend, Dylan Patel from Semianalysis, he wrote a post about the inference market recently and obviously mentioned you guys. In his post, he said, our model indicates that Together is better off using two A180 gig system rather than a H100-based system. The temperature and performance testing also point to Together utilizing speculative decoding. Any thoughts? Is Dylan right? I don't know, what's-Swyx [00:27:26]: What is his model, man? What does he know that they don't know? Yeah, exactly.Alessio [00:27:30]: I wanna know, I guess like from the outside, and sometimes we even do it, we try and speculate on what people are actually doing. So for the first time, now we have a former guest writing about a current guest. So we wanna know what you guys thought and maybe what are some of the misconceptions that people from the outside have on what it takes to run like a GPU cloud today?Vipul [00:27:50]: Yeah, big fan of Dylan's, by the way. I religiously read Semianalysis. I think there were some errors in that analysis. In particular, we were trying to decode it and one of the things we noticed is that it assumed that input tokens weren't being priced. So I think that may have been an error in the model. I also don't think that there's this assumption that people are running this at a loss. I think it's very expensive. You can't do that for very long. And there are trade-offs in terms of batch sizes you use and the kind of tokens per second performance that are kind of system trade-offs. We've done a lot of work. This is one of the key areas of research for us. So our inference stack is a combination of 50 different sort of tricks and techniques and we think there's a lot of room for optimization here. So whichever hardware provides better performance, whether it's H100 or A100s or L40s, we can sort of measure price performance on particular hardware and we tend to use that for that model or in some cases, certain customers have data streams which can be then optimized for a particular configuration regime. So we do fairly detailed work on how to make this more efficient and so it's hard to, from the outside, looking at memory bandwidth and estimating what's actually happening.Alessio [00:29:26]: How much of these 50 tricks are you giving to yourself and how many are you gonna open? Because we have three now, obviously Flash Attention 2 is open source. He mentioned he'd love to come work together because of how much you care about open source. Yeah, how do you weigh that as a CEO and CTO?Vipul [00:29:43]: A lot of it is open, right? Flash Attention, Flash Decoding, et cetera, and we publish something that's very generally universally useful. It's going to produce better open source AI. We tend to publish as open source. I think on the inference stack, there are open source inference stacks which are pretty good and definitely today, it gives us a competitive advantage to have the best one. So we are not sort of rushing out to release everything about it. It's not overall that additive to open source out there and it is particularly useful as a business for us to provide best price performance. Yeah, we make these decisions. We have discussions. Anything that we keep closed, we generally talk about it quite a bit and decide like this is the piece that is closed for today and it may not be the case six months from now. It may not matter as much.Ce [00:30:40]: Yeah, so I think being open is kind of very important, right? So I think the whole company actually built on this idea that there's going to be ecosystem built on our open models, right? And that's also how we are really lucky to attract this top group of talents to actually join us because of the dream and the mission that we have on our side to really facilitate the open ecosystem, right? So I think in general, it's like I think all the ideas should be open. So that's why we publish papers, right? We actually talk about ideas, right? So I don't think it makes any sense to keep idea like close, right? So there are some software artifact that are kind of really deeply embedded into our kind of own kind of like stack. It kind of only useful when you're trying to build a disaggregated cloud, right? Maybe at some point that we're going to be open as people said, right? But at this moment, right? So we are kind of busy actually building it, right? So that's probably kind of getting to the picture about when that piece is going to be open, right? But I think on the research side, the ideas and for our people to publish things, I think that's really, really important, right? So I think that's how we get talent. That's how I think we as a company going to move the field forward.Swyx [00:31:49]: I noticed that you never used the word federated learning or inference. Is there a distinction that you draw?Ce [00:31:55]: So, I mean, it's definitely not intentional, but I think federated learning is, have been used in so many different ways by so many different people. It starts to lose a very precise meaning about what that really mean, right? If you go back to the original Google paper of federated learning, I think that's very different from what people are talking about today when they say federated. Yeah, we kind of want to be really precise about it.Swyx [00:32:18]: And so your term is disaggregated.Ce [00:32:19]: Yeah, so as an infrastructure, right? So that's disaggregated.Swyx [00:32:22]: Aren't most clouds disaggregated? Like what's different about it?Ce [00:32:27]: So one way is that most of the cloud are disaggregated, but some of that is actually being exposed to the user, right? If you go to AWS, you do know which region you are in, right? So I think one thing that we are trying to do is you have this disaggregated cloud, not only about location or geographically where they are, but about this reliability and also this diversity of this infrastructure. So, and if we want to build a reliable, high-quality layer over that, the user actually don't know, right? What's actually happening under the cover, right? So I think that's one of the difference of the way that we are thinking about infrastructure.Swyx [00:33:06]: Yeah, a bit closer to Cloudflare than AWS. Yeah. Yeah. We have one question here, which we'll just throw out, it's kind of fun. So going back to this sort of inference stack piece, maybe if you had to pull out like a call for researcher or just like point out interesting areas of work that you're interested in, what pieces of the stack have the most opportunity for improvement?Ce [00:33:27]: Yeah, so I think the way we are thinking about the inference stack is, so there are multiple things that can happen, right? So you can do better algorithms, like speckle decoding, you can change the model architecture, you can go really crazy on the system side, right? And you can also code it on the hardware, right? So it's not really clear innovation on a single dimension will get you there. So the key thesis on our side is, if you only push on one direction, you are going to reach diminishing return really, really quickly. Yeah, there's only that much you can do on the system side, only that much you can do on the algorithm side. I think the only big thing that's going to happen is when you ask all those dimensions to actually compound, right? So to have algorithm, model, and system all come together, so I think that's how we reach the next 10 times improvement on inference, right? So I don't think there's a single dimension that is particularly important, but looking at this space in a joint way, right? Try to co-optimize jointly multiple dimensions, I think that's going to be really important for the community to look at.Vipul [00:34:28]: Yeah, we often see, I see numbers from the team and you have these multiple methods, not all of them compound. So you mix these together, it's still similar results and some combination of them will have this incredible effect that is really, really super interesting. So it's very systems, you know, a kind of broad systems approach to it that's the most effective.Swyx [00:34:51]: I think I finally get the name of the company, like- Bring it together, yeah. Everything needs to be automated together.Alessio [00:34:57]: All right, just quickly, how does all this work change, just like some of the architectures change? I know a mixture of experts like speculative decoding is a little less efficient because of memory bandwidth. How much of it do you invest when it's a maybe model-specific improvement versus more horizontal thing? Also, you're researching different architectures, so how much do you want to spend time optimizing what state of the art today versus what's coming next?Vipul [00:35:24]: We do spend time on what state of the art today as well as what's next. You know, the value we get from doing specific optimization, even for, you know, what works well for a particular model on A100s with a particular bus versus H100s, it's a worthwhile investment for us. So we will go down fairly deep into a specific architecture and specific hardware. It does also inform what works better where, and you don't have to take the same approach for, you know, every model and every sort of hardware setup. We can take these different approaches and we do have these multiple systems now. We know that this, you know, system B is better for mixed role and system C is going to be better for stripe tying or Mamba.Alessio [00:36:13]: Before we move on from inference, we need to talk about any scale of drama. So we're actually having Sumit on the podcast tomorrow, who also talked about, kind of came to your guys' support about how, yeah, how important it's not just like, oh, together saying this benchmark's not good because they look bad in it. How, I guess like, it's a hard question to ask, but like, why did you decide to just come out and say it? And how maybe does that also reflect the values that you guys have about open source and openness and kind of like being transparent about what's real and maybe hopes for standardizing some of these benchmarks to make it more clear?Ce [00:36:56]: So it's a great service and skills doing for the community, right? I mean, it's very hard to do benchmark. The moment you do benchmark comparing N players, right, N minus one will be unhappy. You have two tables, then maybe N of them will be unhappy, right? So it's a very great thing that they're doing. And in some of the work that we are doing, we actually use RMOperf, right? So it's a great thing that they're actually doing. So I think one thing about benchmark is, and probably the professor part of me are talking, is a good benchmark should think about how it's going to incentivize the field to actually move forward, right? So if the benchmark really become a kind of standard, how are people going to over-optimize to the benchmark if you are going to do that? And when people are doing that, what are we actually trying to incentivize, right? Will that move the world to a better place? Or will that essentially have every single player focus on marketing or spending time or money on something that actually do not matter on technical side, right? It's very hard to actually strike a balance, right? So I think the reason we kind of try to give feedback on the benchmark is kind of want to open up the discussion about how does the industry should come together and define maybe a common way that we compare with each other, right? So like how database people doing TPC, right? Maybe you should have something actually similar, right? So we are trying to start some of the conversation. So it's not really that we jump out to say it's not good because there's no way we can have a perfect benchmark. That doesn't really exist, right? So just try to kickstart a conversation that maybe we should come together and do something that the community agree and align with the benefit a user going to get, right? So just get the conversation started.Vipul [00:38:42]: I've spoken to the AnyScale team after that, and I think they had really great intentions. And partly, I think it felt very objective and everyone sort of had a reaction to it because it just didn't match their benchmarks that we've all run internally against different services. I think a common industry benchmark run by an independent party versus one of the vendors.Swyx [00:39:04]: Is there one that you appoint to?Vipul [00:39:06]: I don't think one exists today. I think there should be. We're having some conversations about someone setting one up. And there's lots of interesting aspects of this. Time to first token is a function of where the test was run from. There is different load on these services at different times of the day and weekday or weekend. So you have to measure that well. And I think if all of that were done very well by an independent source, that will be a very useful service to customers and in the services themselves.Swyx [00:39:39]: Yeah, I'll point people to artificialanalysis.ai, which is a new one that recently emerged. I don't know if they've done it right. It looks like a side project of a couple people. But I think it's in all the provider's interest to work with them. And ensure that there's an independent third party that's measuring these things, right? At least on the baseline. For me, what's worrying is more about what Toa was saying, which is, do these benchmarks skew things in ways that customers might not be mindful of? Like, what are these things overemphasizing that we might be missing? And I don't really know. It seems like a lot of these services bundled together, they're a version of quantization as well. So that means there's performance trade-offs, right? You're not comparing apples to apples, the same model itself, even though it's like a llama variant or whatever. So what do people trade off? They trade off latency, they trade off price. Obviously, those are the first two. But what else, right? What factors matter in an inference business?Ce [00:40:33]: Yeah, so I think there's also the throughput, right? So there's the time to first token, right? So, and then there are things that users do not often see, for example, the reliability, right? The capacity, right? So that also have impact on user experience at a global scale. Maybe not a single query, right? But in aggregation, you can also see a whole bunch of, like, whether you are emphasizing P50, P95, right? So the whole bunch of things that you can actually play with. And of course, there's also quality. So there are different ways to actually make the whole thing faster, specification, quantization, or combination of those, right? So yeah, so there are so many things to actually play with. So they probably need a benchmark that the protocol is transparent to make sure, like, it's very clear what we are doing and a whole bunch of check on the quality to make sure we are putting the right group of stories in the same table. So I think then essentially the user can actually navigate the space. So I think that's going to be good for everyone.Swyx [00:41:27]: Yeah, makes sense. It's a very important field and I think hopefully there's a good third party that emerges from this. So I just want to touch on one more piece, which is I think I'm appreciating from this discussion that fine tuning is a bigger part of your business than I thought. The other big player in fine tuning is Mosaic. Well, Mosaic is more training, but like there's a bunch of other players in the fine tuning space. If I was a prospective fine tuning customer, what do I come to you with? Do I come to you with my custom data and that's it? Do I also have to write the fine tuning code? What level of engagement do you do with your customers?Vipul [00:42:01]: I think across the spectrum, our customers are training models, pre-training models from scratch and many of them will bring their data sets, you know, user infrastructure and training stack to train their models. There are others who have trained smaller models and want to scale up, scale up across infrastructure, scale up across data. So we'll sort of help them do that. We will have customers who are sort of initially started a little bit more consultative. They have a particular task and idea in mind and we will help them get from there to the data set and the right model to achieve that task. So it's a spectrum and, you know, our goal is to, we're trying to productize as much of this as possible. So that the whole process can be fast and scalable. I would say there is a lot more understanding around fine tuning now, like even the last six months, there are, you know, source tools, recipes, literature, podcasts, discord channels where people are figuring out and it really is in many ways, one of the successes of open source is you have small collectives of, you know, engineers who have created, who are now creating the top models on open source leaderboards. And I have tried out all sorts of different sort of, you know, data recipes, creating synthetic data. Merging models. Merging models. So it's, that's really fun to see. And I think that sort of agency that exists now is exciting. And that is, we see a lot of that sort of being applied into products and, you know, more commercial models that people are deploying in their applications.Alessio [00:43:50]: And then just to, I guess, wrap up the together, it's almost becoming like a platform as a service, because now you release together embeddings. How did you get 92.5 accuracy on 32K retrieval? And do you think we're kind of like getting to embeddings or just like, we did everything that we could, you know, we're getting to like the most optimized it's gonna get and then we should just focus on models and inference or do you think there's still room there to improve?Ce [00:44:17]: Oh, I don't think we haven't even got started on embedding. Yeah. So I think there are so many things. So like embedding is really fundamental for many things, for example, rack, right? So deep in application. So that's how people bring knowledge in. That's also the fundamental piece when you want to build a better model, right? So that's give you this understanding about what actually get into the model. You can actually use that to actually build a better data set, get a better model, then get better embedding, you'll start this loop, right? Without the good embedding, the loop is not closed, right? So I think both on the quality side, how to embed more like dedicated semantics, like into those vectors, how to deal with negation, for example, right? So, and how can you make the whole thing really, really fast? So I think for the next couple years, yeah, we will see a whole bunch of new embeddings maybe of different size and much, much faster than today. Yeah, so I think it's a very active research area. I think people should invest more, yeah.Swyx [00:45:14]: I was surprised to see, I think Jina or, yeah, there's Jina AI, and then there's another guy, Tengyu's Voyage. They are coming out as startups purely focused on embeddings.Ce [00:45:25]: Yeah. Yeah, so I think it's a very, very important piece of the system, right? So you people haven't focused on a lot on them before, and they should definitely start to do that.Swyx [00:45:36]: Yeah. Why are the Chinese universities so good at embeddings? You know what I mean, right? Like the BGE and- Yeah, yeah, yeah.Ce [00:45:44]: So I don't know. We just released our first embedded model, so we still try to learn how to build an embedded model. Yeah, so ask me again in six months.Swyx [00:45:53]: I'll probably have more insight about how to build a better one. I just noticed that you saw 8002 was used to be at the top of the MTB chart, and then it's just like sliding down and down and down, and all the new models are coming out of China for some reason. And I'm like, I don't know what's going on there. So we cannot leave this discussion without talking about state space models. But first of all, how much of the company is dedicated to research? Like it's obviously like not production quality yet, but-Vipul [00:46:17]: I would say it's like 40, 45% I was counting this morning. That's huge.Swyx [00:46:22]: Yeah, so that's the biggest- It's a big investment. Yeah. Okay, well, I mean, it looks like it's paying off, so. And then high level, I will confess or admit or mention for the listeners who are also similarly skeptical, I did not used to care about long contexts because I was like, you know, 30K is enough, 100K is enough, right? I'm not, you know, modeling DNA sequences or anything like that. Why do I need long context? And I mean, first of all, I'll throw that open to you. But second of all, I think what Mamba did for me was change that perception of that. It's only about a long context. The only reason you want sub-quadratic architectures is for long context. Actually, that's not true. And it's also just more efficient to train, period. Right? I'll just leave that open to you. Like what's the motivation that people should keep in their heads? There are multiple things, right?Ce [00:47:09]: So one thing is that, I mean, the moment a model can do for long context well, so it often means that it's kind of cheaper. Yeah, so I mean, that's why it's kind of long. I mean, in principle, transformer can do long context. It's just very expensive. So I think what those like state-based models trying to do is try to push the size of the state, right? Like as small as possible. That's why it's kind of long context, right? And try to kind of like decouple this like quadratical dependency, right? To make sure you can have a much better execution pattern.One direct consequence of those is you can do long context really cheaply, but on the other hand, also introduce a whole bunch of benefit even you are not doing long context. Right? So I think that's actually probably equally important. Because data gets smaller, you can do really large batch size, right? You can actually be very faster. Right? So yeah. And another thing is like, one of the hypothesis that we have is, like in Stripe Hyena, it start to have a hybrid architecture, right? It has part of it has like state-based model and part of it is still the transformer. So different component probably deal with different things kind of better. So maybe by putting them together, by thinking about how information propagate, over this whole horizon of this context, you can probably get an even better quality model than transformer. Right? So I think that's why we are kind of invest a lot of things, on those models. Not only for the context, which is very important, but also for a whole bunch of benefit it could get.Swyx [00:48:42]: Yeah. How should people treat the distinction between Mamba and Stripe Hyena? Like what's the point of releasing these two as separate models? Is one like sort of the together proprietary one and then the other is like the more open research one?Ce [00:48:53]: Yeah. So I think it's pretty much a different stage of exploration. So they kind of have different hypothesis when we try to build those. Yeah. Like for instance, there are different view about state-based model. One is Hyena, another is like Mamba, right? They're actually different architecture. So when we build Stripe Hyena, right? So the curiosity that we have is how good can we... So what is the highest quality non-transformer model we can ever build? The goal of Stripe Hyena is try to see whether we can match Mistral. And by fine-tuning well, whether we can outperform that in some way, right? So it has a very, very strong baseline that we are trying to beat. So that's why there's hybrid scene, like getting the picture, right? And for Mamba, it's kind of more... The curiosity was how far can we push for pure architecture? Then we start from this very system make from small to large, right? All the way to 3 billion, right? So the baseline was essentially the best 3 billion model. So I guess at a different stage of exploration, at some point, I think they are going to converge. We actually learn different things, like when building different models. I think they are just like this intermediate stage in the exploration at different points.Alessio [00:50:02]: You mentioned the hybrid architecture. Is that the model grafting that you mentioned in the Stripe Hyena post where I mentioned you can have transformers and not together? Like this is a concept that I hadn't heard before reading about this. So I think most people's mental models, like transformers or something else, it's not transformers AND something else. How do you train a model that is hybrid? Is there any difference in like how you construct your datasets? Is there any difference in then how you run inference on it? How should people think about starting research in this field?Ce [00:50:36]: Yeah, so we were also very surprised. Yeah, so when we come up with this hybrid architecture. So the way to think about it is like you have different layers in the neural network, right? So like the stateless model for some layer will already give you the benefit. For the other layer, they could be transformers, right? They could give you this more global view of the sequence, but for me, for other layer, don't have to have that, right? I still can have all the other things that kick in, right? So we don't know what is the optimal mixture between different architectures. I mean, in principle, we can have a mamba, hyena, and transformer, all those things that come together, right? And then you can see what makes sense. We have no idea what is optimal doing that. So what we are excited about is now the community have a whole bunch of building blocks that they can actually like playing like a Lego, right? So just put together and see what happen, right? So we are kind of very excited about that. Yeah, we are in the process of trying to learn more like about this architecture. And when we know what we are talking about, we will definitely share with the community about how to do that in a systematic way.Swyx [00:51:41]: Cool. What are we still unsure about? Like, why don't we just, you know, put all the money in the world and training these things now? Like what is left to figure out before we scale this thing?Ce [00:51:53]: So like if you look at how transformer like it's been developed, right? In the last like five to 10 years, right? So people don't start from like, you have this attention to all you need the paper and then let's put all the money in, right? Always start from this very systematic understanding about the scaling, about data quality, about essentially the limits, right? I think for a state-based model from the labs to the real world, you kind of need to go through the same process. But of course, the second time doing that is kind of easier, right? But I think there's no way we can get rid of this systematic step of studying scaling law, study what data to put in, right? So what's the impact of different data slices to the data, yeah, to the final model quality.Swyx [00:52:33]: Do you expect that the data inputs will be different?Ce [00:52:37]: I don't know, but I wouldn't take that for granted that they should be the same, right? So that's one of the hypothesis that, so we have no opinion on that because I think that's the result of the study, not the assumption. Yeah, we do not need to assume that.Swyx [00:52:51]: Okay, scaling laws and data, anything else like architectural that we are not sure about? Because now you have this selection mechanism that you're pretty happy with.Ce [00:52:59]: Yeah, so, I mean, first of all, how to mix them, right? So, and second is what is the architecture? So if you look at transformer, right? So one very interesting piece there is people optimize also the hardware, yeah, to make sure that things run very fast, right?They're very efficient kernel, they're very efficient hardware. And then that's add another boost, right, for the transformer architecture, right? So that's something that should happen for state-based model. Which architecture is kind of easier kind of to run on the hardware, right? So, hosting going kind of faster, you can put more data, it add another dimension in the scaling law. So I think we just need to plow the whole space and just be really systematic from small model to 1 billion, 3 billion, 7 billion, just go all the way up, right? So I wouldn't jump around in the space. I would just like be patient and just like be systematic. Yeah, I think we'll get there, yeah.Swyx [00:53:52]: Yeah, well, I'm looking forward for more research from you guys to figure that out. So one dimension, which we didn't talk about, we talked about long context, we talked about efficiency, but speed is very, speed is also very important. A good inference provider provides, let's say 70 tokens per second, and then maybe that's faster than less good inference providers that are more like 30 tokens per second. But that's the rough range, right? State-of-the-art today. That's around the human speaking speed, human reading speed is about 200 words per minute. Why do we need 5,000 tokens per second is my question back to Vipul. And maybe is this something that is an emphasis for research as well, or is this more just an inference only thing?Vipul [00:54:29]: There are applications that are consuming the tokens that are produced from unmodeled, so they're not necessarily being read or heard by humans. That's a place where we see that level of requirement today that really nobody can quite satisfy. There is, can I think about, as intelligence grows, how do you sort of increase the bandwidth of, you know, how do you reduce the latency of it? If we can do 5,000 tokens a second, the same card can produce, the throughput of that card goes up significantly and can support more applications. So I think it's important from that perspective. And then there are, it opens up new UX possibilities. Once you can get sort of an immediate answer

00:00:00 Intro - LCS and TGI updates 00:15:55 treethan's take: we should do team tryouts in NA like they do in KR 00:30:40 play's take: team owners should be livid about Riot's broadcast shortcomings 00:57:09 franz's take: if TL doesn't start picking up wins, they should drop CoreJJ not APA 01:13:12 brandywine's take: the PROS show is the best thing to happen for the scene in years 01:30:10 skizzle's take: SR's failure to keep Chime and pick up a good top laner is the cause of their poor performance and low placement 01:47:55 aemulator's take: C9 are infodoomed like no other team in the West 01:58:14 Outro

With our second to last 2023 episode of TGI, Ryan squeezes in some virtual reality zombie slaying with Arizona Sunshine 2, and Jocelyn eagle dives into Assassin's Creed Mirage. Then before tackling The Game Awards winners and announcements, we discuss the announcement that E3 is officially no longer happening.

00:00:00 Intro 00:16:33 outsane asks Mark why he's the man for the job 00:31:31 fearfactor's take: better comms are necessary for the LCS to improve 00:39:12 farmerginge asks Mark what viewers should keep in mind while they wait for implementation 00:50:22 Avajou asks Mark for his favorite HLL moments 01:03:05 Andrew asks if TGI can be unbiased? 01:11:59 DoubleG's take: MarkZ IS the man for the job 01:19:45 Cody's take: the LCS might struggle to fill Mark's shoes on the broadcast at first 01:29:50 praeco asks if there are differences between esports and tsports that can be leaned into 01:45:45 big angry hobo wonders if it's too late for 2024 LCS changes 01:52:30 CaptFlowers calls in to congratulate Mark 01:59:00 Azael and Emily join HLL 02:10:20 rudy asks Mark what he's excited to see evolve about the product moving forward 02:20:07 Outro