Podcasts about Winograd

In this episode of NIC Chats, host Lisa McCracken sits down with Eric Winograd, Chief Economist at AllianceBernstein, to unpack the economic turbulence shaking markets, businesses, and households across the U.S. and beyond. From dramatic policy shifts and new tariffs to the ripple effects on inflation, growth, and consumer confidence, Winograd offers a candid, data-driven look at what's driving today's volatility, and what it could mean for senior housing.Highlights include:Why “uncertainty” is the economic watchword of the momentThe real impact of tariffs on prices, growth, and investment decisionsHow labor shortages and demographics are shaping construction and caregivingWhat to expect from the Fed and fiscal policy in the months aheadThe outlook for international investment and U.S. market stabilityTune in for expert insights on navigating a bumpy economic ride and why, despite the challenges, the U.S. may be better positioned than many think.Want to join the conversation? Follow NIC on LinkedIn.We want to hear from you! Let us know what you think of NIC Chats by giving us a review on Apple Podcasts, Spotify, or wherever you listen.

FFWF returns once again, and AJ welcomes first time pod guest Adam Winograd! The two break down each of the 10 movies nominated for Best Picture at the 2025 Academy Awards, give their picks for who they think will win, and sprinkle in discussions about some of the other 2025 Oscar races.

Happy holidays! We'll be sharing snippets from Latent Space LIVE! through the break bringing you the best of 2024! We want to express our deepest appreciation to event sponsors AWS, Daylight Computer, Thoth.ai, StrongCompute, Notable Capital, and most of all all our LS supporters who helped fund the gorgeous venue and A/V production!For NeurIPS last year we did our standard conference podcast coverage interviewing selected papers (that we have now also done for ICLR and ICML), however we felt that we could be doing more to help AI Engineers 1) get more industry-relevant content, and 2) recap 2024 year in review from experts. As a result, we organized the first Latent Space LIVE!, our first in person miniconference, at NeurIPS 2024 in Vancouver.Of perennial interest, particularly at academic conferences, is scaled-up architecture research as people hunt for the next Attention Is All You Need. We have many names for them: “efficient models”, “retentive networks”, “subquadratic attention” or “linear attention” but some of them don't even have any lineage with attention - one of the best papers of this NeurIPS was Sepp Hochreiter's xLSTM, which has a particularly poetic significance as one of the creators of the LSTM returning to update and challenge the OG language model architecture:So, for lack of a better term, we decided to call this segment “the State of Post-Transformers” and fortunately everyone rolled with it.We are fortunate to have two powerful friends of the pod to give us an update here:* Together AI: with CEO Vipul Ved Prakash and CTO Ce Zhang joining us to talk about how they are building Together together as a quote unquote full stack AI startup, from the lowest level kernel and systems programming to the highest level mathematical abstractions driving new model architectures and inference algorithms, with notable industry contributions from RedPajama v2, Flash Attention 3, Mamba 2, Mixture of Agents, BASED, Sequoia, Evo, Dragonfly, Dan Fu's ThunderKittens and many more research projects this year* Recursal AI: with CEO Eugene Cheah who has helped lead the independent RWKV project while also running Featherless AI. This year, the team has shipped RWKV v5, codenamed Eagle, to 1.5 billion Windows 10 and Windows 11 machines worldwide, to support Microsoft's on-device, energy-usage-sensitive Windows Copilot usecases, and has launched the first updates on RWKV v6, codenamed Finch and GoldFinch. On the morning of Latent Space Live, they also announced QRWKV6, a Qwen 32B model modified with RWKV linear attention layers. We were looking to host a debate between our speakers, but given that both of them were working on post-transformers alternativesFull Talk on YoutubePlease like and subscribe!LinksAll the models and papers they picked:* Earlier Cited Work* Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention* Hungry hungry hippos: Towards language modeling with state space models* Hyena hierarchy: Towards larger convolutional language models* Mamba: Linear-Time Sequence Modeling with Selective State Spaces* S4: Efficiently Modeling Long Sequences with Structured State Spaces* Just Read Twice (Arora et al)* Recurrent large language models that compete with Transformers in language modeling perplexity are emerging at a rapid rate (e.g., Mamba, RWKV). Excitingly, these architectures use a constant amount of memory during inference. However, due to the limited memory, recurrent LMs cannot recall and use all the information in long contexts leading to brittle in-context learning (ICL) quality. A key challenge for efficient LMs is selecting what information to store versus discard. In this work, we observe the order in which information is shown to the LM impacts the selection difficulty. * To formalize this, we show that the hardness of information recall reduces to the hardness of a problem called set disjointness (SD), a quintessential problem in communication complexity that requires a streaming algorithm (e.g., recurrent model) to decide whether inputted sets are disjoint. We empirically and theoretically show that the recurrent memory required to solve SD changes with set order, i.e., whether the smaller set appears first in-context. * Our analysis suggests, to mitigate the reliance on data order, we can put information in the right order in-context or process prompts non-causally. Towards that end, we propose: (1) JRT-Prompt, where context gets repeated multiple times in the prompt, effectively showing the model all data orders. This gives 11.0±1.3 points of improvement, averaged across 16 recurrent LMs and the 6 ICL tasks, with 11.9× higher throughput than FlashAttention-2 for generation prefill (length 32k, batch size 16, NVidia H100). We then propose (2) JRT-RNN, which uses non-causal prefix-linear-attention to process prompts and provides 99% of Transformer quality at 360M params., 30B tokens and 96% at 1.3B params., 50B tokens on average across the tasks, with 19.2× higher throughput for prefill than FA2.* Jamba: A 52B Hybrid Transformer-Mamba Language Model* We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. * Specifically, Jamba interleaves blocks of Transformer and Mamba layers, enjoying the benefits of both model families. MoE is added in some of these layers to increase model capacity while keeping active parameter usage manageable. * This flexible architecture allows resource- and objective-specific configurations. In the particular configuration we have implemented, we end up with a powerful model that fits in a single 80GB GPU.* Built at large scale, Jamba provides high throughput and small memory footprint compared to vanilla Transformers, and at the same time state-of-the-art performance on standard language model benchmarks and long-context evaluations. Remarkably, the model presents strong results for up to 256K tokens context length. * We study various architectural decisions, such as how to combine Transformer and Mamba layers, and how to mix experts, and show that some of them are crucial in large scale modeling. We also describe several interesting properties of these architectures which the training and evaluation of Jamba have revealed, and plan to release checkpoints from various ablation runs, to encourage further exploration of this novel architecture. We make the weights of our implementation of Jamba publicly available under a permissive license.* SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers* We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096×4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU. Core designs include: * (1) Deep compression autoencoder: unlike traditional AEs, which compress images only 8×, we trained an AE that can compress images 32×, effectively reducing the number of latent tokens. * (2) Linear DiT: we replace all vanilla attention in DiT with linear attention, which is more efficient at high resolutions without sacrificing quality. * (3) Decoder-only text encoder: we replaced T5 with modern decoder-only small LLM as the text encoder and designed complex human instruction with in-context learning to enhance the image-text alignment. * (4) Efficient training and sampling: we propose Flow-DPM-Solver to reduce sampling steps, with efficient caption labeling and selection to accelerate convergence. * As a result, Sana-0.6B is very competitive with modern giant diffusion model (e.g. Flux-12B), being 20 times smaller and 100+ times faster in measured throughput. Moreover, Sana-0.6B can be deployed on a 16GB laptop GPU, taking less than 1 second to generate a 1024×1024 resolution image. Sana enables content creation at low cost. * RWKV: Reinventing RNNs for the Transformer Era* Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence length. In contrast, recurrent neural networks (RNNs) exhibit linear scaling in memory and computational requirements but struggle to match the same performance as Transformers due to limitations in parallelization and scalability. * We propose a novel model architecture, Receptance Weighted Key Value (RWKV), that combines the efficient parallelizable training of transformers with the efficient inference of RNNs.* Our approach leverages a linear attention mechanism and allows us to formulate the model as either a Transformer or an RNN, thus parallelizing computations during training and maintains constant computational and memory complexity during inference. * We scale our models as large as 14 billion parameters, by far the largest dense RNN ever trained, and find RWKV performs on par with similarly sized Transformers, suggesting future work can leverage this architecture to create more efficient models. This work presents a significant step towards reconciling trade-offs between computational efficiency and model performance in sequence processing tasks.* LoLCATs: On Low-Rank Linearizing of Large Language Models* Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear attention -- avoiding the expensive pretraining costs. However, linearizing LLMs often significantly degrades model quality, still requires training over billions of tokens, and remains limited to smaller 1.3B to 7B LLMs. * We thus propose Low-rank Linear Conversion via Attention Transfer (LoLCATs), a simple two-step method that improves LLM linearizing quality with orders of magnitudes less memory and compute. * We base these steps on two findings. * First, we can replace an LLM's softmax attentions with closely-approximating linear attentions, simply by training the linear attentions to match their softmax counterparts with an output MSE loss ("attention transfer").* Then, this enables adjusting for approximation errors and recovering LLM quality simply with low-rank adaptation (LoRA). * LoLCATs significantly improves linearizing quality, training efficiency, and scalability. We significantly reduce the linearizing quality gap and produce state-of-the-art subquadratic LLMs from Llama 3 8B and Mistral 7B v0.1, leading to 20+ points of improvement on 5-shot MMLU. * Furthermore, LoLCATs does so with only 0.2% of past methods' model parameters and 0.4% of their training tokens. * Finally, we apply LoLCATs to create the first linearized 70B and 405B LLMs (50x larger than prior work). * When compared with prior approaches under the same compute budgets, LoLCATs significantly improves linearizing quality, closing the gap between linearized and original Llama 3.1 70B and 405B LLMs by 77.8% and 78.1% on 5-shot MMLU.Timestamps* [00:02:27] Intros* [00:03:16] Why Scale Context Lengths? or work on Efficient Models* [00:06:07] The Story of SSMs* [00:09:33] Idea 1: Approximation -> Principled Modeling* [00:12:14] Idea 3: Selection* [00:15:07] Just Read Twice* [00:16:51] Idea 4: Test Time Compute* [00:17:32] Idea 2: Hardware & Kernel Support* [00:19:49] RWKV vs SSMs* [00:24:24] RWKV Arch* [00:26:15] QWRKWv6 launch* [00:30:00] What's next* [00:33:21] Hot Takes - does anyone really need long context?Transcript[00:00:00] AI Charlie: We're back at Latent Space Live, our first mini conference held at NeurIPS 2024 in Vancouver. This is Charlie, your AI co host. As a special treat this week, we're recapping the best of 2024 going domain by domain. We sent out a survey to the over 900 of you who told us what you wanted, and then invited the best speakers in the Latent Space Network to cover each field.[00:00:24] AI Charlie: 200 of you joined us in person throughout the day, with over 2200 watching live online. Thanks Our next keynote covers the State of Transformers alternative architectures, with a special joint presentation with Dan Fu of Together AI and Eugene Chia of Recursal AI and Featherless AI. We've featured both Together and Recursal on the pod before, with CEO Veepal Vedprakash introducing them.[00:00:49] AI Charlie: And CTO CE Zhang joining us to talk about how they are building together together as a quote unquote full stack AI startup from the lowest level kernel and systems [00:01:00] programming to the highest level mathematical abstractions driving new model architectures and inference algorithms with notable industry contributions from Red Pajama V2, Flash Attention 3, Mamba 2, Mixture of Agents.[00:01:15] AI Charlie: Based, Sequoia, Evo, Dragonfly, Danfoo's Thunder Kittens, and many more research projects this year. As for Recursal and Featherless, we were the first podcast to feature RWKV last year, and this year the team has shipped RWKV v5, codenamed Eagle, to 1. 5 billion Windows 10 and Windows 11 machines worldwide to support Microsoft's on device, end Energy Usage Sensitive Windows Copilot Use Cases and has launched the first updates on RWKV v6, codenamed Finch and Goldfinch.[00:01:53] AI Charlie: On the morning of Latent Space Live, they also announced QRdata UKv6, a QEN32B model [00:02:00] modified with RDWKV linear attention layers. Eugene has also written the most single most popular guest post on the Latent Space blog this year. Yes, we do take guest posts on what he has discovered about the H100 GPU inference NeoCloud market since the successful launch of Featherless AI this year.[00:02:20] AI Charlie: As always, don't forget to check the show notes for the YouTube link to their talk as well as their slides. Watch out and take care.[00:02:27] Intros[00:02:27] Dan Fu: Yeah, so thanks so much for having us. So this is going to be a little bit of a two part presentation. My name is Dan. I'm at Together AI, and I'll be joining UCSD as faculty in about a year. And Eugene, you want to introduce yourself?[00:02:46] Eugene Cheah: Eugene, I lead the art activity team, and I, I'm CEO of Featherless, and we both work on this new post transformer architecture space.[00:02:55] Dan Fu: Yeah, so yeah, so today we're really excited to talk to you a little bit [00:03:00] about that. So first I'm going to give a broad overview of kind of the last few years of progress in non post transformer architectures. And then afterwards Eugene will tell us a little bit about the latest and the greatest and the latest frontier models in this space.[00:03:16] Why Scale Context Lengths? or work on Efficient Models[00:03:16] Dan Fu: So, the story starts with Scaling. So this is probably a figure or something like this that you've seen very recently. Over the last five to six years, we've seen models really scale up in parameter size, and that's brought with it a bunch of new capabilities, like the ability to talk to you and tell you sometimes how to use your Colab screens.[00:03:35] Dan Fu: But another place where we've seen scaling especially recently is scaling in context length. So this can mean Having more text inputs for your models, but it can also mean things like taking a lot of visual token inputs image inputs to your models or generating lots of outputs. And one thing that's been really exciting over the last few months or so is that we're, we're seeing scaling, not only during training time, but also [00:04:00] during test time.[00:04:00] Dan Fu: So this is one of the, the, this is the iconic image from the OpenAI 01 release. Not only are we starting to scale train time compute, but we're also starting to scale test time compute. Now if you're familiar with our attention and our transformer architectures today, this graph on the right might look a little bit scary.[00:04:19] Dan Fu: And one of the reasons is that the implications are a little bit Interesting. So what does it mean if we want to continue having smarter and smarter models? Do we just need to start building bigger, bigger data centers, spending more flops? Is this this little Dolly 3, we need more flops, guys? Is this going to be the future of all of AI?[00:04:39] Dan Fu: Or is there a better way, another path forward? Maybe we can get the same capabilities that we've gotten used to, But for a lot less compute, a lot less flops. And one of the things that we're going to talk about today is specifically looking at that core attention operator in some of these models.[00:04:57] Dan Fu: And the reason is that so this is just some, some [00:05:00] basic you know, scaling curves, but attention has compute that scales quadratically in the context length. So that means that if you're doing something like test time compute and you want to spend a bunch of tokens thinking about what comes next, the longer that that goes the, the, the more tokens you spend on that, that compute grows quadratically in that.[00:05:19] Dan Fu: One of the questions that we're interested in is, can we take that basic sequence model, that basic sequence primitive at the bottom, and get it to scale better? Can we scale in, let's say, n to the 3 halves or n log n? So in, in the first part of the talk, so we just went over the introduction. What I'm gonna do over the next few slides is just talk about some of the key advances and ideas that have shown over the past few years since maybe early 2020 to, to now that shown promise that this might actually be possible.[00:05:48] Dan Fu: That you can actually get potentially the same quality that we want while scale, while scaling better. So to do that, we're and, and basically the, the story that we're gonna look is we're gonna start to see [00:06:00] how. So this is a basic graph of just the past couple years of progress of perplexity where that blue line, that dotted blue line, is attention.[00:06:07] The Story of SSMs[00:06:07] Dan Fu: It's your basic transformer, full dense attention. And then the dots coming down are some of the methods that you'll see in this presentation today. We're going to turn the clock back all the way to 2020. So this, this, this question of can we make attention subquadratic? Basically, as soon as we said attention is all you need, People started asking this question.[00:06:28] Dan Fu: So we have this quadratic attention operator. Can we do better? I'll briefly talk about why attention is quadratic. And the basic thing that happens, if you're not familiar, is that you have these inputs, these keys and queries. And what you do in this attention matrix, this S matrix over here, is that you're using, you're comparing every token in your input to every other token.[00:06:49] Dan Fu: So when I try to do something like upload a whole book to Gemini, what happens beyond the Maybe not Gemini, because we don't necessarily know what architecture is. But let's say we upload it to LLAMA, what happens beyond [00:07:00] the scenes, behind the scenes, is that it's going to take every single word in that book and compare it to every other word.[00:07:05] Dan Fu: And this has been a really, it's, it's led to some pretty impressive things. But it's kind of a brute forcing of the way that you would try to interpret a interpret something. And what attention does in particular is the, and then what attention, sorry, don't want to. Okay, no, no laser pointer. What, what attention does afterwards is that instead of always operating in this quadratic thing, it takes a row wise softmax over this matrix, and then multiplies it by this values matrix.[00:07:32] Dan Fu: So, one of the key points to notice is that the output size is always going to be the same as the inputs, at least in standard self attention. So one of the first things that folks tried to do around 2020 is this thing called linear attention, which is just, just noticing that if we take out this softmax from here, if we take out this non linearity in the middle of the attention operation, and then if you compute the keys and the values operation first, you actually never hit this quadratic bottleneck.[00:07:57] Dan Fu: So that, that's potentially a way [00:08:00] to get a lot more computationally efficient. And there are various ways to do this by basically using feature maps or try to approximate this overall attention computation. But some of this work sort of started to hit a wall in 2020. And the basic challenges were, were two.[00:08:16] Dan Fu: So one was quality. It was back then, it was kind of hard to, to get good quality with these linear attention operators. The other one was actually hardware efficiency. So these, this feature map that was just shown by a simplify simplify here. Actually ends up being quite computationally expensive if you just implement it naively.[00:08:34] Dan Fu: So you started having these operators that not only were you sure, you're not really sure if they have the same quality, but also they're actually just wall clock slower. So you kind of end up getting the worst of both worlds. So this was the the stage. So that kind of sets the stage for four years ago.[00:08:49] Dan Fu: Keep this in mind because linear attention is actually going to come back in a few years once we have a better understanding. But one of the works that started kicking off this, this [00:09:00] mini revolution in post transformer architectures was this idea called states based model. So here the seminal work is, is one about our work queue in 2022.[00:09:09] Dan Fu: And this, this piece of work really brought together a few ideas from, from some long running research research lines of work. The first one was, and this is really one of the keys to, to closing the gap in quality was just using things that, that if you talk to a, a, an electrical engineer off the street, they might know off, off the, like the back of their hand.[00:09:33] Idea 1: Approximation -> Principled Modeling[00:09:33] Dan Fu: But taking some of those properties with how we model dynamical systems in signal processing and then using those ideas to model the inputs, the, the text tokens in, for example a transformer like Next Token Prediction Architecture. So some of those early states-based model papers were looking at this relatively, relatively simple recurrent update model that comes from maybe chapter one of a signal processing class.[00:09:59] Dan Fu: But then using [00:10:00] some principle theory about how you should do that recurrent update in order to really get the most that you can out of your hidden state, out of your out of your sequence. So that, that was one key idea for quality and. When this was eventually realized, you started to see a bunch of benchmarks that were pretty sticky for a few years.[00:10:20] Dan Fu: Things like long range arena, some long sequence evaluation benchmarks, There was stuff in time series, time series analysis. They started to, you started to see the quality tick up in meaningful ways. But the other key thing that What's so influential about these states based models is that they also had a key idea about how you can compute these things efficiently.[00:10:45] Dan Fu: So if you go back to your machine learning 101 class where you learned about RNNs, one thing that you may have learned is that they don't paralyze as well as detention, because if you just run them naively, you have to do this kind of sequential update to process new tokens, [00:11:00] whereas in attention, you can process all the tokens in parallel at one time.[00:11:04] Dan Fu: One of the key insights behind the S4 paper was that these recurrent models, you could take them and you could also formulate them as a convolution. And in particular, with a convolution, you could, instead of using a PyTorch conv1d operation, you can compute that with the FFT. And that would give you n log n compute in the in the sequence length n with an operator that was relatively well optimized for modern hardware.[00:11:28] Dan Fu: So those are really, I'd say, the two key ideas in 2022 that started allowing these breakthroughs to happen in these non transformer architectures. So, these ideas about how to principally model sorry, how to model the recurrent updates of a mo of, of a sequence in a principled way, and also these key ideas in how you can compute it efficiently by turning it into a convolution and then scaling it up with the FFT.[00:11:53] Dan Fu: Along those same lines, so afterwards we started putting out some work on specialized kernels, so just [00:12:00] like we have flash attention for transformers, we also have works like flash fft conf, and if you look at these lines of work oftentimes when, whenever you see a new architecture, you see a new primitive one of the, one of the table stakes now is, do you have an efficient kernel so that you can actually get wall clock speed up?[00:12:14] Idea 3: Selection[00:12:14] Dan Fu: So by 2022, We are starting to have these models that had promising quality primitives, but and, and also promising wall clocks. So you could actually see regimes where they were better than transformers in meaningful ways. That being said, there were, there's still sometimes a quality gap, particularly for language modeling.[00:12:33] Dan Fu: And because languages, It's so core to what we do in sequence modeling these days the, the next, the next key idea that I'm going to talk about is this idea of selection mechanisms. And this is basically an idea of, so you have this recurrent state that you're keeping around that just summarizes everything that, that came before.[00:12:50] Dan Fu: And to get a good sequence model, one of the things that you really need to be able to do is have the model learn what's the best way to pick out pieces from that recurrent [00:13:00] state. So one of the, one of the major ideas here in a line of work called H3, Hungry Hungry Hippos, and also these hyena models were One way you can do this is by just adding some simple element wise gates.[00:13:13] Dan Fu: So versions of these ideas have been around for decades. If you squint at the LSTM paper you, you can probably find, find this gating mechanism. But turns out you can take those old ideas, add them into these new. state space models, and then you can see quality start to pick up. If you've heard of the Mamba model, this also takes the selection to the next level by actually making some changes in that fundamental recurrent state space.[00:13:40] Dan Fu: So, it's not only just this gating that happens around the SSM layer, but also you can actually make The ABCD matrices of your state space model, you can make them data dependent, which will allow you to even better select out different pieces from your hidden state depending on what you're seeing. I'll also point out if you look at the [00:14:00] bottom right of this figure, there's this little triangle with a GPU SRAM, GPU HBM, and this, this is just continuing that trend of when you have a new architecture you, you, you also release it with a kernel to, to, to show that it is hardware efficient, that it, that it can be hardware efficient on modern hardware.[00:14:17] Dan Fu: The, the, one of the next cool things that happened is once we had this understanding of these are the basic pieces, these are the basic principles behind some of the sequence models linear attention actually started to come back. So in earlier this year, there was a model called BASED the, from Simran Arora and, and some other folks, that combined a more principled version of linear attention that basically the, the, the, the two second summary is that it used a Taylor approximation of the softmax attention, combined that with a simple sliding window attention and was starting to able, starting to be able to expand the Pareto frontier of how much data can you recall from your sequence, versus how small is your recurrent state size.[00:14:58] Dan Fu: So those orange dots [00:15:00] are, at the top there, are just showing smaller sequences that can recall more memory.[00:15:07] Just Read Twice[00:15:07] Dan Fu: And the last major idea I think that has been influential in this line of work and is very relatively late breaking just a few months ago, is just the basic idea that when you have these models that are fundamentally more efficient in the sequence length, you maybe don't want to prompt them or use them in exactly the same way.[00:15:26] Dan Fu: So this was a really cool paper called Just Read Twice, also from Simran. That basically said, hey, all these efficient models can process tokens so much more efficiently than transformers that they can sometimes have unfair advantages compared to a simple transformer token. So, or sorry, a simple transformer model.[00:15:44] Dan Fu: So take, for example the standard, the standard use case of you have some long document, you're going to pass it in as input, and then you're going to ask some question about it. One problem you might imagine for a recurrent model where you have a fixed state size is, let's say that [00:16:00] you're. Article is very long, and you're trying to ask about some really niche thing.[00:16:04] Dan Fu: You can imagine it might be hard for the model to know ahead of time what information to put into the hidden state. But these, these, these models are so much more efficient that you can do something really stupid, like, you can just put the document write down the document, write down the question, write down the document again, and then write down the question again, and then this time, the second time that you go over that document, you know exactly what to look for.[00:16:25] Dan Fu: And the cool thing about this is, so this is, And this this results in better quality, especially on these recall intensive tasks. But the other interesting thing is it really takes advantage of the more efficient architectures that, that we're having here. So one of the other, I think, influential ideas in this line of work is if you change the fundamental compute capabilities of your model and the way that it scales, you can actually start to query it at test time differently.[00:16:51] Idea 4: Test Time Compute[00:16:51] Dan Fu: And this actually, of course, goes back to those slides on test time compute. So while everybody's looking at, say, test time compute for big transformer models, [00:17:00] I think potentially a really interesting research question is, how can you take those and how does it change with this new next generation of models?[00:17:09] Dan Fu: So the, I'll just briefly summarize what some of those key ideas were and then talk and then show you briefly kind of what the state of the art is today. So, so the four key ideas are instead of just doing a simple linear attention approximation, instead take ideas that we know from other fields like signal processing, do a more principled approach to your modeling of the sequence.[00:17:32] Idea 2: Hardware & Kernel Support[00:17:32] Dan Fu: Another key idea throughout all these lines of work is you really want. Hardware and kernel support from day one. So, so even if your model is theoretically more efficient if somebody goes and runs it and it's two times slower one of the things that, that we've learned is that if, if you're in that situation, it's, it's just gonna be dead on arrival.[00:17:49] Dan Fu: So you want to be designing your architectures one of the key, key machine learning ideas that has been important for the quality is just making sure that you encode different ways that you can [00:18:00] select from your hidden state and, and really focus on that as a key decider of quality. And finally, I think one of the, the, the emerging new, new things for, for this line of work and something that's quite interesting is, What are the right test time paradigms for these models?[00:18:15] Dan Fu: How do they change relative to relative to what you might do for a standard transformer? I'll briefly end this section. So I've labeled this slide where we are yesterday because Eugene is going to talk about some new models that he released literally this morning. But as of yesterday, some of the really cool results out of the, these efficient alternative models were so AI2 trained this hybrid MOE called Jamba.[00:18:40] Dan Fu: That, that, that seems, that is currently the state of the art for these non transformer architectures. There's this NVIDIA and MIT put out this new diffusion model called SANA recently that one of their key key observations is that you can take a standard diffusion transformer diffusion model, replace the layers with linear [00:19:00] attention, and then that lets you scale to much larger much larger images, much, much Much larger sequences more efficiently.[00:19:07] Dan Fu: And and one thing that I don't think anybody would have called when a few years ago is that one of those gated SSM, gated states based models ended up on the cover of Science because a great group of folks went and trained some DNA models. So that's Michael Polley, Eric Yuen from from Stanford and the Arc Institute.[00:19:26] Dan Fu: So it's, we're really at an exciting time in 2024 where these non transformer, post transformer architectures are showing promise across a wide range. Across a wide range of, of modalities, of applications, and, and of tasks. And with that, I'll pass it on to Eugene, who can tell you a little bit about the latest and greatest with RWKV.[00:19:49] RWKV vs SSMs[00:19:49] Eugene Cheah: So, that's useful? Yeah. You're talking to here. Oh, I'm talking to here. Okay. So, yeah, two streams. Yeah. So, I think one common questions that we tend to get asked, right, is what's the difference between [00:20:00] RWKV and state space? So I think one of the key things to really understand, right the difference between the two groups, right, is that we are actually more like an open source, random internet meets academia kind of situation.[00:20:11] Eugene Cheah: Like, most of us never wrote any paper, but we, we basically look at RNNs and linear intention when intention is all you need came out, and then we decided to like, hey there is a quadratic scaling problem. Why don't we try fixing that instead? So, so, so we end up developing our own branch, but we end up sharing ideas back and forth.[00:20:30] Eugene Cheah: So, and, and we do all this actively in Discord, GitHub, etc. This was so bad for a few years, right, that basically, the average group's H index was so close to zero, right, Illuter. ai actually came in and helped us write our first paper. Great, now our H index is now three, apparently. So, so, so, but, but the thing is, like, a lot of these experiments led to results, and, and, essentially, essentially, we we took the same ideas from linear attention, [00:21:00] and we built on it.[00:21:01] Eugene Cheah: So, to take a step back into, like, how does RWKB handle its own attention mechanic and achieve the same goals of, like, O and compute, respectively, and in focus of our overall goal to make AI accessible to everyone, regardless of language, nation, or compute, that's our goal. We actually train our models primarily on over a hundred languages, which is another topic altogether.[00:21:23] Eugene Cheah: And our goal is to train to even 200 languages to cover all languages in the world. But at the same time, we work on this architecture, To lower the compute cost so that people can run it on Raspberry Pis and on anything. So, how did RWKB break the dependency of LSTM token flow? Because I think to understand architecture, right, it's probably easier to understand it from the RNN lens.[00:21:46] Eugene Cheah: Because that's where we built on. We all, we all state space kind of like try to, try to start anew and took lessons from that and say, So there's a little bit of divergence there. And AKA, this our version of linear attention. So to take step back [00:22:00] all foundation models, be it transformers or non transformers at a very high level, right?[00:22:05] Eugene Cheah: Pumps in the token. I mean, text that things into embeddings and go through a lot of layers. Generate a lot of states where the QKV cache or be iron in states or RW KB states. And outputs and embedding, they are not the same thing. And we just take more layers and more embeddings. And somehow that magically works.[00:22:23] Eugene Cheah: So, if you, if you remember your ancient RNN lessons which we, which we, which we we call best learning these days the general idea is that you have the embedding information flowing all the way up, and when, and you take that information and you flow it back down, and then you process it as part of your LSTM layers.[00:22:41] Eugene Cheah: So, this is how it generally works. Kapati is quoted saying that RNNs are actually unreasonably effective. The problem is this is not scalable. To start doing work on the second token, you need to wait for the first token. And then you need to, and likewise for the third token and fourth token, yada yada.[00:22:55] Eugene Cheah: That is CPU land, not GPU land. So, so, so, you [00:23:00] can have a H100 and you can't even use 1 percent of it. So, so that's kind of why RNNs didn't really take off in the direction that we wanted, like, billions of parameters when it comes to training. So, what did RDAP KV version 0 do? Boom. We just did the dumbest, lamest thing.[00:23:13] Eugene Cheah: Sorry, this is the bottleneck for RNN. We did the dumb thing of removing that line. And it kind of worked. It trained. It sucked, but it kind of worked. Then we were like, hey, then no one cared because the loss was crap, but how do we improve that? And that's essentially where we move forward, because if you see this kind of flow, right, you can actually get your GPU saturated quickly, where it essentially cascades respectively.[00:23:41] Eugene Cheah: So I'm just waiting for this to loop again. So it's like, once you get your first layer, your token to be computed finish. You start to cascade your compute all the way until you are, Hey, I'm using 100 percent of the GPU. So we, we worked on it, and we started going along the principle of that as long as we keep this general architecture [00:24:00] where, where we can cascade and, and be highly efficient with our architecture, nothing is sacred in our architecture.[00:24:06] Eugene Cheah: And we have done some crazy ideas. In fact, you ask us, if you ask me to explain some things in the paper, right, officially in the paper, I'll say we had this idea and we wrote it this way. The reality is someone came with a code, we tested it, it worked, and then we rationalized later. So, so the general[00:24:24] RWKV Arch[00:24:24] Eugene Cheah: The idea behind rwkbr is that we generally have two major blocks that we do.[00:24:30] Eugene Cheah: We call time mix and channel mix. And time mix generally handles handles long term memory states, where essentially, where essentially where we apply the matrix multiplication and Cilu activation functions into processing an input embedding and an output embedding. I'm oversimplifying it because this, This calculation changed every version and we have, like, version 7 right now.[00:24:50] Eugene Cheah: ChannelMix is similar to Base in the sense that it does shorter term attention, where it just looks at the sister token, or the token before it, because [00:25:00] there's a shift in the token shift matrix. I don't really want to go too much into the papers itself, because, like, we do have three papers on this.[00:25:09] Eugene Cheah: Basically, RWKB, RNN for the transformer, ERA, Ego and Pinch, RWKB, Matrix Value State. This is the updated version 5, version 6. And Goldfinch is our, is, is, is, is our hybrid model respectively. We are writing the paper already for V seven and which is, which is for R wk V seven. Called, named Goose, or architectures are named by Bird.[00:25:30] Eugene Cheah: And, I'm going to cover as well, qrwkb, and mama100k, and rwkb, and Where did that lead to? Great! Because we are all GPU poor and to be clear, like, most of this research is done, like, only on a handful H100s, which I had one Google researcher told me that was, like, his experiment budget for a single researcher.[00:25:48] Eugene Cheah: So, our entire organization has less compute than a single researcher in Google. So We, we, one of the things that we explored into was to how do we convert transformer models instead? Because [00:26:00] someone already paid that billion dollars, a million dollars onto training, so why don't we take advantage of those weights?[00:26:05] Eugene Cheah: And, and to, I believe, together AI worked on the lockets for, for the Lambda side of things, and, and we took some ideas from there as well, and we essentially did that for RWKB.[00:26:15] QWRKWv6 launch[00:26:15] Eugene Cheah: And that led to, Q RWKB6, which we just dropped today, a 32 bit instruct preview model, where we took the Quen 32 bit instruct model, freeze the feedforward layer, remove the QKB attention layer, and replace it with RWKB linear layers.[00:26:32] Eugene Cheah: So to be clear, this means we do not have the rwkv channel mix layer, we only have the time mix layer. But but once we do that, we train the rwkv layer. Important is that the feedforward layer needs to be frozen, so the new attention can be learned. And then we unfreeze the feedforward layer, and train all the layers together with a custom learning rate schedule, so that they can learn how to work together.[00:26:54] Eugene Cheah: The end result, surprisingly, And, to be honest, to the frustration of the R. W. [00:27:00] KV MOE team, which ended up releasing the model on the same day, was that, with just a few hours of training on two nodes, we managed to get it to be on par, kind of, with the original QUAN32B model. So, in fact, when the first run, right, that completely confused us, it was like, and I was telling Daniel Goldstein, Smirky, who kind of leads most of our research coordination, When you pitched me this idea, you told me at best you'll get the same level of performance.[00:27:26] Eugene Cheah: You didn't tell me the challenge and score and Winograd score will shoot up. I don't know what's happening there. But it did. MMLU score dropping, that was expected. Because if you think about it, when we were training all the layers, right, we were essentially Like, Frankenstein this thing, and we did brain damage to the feedforward network layer 2 with the new RWKB layers.[00:27:47] Eugene Cheah: But, 76%, hey, somehow it's retained, and we can probably further train this. We didn't even spend more than 3 days training this, so there's a lot more that can be done, hence the preview. This brings up [00:28:00] a big question, because We are already now in the process of converting to 7TB. We are now, this is actually extremely compute efficient to test our attention mechanic.[00:28:10] Eugene Cheah: It's like, it becomes a shortcut. We can, we are already planning to do our version 7 and our hybrid architecture for it. Because we don't need to train from scratch. And we get a really good model out of it. And the other thing that is uncomfortable to say is that because we are doing right now on the 70b is that if this scales correctly to 128k context length, I'm not even talking about a million 128, majority of enterprise workload today is just on 70b at under 32k context length.[00:28:41] Eugene Cheah: That means if this works and the benchmark matches it, It means we can replace the vast majority of current AI workload, unless you want super long context. And then sorry, can someone give us more GPUs? Because we do need the VRAM for super long context, sadly. So yeah, that's what we are working on, and essentially, [00:29:00] we are excited about this to just push it further.[00:29:02] Eugene Cheah: And this conversion process, to be clear, I don't think it's going to be exclusive to RWKB. It probably will work for Mamba as well, I don't see why not. And we will probably see more ideas, or more experiments, or more hybrids, or Yeah, like, one of the weirdest things that I wanted to say outright, and I confirmed this with the Black Mamba team and the Jamba team, which because we did the GoFinch hybrid model, is that none of us understand why a hard hybrid with a state based model to be R.[00:29:28] Eugene Cheah: QA state space and transformer performs better when, than the baseline of both. It's like, it's like when you train one, you expect, and then you replace, you expect the same results. That's our pitch. That's our claim. But somehow when we jam both together, it outperforms both. And that's like one area of emulation that, like, we only have four experiments, plus four teams, that a lot more needs to be done.[00:29:51] Eugene Cheah: But, but these are things that excite me, essentially, because that is what it's potentially we can move ahead for. Which brings us to what comes next.[00:30:00] What's next[00:30:00] [00:30:00][00:30:00] Dan Fu: So, this part is kind of just some, where we'll talk a little bit about stuff that, that we're excited about. Maybe have some wild speculation on, on what, what's, what's coming next.[00:30:12] Dan Fu: And, of course this is also the part that will be more open to questions. So, a couple things that, that I'm excited about is continued hardware model co design for, for these models. So one of the things that we've put out recently is this library called ThunderKittens. It's a CUDA library.[00:30:29] Dan Fu: And one of the things that, that we found frustrating is every time that we built one of these new architectures, and I'm sure you had the exact same experience, we'd have to go and spend two months in CUDA land, like writing these, these new efficient things. And. If we decided to change one thing in PyTorch, like one line of PyTorch code is like a week of CUDA code at least.[00:30:47] Dan Fu: So one of our goals with, with a library like Thunderkitten, so we, we just broke down what are the key principles, what are the key hardware things what are the key, Compute pieces that you get from the hardware. So for example on [00:31:00] H100 everything is really revolves around a warp group matrix multiply operation.[00:31:06] Dan Fu: So you really want your operation to be able to split into relatively small matrix, matrix multiply operations. So like multiplying two 64 by 64 matrices, for example. And so if you know that ahead of time when you're designing your model, that probably gives you you know, some information about how you set the state sizes, how you set the update, how you set the update function.[00:31:27] Dan Fu: So with Thunderkittens we basically built a whole library just around this basic idea that all your basic compute primitives should not be a float, but it should be a matrix, and everything should just be matrix compute. And we've been using that to, to try to both re implement some existing architectures, and also start to design code.[00:31:44] Dan Fu: Some new ones that are really designed with this core with a tensor core primitive in mind. Another thing that that we're, that at least I'm excited about is we, over the last four or five years, we've really been looking at language models as the next thing. But if you've been paying [00:32:00] attention to Twitter there's been a bunch of new next generation models that are coming out.[00:32:04] Dan Fu: So there, there are. So, video generation models that can run real time, that are supported by your mouse and your keyboard, that I'm told if you play with them that, you know, that they only have a few seconds of memory. Can we take that model, can we give it a very long context length so that you could actually maybe generate an entire game state at a time?[00:32:25] Dan Fu: What does that look like for the model? You're certainly not going to do a giant quadratic attention computation to try to run that. Maybe, maybe use some of these new models, or some of these new video generation models that came out. So Sora came out I don't know, two days ago now. But with super long queue times and super long generation times.[00:32:43] Dan Fu: So that's probably a quadratic attention operation at the, at the bottom of it. What if we could remove that and get the same quality, but a lot faster generation time? Or some of the demos that we saw from Paige earlier today. You know, if I have a super long conversation with my [00:33:00] Gemini bot, what if I wanted to remember everything that it's seen in the last week?[00:33:06] Dan Fu: I mean, maybe you don't for personal reasons, but what if I did, you know? What does that mean for the architecture? And I think, you know, that's certainly something I'm pretty excited about. I'm sure you're excited about it too. So, I think we were supposed to have some hot takes, but I honestly don't remember what our hot takes were.[00:33:21] Hot Takes - does anyone really need long context?[00:33:21] Eugene Cheah: Yeah, including the next slide. Hot takes, yes, these are our[00:33:25] Dan Fu: hot takes.[00:33:25] Eugene Cheah: I think the big one on Twitter that we saw, that we shared, was the question is like, is RAG relevant? In the case of, like, the future of, like, state based models?[00:33:38] Dan Fu: Let's see, I haven't played too much with RAG. But when I have. I'll say I found it was a little bit challenging to do research on it because we had this experience over and over again, where you could have any, an embedding model of any quality, so you could have a really, really bad embedding model, or you could have a really, really [00:34:00] good one, By any measure of good.[00:34:03] Dan Fu: And for the final RAG application, it kind of didn't matter. That's what I'll say about RAG while I'm being recorded. I know it doesn't actually answer the question, but[00:34:13] Eugene Cheah: Yeah, so I think a lot of folks are like, extremely excited of the idea of RWKB or State Space potentially having infinite context.[00:34:21] Eugene Cheah: But I think the reality is that when we say infinite context, we just mean a different kind of infinite context, or you, or as it's previously covered, you need to test the model differently. So, think of it more along the lines of the human. Like, I don't remember what I ate for breakfast yesterday.[00:34:37] Eugene Cheah: Yeah, that's the statement that I'll say. And And we humans are not quadratic transformers. If we did, if let's say we increased our brain size for every second we live, we would have exploded by the time we are 5 years old or something like that. And, and I think, I think basically fundamentally for us, right, be it whether we, regardless of whether RWKB, statespace, XLSTM, [00:35:00] etc, our general idea is that instead of that expanding state, that increase in computational cost, what if we have a fixed state size?[00:35:08] Eugene Cheah: And Information theory detects that that fixed state size will have a limit. Just how big of a limit is a question, like, we, like, RWKB is running at 40 megabytes for, for its state. Its future version might run into 400 megabytes. That is like millions of tokens in, if you're talking about mathematically, the maximum possibility.[00:35:29] Eugene Cheah: It's just that I guess we were all more inefficient about it, so maybe we hit 100, 000. And that's kind of like the work we are doing, trying to like push it and maximize it. And that's where the models will start differing, because it will choose to forget things, it will choose to remember things. And that's why I think that there might be some element of right, but it may not be the same right.[00:35:49] Eugene Cheah: It may be the model learn things, and it's like, hmm, I can't remember that, that article. Let me do a database search, to search. Just like us humans, when we can't remember the article in the company. We do a search on Notion. [00:36:00][00:36:00] Dan Fu: I think something that would be really interesting is if you could have facts that are, so right now, the one intuition about language models is that all those parameters are around just to store random facts about the world.[00:36:14] Dan Fu: And this intuition comes from the observation that if you take a really small language model, it can do things like talk to you, or kind of has like the The style of conversation, it can learn that, but where it will usually fall over compared to a much larger one is it'll just be a lot less factual about things that it knows or that it can do.[00:36:32] Dan Fu: But that points to all those weights that we're spending, all that SGD that we're spending to train these models are just being used to store facts. And we have things like databases that are pretty good at storing facts. So I think one thing that would be really interesting is if we could actually have some sort of outside data store that a language model can can look at that that maybe is you know, has has some sort of gradient descent in it, but but would be quite interesting.[00:36:58] Dan Fu: And then maybe you could edit it, delete [00:37:00] facts, you know, change who's president so that it doesn't, it doesn't get lost.[00:37:04] Vibhu: Can we open up Q& A and hot takes for the audience? I have a hot take Q& A. Do these scale? When, when 405B state space model, RAG exists, no one does long context, who's throwing in 2 million token questions, hot takes?[00:37:24] Dan Fu: The, the who's throwing in 2 million token question, I think, is, is a really good question. So I actually, I was going to offer that as a hot take. I mean, my hot take was going to be that long context doesn't matter. I know I just gave a whole talk about it, but you know, what, what's the point of doing research if you can't, you know, play both sides.[00:37:40] Dan Fu: But I think one of the, so I think for both of us, the reason that we first got into this was just from the first principled questions of there's this quadratic thing. Clearly intelligence doesn't need to be quadratic. What is going on? Can we understand it better? You know, since then it's kind of turned into a race, which has [00:38:00] been exciting to watch, like, how much context you can take in.[00:38:03] Dan Fu: But I think it's right. Nobody is actually putting in a two million context prompt into these models. And, and, you know, if they are, maybe we can go, go You know, design a better model to do that particular thing. Yeah, what do you think about that? So you've also been working on this. Do you think long context matters?[00:38:19] Eugene Cheah: So I'm going to burn a bit. How many of you remember the news of Google Gemini supporting 3 million contacts, right? Raise your hand.[00:38:28] Vibhu: Yeah, 2 million.[00:38:29] Eugene Cheah: Oh, it's 2 million.[00:38:31] Eugene Cheah: Yeah, how many of you actually tried that? See?[00:38:34] Vibhu: I use it a lot. You? You work for MindsTV. I use it a lot.[00:38:41] Eugene Cheah: So, for some people that has used, and I think, I think that's the, that's might be, like, this is where my opinion starts to differ, because I think the big labs may have a bigger role in this, because Like, even for RWKB, even when we train non contacts, the reason why I say VRAM is a problem is that because when we did the, we need to backprop [00:39:00] against the states, we actually need to maintain the state in between the tokens by the token length.[00:39:05] Eugene Cheah: So that means we need to actually roll out the whole 1 million contacts if we are actually training 1 million. Which is the same for transformers, actually, but it just means we don't magically reuse the VRAM consumption in the training time space. So that is one of the VRAM bottlenecks, and I'm neither OpenAI nor Google, so donate GPUs if you have too much of them.[00:39:27] Eugene Cheah: But then, putting it back to another paradigm, right, is that I think O1 style reasoning might be actually pushing that direction downwards. In my opinion, this is my partial hot take is that if, let's say you have a super big model, And let's say you have a 70B model that may take double the tokens, but gets the same result.[00:39:51] Eugene Cheah: Strictly speaking, a 70B, and this is even for transformer or non transformer, right? We we'll take less less resources than that 400 B [00:40:00] model, even if it did double the amount thinking. And if that's the case, and we are still all trying to figure this out, maybe the direction for us is really getting the sub 200 B to be as fast as efficient as possible.[00:40:11] Eugene Cheah: We a very efficient architecture that some folks happen to be working on to, to just reason it out over larger and larger context thing.[00:40:20] Question: Yeah. One thing I'm super interested in is. Models that can watch forever? Obviously you cannot train something on infinite context length. How are y'all thinking about that, where you run on a much longer context length than is possible to train on?[00:40:38] Dan Fu: Yeah, it's a, it's a great question. So I think when I think you guys probably had tweets along these lines, too. When we first started doing these things, because these are all recurrent models in theory you could just run it forever. You could just run it forever. And at the very least it won't, it won't like error out on your crash.[00:40:57] Dan Fu: There's another question of whether it can actually [00:41:00] use what it's seen in that infinite context. And I think there, so one place where probably the research and architectures ran faster Then another research is actually the benchmarks for long context. So you turn it on forever. You want to do everything or watch everything.[00:41:16] Dan Fu: What is it that you actually wanted to do? Can we actually build some benchmarks for that? Then measure what's happening. And then ask the question, can the models do it? Is there something else that they need? Yeah, I think that if I were to turn back the clock to 2022, that's probably one of the things I would have done differently, which would have been actually get some long context benchmarks out at the same time as we started pushing context length on all these models.[00:41:41] Eugene Cheah: I will also say the use case. So like, I think we both agree that there's no Infinite memory and the model needs to be able to learn and decide. I think what we have observed for, I think this also fits the state space model, is that one of the key advantages of this alternate attention mechanic that is not based on token position is that the model don't suddenly become crazy when you go past the [00:42:00] 8k training context tank, or a million context tank.[00:42:03] Eugene Cheah: It's actually still stable. It's still able to run, it's still able to rationalize. It just starts forgetting things. But some of these things are still there in latent memory. Some of these things are still somewhat there. That's the whole point of why reading twice works. Things like that. And one of the biggest pushes in this direction is that I think both Statespace and RWKB have Separate papers by other researchers where they use this architecture for time series data.[00:42:26] Eugene Cheah: Weather modeling. So, you are not asking what was the weather five days ago. You're asking what's the weather tomorrow based on the infinite length that we, as long as this Earth and the computer will keep running. So, so, and they found that it is like, better than existing, like, transformer or existing architecture in modeling this weather data.[00:42:47] Eugene Cheah: Control for the param size and stuff. I'm quite sure there are people with larger models. So, so there are things that, that in this case, right, there is future applications if your question is just what's next and not what's 10 years ago.[00:42:59] Dan Fu: Thanks so [00:43:00] much for having us. Get full access to Latent Space at www.latent.space/subscribe

How can you make a difference? Do you enjoy blessing others? Join guest speaker Rabbi Stewart Winograd of Reach Initiative International as he provides an update on the endeavors of Reach Initiative and how they, utilizing contributions from Beth Hallel and beyond, have been able to bless Israel and the Jewish people. You can be a blessing to others by loving others and reflecting Yeshua in how you treat people. Be blessed today!Isaiah 9.2; Matthew 5.14-16Prayer Requests or send an email to info@bethhallel.orgCBH WebsiteDonateYouTube Channel

In a touching an emotionally charged convo, Stewart recounts the experience of being in Israel on October 7th, and the many ways that Reach Initiative International [reachii.org] has been serving the victims and families of that horrific day. For over 3 decades Stewart and Chantal have been comforting, serving and sharing their faith with our Jewish people. Their journey also impacts holocaust survivors in Belarus, Ukraine, Russia, Israel and the  young Israeli seekers on the New Age trail in India! You can contact them for their year end newsletter, obtain their latest book,  and support their work at:http://reachii.org/Support the show

Text us is you have a question for Matt or a guest. We want to hear from you. We're excited to bring you an update on what is happening with Together with Families (TwF) since our last episode with Sarah Winograd. TwF is, in my opinion, one of the more innovative organizations working with families impacted by the child welfare system. I had the pleasure of catching up with Sarah and her partner and co-founder Andel Jones-Foster.TwF is an interesting organization for us all to learn from but not because of what they do but rather how they do it. I think what makes them unique is their ability to build relationships, to share power, and to co-design with the families they serve. Their approach leads to an organizational culture and programming that is far more relevant, responsive, and impactful. TwF is about to launch The Hub, which is their new office location and family resource center. They'll also be launching a new website, a short film about their work, and a playbook that anyone can use to learn about and apply their unique approach. We will update these notes with links as soon as the website is live. You are invited to join the Proximity Podcast Club, a community of people supporting one another through their process of becoming who they want to be in this work. We have two options to make this community accessible. We meet every Monday at 9am ET and 9am PT (12pm ET). You can join the club by filling out this form and we'll send you the meeting invite. Proximity Podcast Club - Google FormsPlease connect with me, Matt Anderson, on LinkedIn - Matt Anderson | LinkedIn

How does pelvic health impact your travel experiences?In this episode of the Transform with Travel podcast, I'm joined by Susan Winograd, a pelvic health expert. This insightful discussion explored the often-overlooked connection between pelvic health and travel, providing valuable advice for travelers of all kinds.Susan emphasized the importance of a whole-body wellness approach, integrating various modalities to support clients comprehensively. This approach isn't just about addressing immediate health issues but ensuring long-term wellness. Susan's clinic offers a full spectrum of services, including nutrition and wellness programs, to help clients feel their best throughout life.Susan provided practical tips to maintain health while traveling. Simple practices like changing positions frequently, staying hydrated, and planning for potential constipation can make a big difference. Breathwork was also highlighted as a powerful tool for pelvic health, helping balance the body's systems with every inhale and exhale.Traveling across time zones often disrupts sleep patterns. Susan explained the importance of adjusting to local time as quickly as possible to help regulate bodily functions, including sleep and bladder control. Getting immediate exposure to sunlight upon arrival and integrating familiar routines can aid in this adjustment.Susan's overarching message is to be proactive and plan for your health. By modifying what can be controlled and embracing new experiences, travelers can enhance their well-being. For those looking to connect with Susan, her practice Pelvicore Rehab and Wellness in Boca Raton, Florida, offers a wealth of resources, and she can be found across various online platforms.We talk about:00:00 Intro06:48 Susan's Journey to Pelvic Health10:00 The Whole Body Approach to Wellness17:37 Travel's Impact on Health and Wellness27:33 Maintaining Health on the Road34:56 Exploring Bladder and Bowel Health35:20 Understanding Sleep and Bladder Connection38:55 Tips for Adjusting to New Time Zones45:14 Breathing Techniques for Pelvic Health48:11 Travel Tips and Personal InsightsIn this episode:Episode 33: https://transform-with-travel.captivate.fm/episode/crossing-the-ocean-on-a-paddleboard-philanthropy-travel-with-romi-wallach Connect with Susan LinkedIn: https://www.linkedin.com/in/susan-winograd-8400bb156/Instagram: https://www.instagram.com/pelvicorerehabandwellnessYouTube: https://www.youtube.com/@PelvicHealthWellness Facebook: https://www.facebook.com/pelvicorerehabandwellness https://www.pelvicorerehab.com/ Connect with meFollow the Podcast on IG: http://www.instagram.com/transformwithtravel Follow Kelly Tolliday on IG:

In the first episode of Season 6, we chat with Sofia Winograd, the current Editor-in-Chief of the Houston Law Review. Sofia takes us through her background from growing up in Mexico City to deciding to go to law school. She shares some great tips on how she found success in her law school experience and explains why she decided to join the Law Review, ultimately stepping into the EIC role.Sofia opens up about how being EIC has been both what she expected and a bit different. We also talk about the strong community that Law Review creates and how the Houston Law Review makes a real impact on legal scholarship, even getting cited by the Supreme Court! Plus, she gives us a sneak peek into her upcoming article on Texas Senate Bill 17 and shares her vision for the future of the HLR. Tune in for a great conversation!To get a mailing or electronic subscription to the Houston Law Review click here. For more Emphasis Added content, follow us on Instagram and check out our video content on YouTube!

Send us a textSarah Winograd Babayeuski is a relentless advocate for families involved in the foster care system. Sarah joins me for the second time on the show to update us on the latest transformation of the nonprofit she founded. Together With Families (TWF) harnesses grassroots efforts and community resources to prevent family separations due to poverty. Sarah's shares the evolution of Together With Families, emphasizing the importance of equity, justice, and active family participation in decision-making. We'll dive into the significance of trusting relationships and the harmful effects of pathologizing poverty and trauma. TWF groundbreaking initiatives like the Parent Ally and design team programs, supported by the Annie E. Casey Foundation,  empower families to navigate crises and reduce unnecessary Child Protective Services involvement.Lastly, we'll tackle the formidable challenges that low-income families face, from housing affordability and inadequate wages to bureaucratic obstacles in accessing essential services. Sarah's is an unexpected journey as a nonprofit executive director. Don't miss this inspiring conversation that promises to deepen your understanding of the complexities and solutions in family advocacy.Guest BioAs a missionary kid, Sarah grew up amongst poverty and oppression in the former Soviet Republic of Belarus. After returning to the states with her husband and daughter, she completed her bachelor's degree in psychology at Kennesaw State University.  A former Court Appointed Special Advocate (CASA) in the Cobb County Juvenile Court, where for two years Sarah advocated for the best interests of children involved in dependency hearings. SheI was heartbroken to see so many children separated from loving parents for underlying issues of poverty. She is the founder of Together With Families, a nonprofit working to prevent family seperation due to poverty.Like what you heard? Please like and share wherever you get your podcasts! Connect with Ann: Community Evaluation Solutions How Ann can help: · Support the evaluation capacity of your coalition or community-based organization. · Help you create a strategic plan that doesn't stress you and your group out, doesn't take all year to design, and is actionable. · Engage your group in equitable discussions about difficult conversations. · Facilitate a workshop to plan for action and get your group moving. · Create a workshop that energizes and excites your group for action. · Speak at your conference or event. Have a question or want to know more? Book a call with Ann .Be sure and check out our updated resource page! Let us know what was helpful. Music by Zach Price: Zachpricet@gmail.com

These days, artificial intelligence is a hot topic on the news, social media, our workplaces, and our conversations. Yet AI has a long history. One of its pioneers, Dr. Terry Winograd, has been engaging it for well over 50 years. After finishing his doctorate at MIT, Terry was hired by Stanford University to teach computer science. He's now professor emeritus at Stanford where he spent the bulk of his career teaching and contributing mightily to the field of computer science. And just to give you a sense of his international impact, let's consider the h-index, a way to measure the impact of a person's research. An h-index of 40 is considered outstanding while an h-index of 60 is considered truly exceptional. Terry's is 70, which means his research made a massive splash and has been frequently cited in order to further knowledge on his subject.  And while at Stanford, two of his doctoral students were none other than Larry Page and Sergey Brin, who would later go on to found Google. You'll hear that story and so many others. So, listen in as Terry and I talk about AI and Google, then and now. Sources: https://web.cs.ucla.edu/~palsberg/h-number.html and https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10025721/#:~:text=It%20is%20simple%20to%20calculate,had%20h%2Dindex%20of%2030.

Psychotherapist Arie Winograd (LMFT) has spent the vast majority of his 25-year psychotherapy career treating clients with body dysmorphic disorder. He founded and was the director of the Los Angeles BDD & Body Image Clinic. He is the author of Face to Face with Body Dysmorphic Disorder: Psychotherapy and Clinical Insights. He currently works with BDD clients in his private practice. Interviewed by Dr. Alex Curmi, consultant psychiatrist. If you would like to enquire about an online psychotherapy appointment with Dr. Alex, you can email - alexcurmitherapy@gmail.com.Give feedback here - thinkingmindpodcast@gmail.com -  Follow us here: Twitter @thinkingmindpod Instagram @thinkingmindpodcastJoin Our Mailing List! - https://thinkingmindpod.aidaform.com/mailinglistsignupSUPPORT: buymeacoffee.com/thinkingmind

As Lauren says again and again, if you want to see less human trafficking in the USA, start by surrounding vulnerable families while children are young. The majority of domestic trafficking cases involve children who have aged out of or who have interacted with the U.S. foster system.  Lauren was eager to interview Sarah because of her extensive experience in family advocacy. This conversation turned vulnerable and introspective quickly, as Sarah and Lauren both shared their journeys of waking up to their own misunderstandings about what children need to thrive, as well as their grief over the lack of evolution and learning within the American church. ... Episode Highlights Intro to Sarah and her work (2:48) The U.S. justice system is failing families (11:07) A human-centered approach to poverty (14:06) Humanizing the hurt within the system (22:03) A call for faith communities to protect families (25:56) Ethically portraying the stories of vulnerable families (29:16) Sarah's experience as a CASA volunteer (37:59) Child protection versus family preservation (41:49) How to take action today (50:30) ... For more information, details, and episode transcriptions, visit ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠upwardlydependent.com/shownotes⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. ... Links ⁠Kindred Exchange⁠ - Become a monthly donor to support this show and our mission ⁠Emmaus House⁠ - Learn more about their work with orphaned youth in Haiti  CASA - Learn how you can become a court appointed special advocate in your local community Together for Families - Learn more about their work to protect children, empower families, and strengthen communities ... Connect with Lauren ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Email⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ / ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Instagram⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ / ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Leave a Review⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Connect with Sarah Linkedin

In today's fragmented world, how do we reconcile our understanding of history with the complex issues of our time? With ideologies, conflicts, and historical events so intricately woven into the fabric of our society, it can be challenging to separate truth from misinformation and historical fact from revisionist narratives. This episode of the Biblical Anarchy Podcast delves deep into these issues, examining the layers of history that often go unspoken or misunderstood.How do we, as followers of Christ, navigate the morally gray areas of historical context, especially when looking at events such as the rise of Nazi Germany or the repercussions of Zionism? This episode takes us through a careful examination of historical events that have shaped our understanding of good and evil, shedding light on the often uncomfortable truths that aid in our pursuit of righteousness.How do we approach controversial subjects such as sexual morality and transgender issues from a biblical standpoint? How can we address growing concerns in our society without falling into the same patterns of historical atrocities? With compassion and clarity, Jacob addresses these difficult questions and aims to discern the underlying grievances that can give rise to sinister movements.Are sacrifices of personal freedom and moral integrity ever warranted in the quest to maintain societal order, or does Christ's kingship call us to a higher standard? The episode draws a compelling line between understanding and action, urging Christians not only to cognize historical events but also to become vocal advocates for godly principles in a world that so desperately needs them.Is it possible to honor the memory of the Holocaust while not also recognizing the suffering of the Palestinian people? Winograd demonstrates that remembrance and empathy are not mutually exclusive to one group or event, advocating for a Christ-like view where each individual is cherished and the collective is not vilified.Join us as we unravel the nuanced answers to these complex questions, urging our brothers and sisters in faith to awaken to the dangers of collectivism and embrace individuality through the love of Christ. Listen to this episode for an impassioned plea to view history through the lens of the Gospel and Scripture—to overcome evil with good, to judge fairly, and to stand firmly in the truth that is rooted in God's word.Tune in and let us together find the courage to stand against the tide, resist the allure of collective dogma, and remember that we serve a God who transcends history—a king who calls us to be informed, prayerful, and proactive agents of His unending love and justice.The Biblical Anarchy Podcast is part of the Christians For Liberty Network, a project of the Libertarian Christian Institute. Check out Jacob's other podcast at: daniel318.comAudio Production by Podsworth Media.

We caught up with multi media artist Joe Winograd on working with Dogs In A Pile and The Disco Biscuits  Ep 433 #ATJPod More: https://www.joetut.com/ @allthatjampod on IG, FB, and Twitter - www.allthatjampod.com - Subscribe - leave a review - tell a friend. Merch: https://t.co/QgtAisVtbV All That Jam is brought to you by Executive Producers Amanda Cadran and Kevin Hogan. Produced and edited by Amanda Cadran and Kevin Hogan. Mixed and Mastered by Kevin Hogan. Original Music by Aaron Gaul. Art by Amanda Cadran.

In this episode of Creating the Vision, I'm joined by Susan Winograd, an expert on integrative wellness and owner of an integrative wellness company called Lead Wellness.  We talk about the company's vision and the strategies they use to help so many men and women optimize their health and achieve health independence. Susan emphasizes the significance of personalized approaches, mentorship, and understanding the root of health issues rather than just treating symptoms. She also talks about her services for men's health issues, women's health issues, pre and postnatal care, and digestive, bowel, bladder, and sexual issues. We talk about: 0:00 Intro 1:39 Getting more comfortable with talking about our mental and physical health as women 3:10 Lead Wellness - where did it come from 7:42 The interconnectedness of the work on our self and what we want to achieve 12:36 The difference in seeing your PCP 15:17 Our most important pillar of health 18:24 Most common sexual, bladder and bowel issues 22:35 Kegels after pregnancy and their usefulness 26:00 Communicating our health and wellness needs and goals to advocate for ourselves and create our vision 30:21 Where is lead wellness headed in 2024 32:13 What is her do the dang goal Resources: Workbook: https://a.co/d/9GpwurV Connect with Susan: LinkedIn: https://www.linkedin.com/company/lead-wellness/ https://www.lead-wellness.com/ Connect with me Instagram: https://www.instagram.com/mariamaldonadosmith/ Facebook: https://www.facebook.com/confessionsofacorporatemom LinkedIn: https://www.linkedin.com/in/maria-maldonado-smith-a506805/ Website: https://mmsconsultingfirm.com/  

We caught up with poster artist Joe Winograd about how he got his start and working with Color Red Music Ep 394 #ATJPod More: https://www.joetut.com/ @allthatjampod on IG, FB, and Twitter - www.allthatjampod.com - Subscribe - leave a review - tell a friend. Merch: https://t.co/QgtAisVtbV All That Jam is brought to you by Executive Producers Amanda Cadran and Kevin Hogan. Produced and edited by Amanda Cadran and Kevin Hogan. Mixed and Mastered by Kevin Hogan. Original Music by Aaron Gaul. Art by Amanda Cadran.    

In this episode I talk with Jacob Winograd, host of the Biblical Anarchy podcast, about his political predictions for American politics in 2024. We discuss the depressing state of American foreign policy, the federal debt and spending crisis, and if there will be accountability for Epstein's associates and the politicians and bureaucrats that lied about Covid. We then discuss the presidential election, Trump and Biden, and the prospects of success for the Libertarian party in 2024.  Media Referenced:Jacob Winograd Twitter: @BiblicalAnarchyThe Biblical Anarchy Podcast: https://libertarianchristians.com/shows/biblical-anarchy/Take Human Action PA Podcast Apple: https://podcasts.apple.com/us/podcast/take-human-action-pa/id1575925285Take Human Action PA Podcast Spotify: https://open.spotify.com/show/7qvCAl0xiFPTFzvjDPTpIQ?si=fd6b7b753c9644dc  The Protestant Libertarian Podcast is a project of the Libertarian Christian Institute and a part of the Christians For Liberty Network. The Libertarian Christian Institute can be found at www.libertarianchristians.com. Questions, comments, suggestions? Please reach out to me at theprotestantlibertarian@gmail.com.  You can also follow the podcast on Twitter: @prolibertypod. For more about the show, you can go to theprotestantlibertarianpodcast.com. If you like the show and want to support it, you can! Check out the Protestant Libertarian Podcast page at https://www.buymeacoffee.com/theplpodcast. Also, please consider giving me a star rating and leaving me a review, it really helps expand the shows profile! Thanks! 

Bernstein's Chief Economist Eric Winograd joins host Stacie Jacobsen for a look at U.S. fiscal policy. They discuss the potential costs of running large deficits and accumulating mounting long-term debt in the U.S. and elsewhere. For more context on how the U.S. dollar's reserve currency status could be affected by the national debt, listen to the Disruptor Series roundtable Eric mentions in this episode: http://bit.ly/48GqRZd. Note to All Readers: The information contained here reflects the views of AllianceBernstein L.P. or its affiliates and sources it believes are reliable as of the date of this podcast. AllianceBernstein L.P. makes no representations or warranties concerning the accuracy of any data. There is no guarantee that any projection, forecast or opinion in this material will be realized. Past performance does not guarantee future results. The views expressed here may change at any time after the date of this podcast. This podcast is for informational purposes only and does not constitute investment advice. AllianceBernstein L.P. does not provide tax, legal or accounting advice. It does not take an investor's personal investment objectives or financial situation into account; investors should discuss their individual circumstances with appropriate professionals before making any decisions. This information should not be construed as sales or marketing material or an offer or solicitation for the purchase or sale of any financial instrument, product or service sponsored by AllianceBernstein or its affiliates. The [A/B] logo is a registered service mark of AllianceBernstein, and AllianceBernstein® is a registered service mark, used by permission of the owner, AllianceBernstein L.P. © 2024 AllianceBernstein L.P.  

In this episode, Jimmy is joined by guest Brian Winograd. An automotive industry stalwart, Brian delves into an array of thought-provoking topics, from the nuances of political stances on election outcomes to crucial discussions on the "right to repair" legislative movements and Winograd's direct involvement in these initiatives.00:00 Almost died, spent 3 weeks on vent.09:37 Marketing is key to continued business success.13:56 Self-funded school and orphanage in Ghana.18:31 Empathy fuels success, but mindset needs balance.21:42 Passionate about vocational education for young people.27:21 Women are disproving male domination in the automotive industry.35:20 Tough times in the auto industry, rising trade costs.41:22 Mechanics face disconnect and unequal treatment.43:19 Skilled mechanics demand higher pay but feel neglected.49:48 The Podcast discusses success in the transmission field.55:44 Reasonable person was willing to give away services.01:01:19 Work done, paperwork signed, not my responsibility.01:06:22 Consumers need to take responsibility for vehicles.01:15:36 Prioritize tasks and seize every opportunity.01:16:29 Finding positivity and moving forward after adversity.

CBSP sit down with Clay Whiteley and Mike Winograd . Clay is the commander of training for the Chicago Fire Department at the historic Quinn Fire Academy and Mike has been an instructor with Clay for more than a few academy classes. Clay and Mike are assigned to T20 “America's truck” in Engelwood, located on the city's south side

CBSP sit down with Clay Whiteley and Mike Winograd . Clay is the commander of training for the Chicago Fire Department at the historic Quinn Fire Academy and Mike has been an instructor with Clay for more than a few academy classes. Clay and Mike are assigned to T20 “America's truck” in Engelwood, located on the city's south side

What might it have been like in Israel the last two months? How can we have an impact there even now? Join guest speaker Rabbi Stewart Winograd of Reach Initiative International as he recounts his personal experience in Israel on October 7th and provides updates on how Israel is doing and how Reach Initiative is helping. Even in times of tragedy and conflict, the L-rd can move in a way that is more powerful than anything that may come against. Please continue to pray for Israel, and Shabbat Shalom!Psalm 83.2-4, 18; Galatians 6.10Prayer Requests or send an email to info@bethhallel.orgCBH WebsiteDonateYouTube Channel

Jacob Winograd is the creator of the Biblical Anarchy podcast under the auspices of the Libertarian Christian Institute. He joins Bob to discuss the famous passages from the New Testament where Christians are ostensibly told to pay their taxes and obey the government.Mentioned in the Episode and Other Links of Interest:The YouTube version of this interview.Jacob's Biblical Anarchy Podcast.Jeff Barr's article on rendering unto Caesar.Help support the Bob Murphy Show.

In this episode, I talk with Jacob Winograd, who hosts the Biblical Anarchy podcast for LCI. Outside of his work for LCI, he is heavily involved in both the Libertarian Party and the Mises Caucus; he is the chair of the York County, PA LP affiliate, the chair of the LPPA Social Media Committee, and a Mises Caucus PA State Organizer and Strategist. We discuss the current state of the Libertarian Party, the goals of the Mises Caucus, libertarian infighting, the debate between the Mises Caucus and the Classical Liberal Caucus, the controversy surrounding LP New Hampshire, and the libertarian presidental candidate field. This conversation is both an excellent introduction to the Libertarian Party and a great way to begin the next election cycle.  Media Referenced:The LP Mises Caucus: https://lpmisescaucus.com/The Libertarian Party: https://www.lp.org/Biblical Anarchy Podcast: https://libertarianchristians.com/shows/biblical-anarchy/Jacob on Twitter: @BiblicalAnarchyLP Mises Caucus Pennsylvania: LPMisesCaucusPA  The Protestant Libertarian Podcast is a project of the Libertarian Christian Institute and a part of the Christians For Liberty Network. The Libertarian Christian Institute can be found at www.libertarianchristians.com. Questions, comments, suggestions? Please reach out to me at theprotestantlibertarian@gmail.com.  You can also follow the podcast on Twitter: @prolibertypod. For more about the show, you can go to theprotestantlibertarianpodcast.com. If you like the show and want to support it, you can! Check out the Protestant Libertarian Podcast page at https://www.buymeacoffee.com/theplpodcast. Also, please consider giving me a star rating and leaving me a review, it really helps expand the shows profile! Thanks!

What is pelvic core? why do we need to work on this area of our body? Susan Winograd shares the importance of either keeping your pevic strong or what can happen if its too tight. Reach out to me on Instagram to connect with my guest! @arielle.leavitt

Our guest today is Vid Kocijan, a Machine Learning Engineer at Kumo AI. Vid has a Ph.D. in Computer Science at the University of Oxford. His research focused on common sense reasoning, pre-training in LLMs, pretraining in knowledge-based completion, and how these pre-trainings impact societal bias. He joins us to discuss how he built a BERT model that solved the Winograd Schema Challenge.

In episode 87 of The Gradient Podcast, Daniel Bashir speaks to Professor Terry Winograd. Professor Winograd is Professor Emeritus of Computer Science at Stanford University. His research focuses on human-computer interaction design and the design of technologies for development. He founded the Stanford Human-Computer Interaction Group, where he directed the teaching programs and HCI research. He is also a founding faculty member of the Stanford d.school and a founding member and past president of Computer Professionals for Social Responsibility.Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pubSubscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterOutline:* (00:00) Intro* (03:00) Professor Winograd's background* (05:10) At the MIT AI Lab* (05:45) The atmosphere in the MIT AI Lab, Minsky/Chomsky debates* (06:20) Blue-sky research, government funding for academic research* (10:10) Isolation and collaboration between research groups* (11:45) Phases in the development of ideas and how cross-disciplinary work fits in* (12:26) SHRDLU and the MIT AI Lab's intellectual roots* (17:20) Early responses to SHRDLU: Minsky, Dreyfus, others* (20:55) How Prof. Winograd's thinking about AI's abilities and limitations evolved* (22:25) How this relates to current AI systems and discussions of intelligence* (23:47) Repetitive debates in AI, semantics and grounding* (27:00) The concept of investment, care, trust in human communication vs machine communication* (28:53) Projecting human-ness onto AI systems and non-human things and what this means for society* (31:30) Time after leaving MIT in 1973, time at Xerox PARC, how Winograd's thinking evolved during this time* (38:28) What Does It Mean to Understand Language? Speech acts, commitments, and the grounding of language* (42:40) Reification of representations in science and ML* (46:15) LLMs, their training processes, and their behavior* (49:40) How do we coexist with systems that we don't understand?* (51:20) Progress narratives in AI and human agency* (53:30) Transitioning to intelligence augmentation, founding the Stanford HCI group and d.school, advising Larry Page and Sergey Brin* (1:01:25) Chatbots and how we consume information* (1:06:52) Evolutions in journalism, progress in trust for modern AI systems* (1:09:18) Shifts in the social contract, from institutions to personalities* (1:12:05) AI and HCI in recent years* (1:17:05) Philosophy of design and the d.school* (1:21:20) Designing AI systems for people* (1:25:10) Prof. Winograd's perspective on watermarking for detecting GPT outputs* (1:25:55) The politics of being a technologist* (1:30:10) Echos of the past in AI regulation and competition and learning from history* (1:32:34) OutroLinks:* Professor Winograd's Homepage* Papers/topics discussed:* SHRDLU* Beyond Programming Languages* What Does It Mean to Understand Language?* The PageRank Citation Ranking* Stanford Digital Libraries project* Talk: My Politics as a Technologist Get full access to The Gradient at thegradientpub.substack.com/subscribe

Welcome to the EMPOWER SERIES (I)! We're kicking off this series with an exclusive interview with Susan Winograd, a physical therapist and owner of Pelvic Core Rehab in Boca Raton, Florida. Susan discusses how she turned her moment of pain in pregnancy - to helping and supporting other women in their Pelvic Health journey... and built a thriving business at the same time! Connect with Susan! www.pelvicorerehab.com Facebook: @pelvicorerehab Private FCBK group: Pelvic Core Plus Total Body Wellness Instagram: @pelvicorerehab Susan's podcast: Your Pelvic Health YouTube: Pelvic Core Plus This episode is sponsored by: Classic Jewlz Head over to www.classicjewelz.com or on Instagram @keepin_it_classic to find the latest collection of beautiful jewelry and enter promo code Devora30 for a 30% discount! --- Send in a voice message: https://podcasters.spotify.com/pod/show/devora-goldberg/message

Powered by Eaton Financial Group: where their motto is to live well, work hard and give generously. Under the leadership of Doug Eaton - Eaton provides counsel to its family of clients on complex financial issues, delivering a high level of proactive care and customized financial plans that meet each client's individual needs. The firm specializes in assisting women who handle household financial planning, by acting in a fiduciary capacity – meaning your best interests are their first priority.  For a relaxed and no obligation cup of coffee and second opinion – live or virtually – visit eatonfinancialgroup.com to set an appointment. Susan Winograd, MSPT, owns a private Pelvic Health and Wellness practice in Boca Raton, Florida called Pelvicore Rehab ​Pelvicore Rehab which specializes in Women's Health, Men's Health issues, and Pre and Postnatal Care and digestive, bowel, bladder and sexual issues. ​Susan enjoys teaching workshops internationally. Her hands-on treatment approach of treating the whole person has often led her to discover the primary or contributing cause of pain, dysfunction, or imbalance was pelvic floor related. Susan has expanded Pelvicore Rehab to provide clients with complete and comprehensive healing and wellness programs using a revolutionary system and approach to educate, empower and inspire people to optimize their health and achieve health independence for the rest of their life! ​ Her passion and commitment now is helping to empower and educate her patients to address the underlying root cause of their health concerns for the long term using a whole body approach to relieve pain, restore optimal function, and improve quality of life. ​She is committed to providing the highest quality of individualized care that every patient deserves. In this episode, we discuss; The importance of pelvic floor physical therapy for women and men.  How to tell if your pelvic floor is in need of therapy.  Why common pelvic floor issues may NOT be normal and should be corrected.  How your current pelvic health affects you throughout your lifespan.  The inspiration behind her new wellness program and her 5 pillars of health.  Her new transition of motherhood and building her business.  Connect with Susan at https://www.facebook.com/pelvicorerehab/ , https://www.instagram.com/pelvicorerehab and https://podcasts.apple.com/us/podcast/your-pelvic-health/id1661663899 Connect with me at: Instagram: https://www.instagram.com/the.motherhood.village1/ YouTube: The Motherhood Village - YouTube LinkedIn: https://www.linkedin.com/in/nicolegcumberbatch/  Website: https://www.themotherhoodvillage.com/ Loved this episode? Leave a review and rating here: https://podcasts.apple.com/us/podcast/the-motherhood-village-podcast/id1487274178

We are now launching our dedicated new YouTube and Twitter! Any help in amplifying our podcast would be greatly appreciated, and of course, tell your friends! Notable followon discussions collected on Twitter, Reddit, Reddit, Reddit, HN, and HN. Please don't obsess too much over the GPT4 discussion as it is mostly rumor; we spent much more time on tinybox/tinygrad on which George is the foremost authority!We are excited to share the world's first interview with George Hotz on the tiny corp!If you don't know George, he was the first person to unlock the iPhone, jailbreak the PS3, went on to start Comma.ai, and briefly “interned” at the Elon Musk-run Twitter. Tinycorp is the company behind the deep learning framework tinygrad, as well as the recently announced tinybox, a new $15,000 “luxury AI computer” aimed at local model training and inference, aka your “personal compute cluster”:* 738 FP16 TFLOPS* 144 GB GPU RAM* 5.76 TB/s RAM bandwidth* 30 GB/s model load bandwidth (big llama loads in around 4 seconds)* AMD EPYC CPU* 1600W (one 120V outlet)* Runs 65B FP16 LLaMA out of the box (using tinygrad, subject to software development risks)(In the episode, we also talked about the future of the tinybox as the intelligence center of every home that will help run models, at-home robots, and more. Make sure to check the timestamps

A psychotherapist and the founder and director of the Los Angeles BDD & Body Image Clinic, Arie has dedicated his career to working with people with BDD. In this episode, he shares some of the fascinating insights he's gained from those thousands of hours of experience.

During the past several decades, the United States has experienced several distinct waves of opioid epidemic, prompting those providing services to think of novel ways to provide services. to those in need. Dr. Rachel Winograd found her passion for community-based harm reduction services in St. Louis, a city ravaged by wave after wave of the opioid epidemic, and quickly found her place within a network of harm reductionists, community-based providers, and policy makers in Missouri with a common goal. In this episode, Dr. Winograd teaches us about the opioid epidemic and shares about the work she and her team has done to test and disseminate evidence-based harm reduction services in the greater St. Louis area. Dr. Winograd is an Associate Professor in the Department of Psychological Sciences and the Missouri Institute of Mental Health at the University of Missouri-St Louis. To learn more, please visit www.mimhaddisci.org.

#RiseToLiberty #BiblicalAnarchy #Christianity I sit down with Jacob Daniel, host of the Biblical Anarchy podcast. We sit down and discuss the connection of Christianity and liberty. We also discuss the naysayers within the libertarian party and the lack of morals that some people have within the party. Make sure to check his links out below! ------------- GUEST LINKS: Podcast: https://libertarianchristians.com/shows/biblical-anarchy/ Twitter: https://twitter.com/biblicalanarchy?s=21&t=KME2TcbHVS8oYyThQyuJnA YouTube: https://youtube.com/@thebiblicalanarchypodcast ------------- RISE TO LIBERTY LINKS: - Master Link: https://risetoliberty.com/links - RTL on Podbean: https://risetoliberty.com/podbean-home - Rise To Liberty Merch Store: https://risetoliberty.store - RTL On Odysee: https://risetoliberty.com/odysee - RTL On Rumble: https://risetoliberty.com/rumble - RTL Telegram Chat: https://risetoliberty.com/freespeech - RTL On Substack - Liberty Letter!: https://risetoliberty.substack.com - AUDIO PLATFORMS: https://risetoliberty.com/audio - Buy Fresh Coffee w/ Monero! (XMR): https://risetoliberty.com/gratuitas-xmr  

Jacob Winograd, host of the Biblical Anarchy podcast, joins me on this episode to discuss anarchism. The word 'anarchy' is often used synonymously with terms like 'chaos', 'disorder', and 'lawlessness', but Jacob argues that anarchy actually means not giving one institution in society a monopoly on power and instead producing cooperation and peace through decentralized, voluntary association and hierarchy. He contends that we have misunderstood the word and that it is the ideal political arrangement. We then discuss the compatibility between anarchy, the Bible, and Christian faith, and explore the current state of the liberty movement in the US. Media Referenced:Jacob Winograd Twitter: @BiblicalAnarchy The Biblical Anarchy Podcast: https://libertarianchristians.com/shows/biblical-anarchy/ Take Human Action PA Podcast Apple: https://podcasts.apple.com/us/podcast/take-human-action-pa/id1575925285 Take Huiman Action PA Podcast Spotify: https://open.spotify.com/show/7qvCAl0xiFPTFzvjDPTpIQ?si=fd6b7b753c9644dcThe Protestant Libertarian Podcast is a project of the Libertarian Christian Institute and a part of the Christians For Liberty Network. The Libertarian Christian Institute can be found at www.libertarianchristians.com. Questions, comments, suggestions? Please reach out to me at theprotestantlibertarian@gmail.com.  You can also follow the podcast on Twitter: @prolibertypod. For more about the show, you can go to theprotestantlibertarianpodcast.com. If you like the show and want to support it, you can! Check out the Protestant Libertarian Podcast page at https://www.buymeacoffee.com/theplpodcast. Also, please consider giving me a star rating and leaving me a review, it really helps expand the shows profile! Thanks!

Are you in search of truth? Do you feel spiritually burned out? Join guest speakers Rabbi Stewart and Chantal Winograd as they reminisce over their spiritual journeys and how they found the Messiah, Yeshua. May these powerful testimonies serve as an encouragement to you! Shabbat Shalom!Romans 3.10; John 8.3-11; John 3.3-7Prayer Requests or send an email to info@bethhallel.orgCBH WebsiteDonateYouTube Channel

Join Susan Winograd, PT and founder of PelviCore Rehab, for an enlightening conversation on pelvic health. In this episode, Susan and our host discuss how to break the stigma and reshape the conversation around pelvic health. Susan shares her unique approach to helping women and men of all ages optimize their pelvic health and overall well-being. Whether you're experiencing incontinence, pelvic pain, or other pelvic-related concerns, this episode offers practical insights and strategies for improving your quality of life." Learn about Susan Winograd at www.pelvicorerehab.com. Show sponsored by www.FusionaryFormulas.com. [Warning: Adult topics discussed in this podcast]

Jacob and I talked about liberty and why he converted from the left. GUEST LINKS: https://libertarianchristians.com/shows/biblical-anarchy/ ASK ME ANYTHING: https://anchor.fm/rebelwithacause/message rebelwithacausepodcast@protonmail.com MY LINKS: https://cointr.ee/rwac-podcast --- Support this podcast: https://anchor.fm/rebelwithacause/support

Jacob Winograd of the Biblical Anarchy Podcast joins us again, this time to talk about his journey from being a hardcore Bernie Bro leftist to embracing biblical anarchy nearly overnight! Learn how Jacob grew to become the advocate for liberty he is today! Audio Production by Podsworth Media - https://podsworth.com

In episode 216, we hear from US care reform leader, Sarah Winograd from Together for Families. Sarah speaks passionately about the complexities in child welfare and foster care as a system, how poverty contributes to family separation, and the problematic ways we've seen and judged birth families. She asks the hard questions- if most children aren't coming into foster care because of abuse, what resources and systems are we investing in to keep those families together? What are we doing to intervene and stabilize families in order to prevent separation and keep children in families? Can the church do better? Full show notes can be found at thinkorphan.com

This week on Antiwar Radio, Scott talks with Hassan El-Tayyab about the vote next week to end support for the Saudi war in Yemen. The two give a brief background on the war to explain how we got here before explaining next week's vote. Next Scott talks with Marcy Winograd of CODEPINK. She recently published a letter to the Left, urging leftists to commit to their principles and call for negotiations to bring about peace in Ukraine.   Discussed on the show: 1833stopwar.com  “Letter to the Left on Ukraine” (CODEPINK) Joint Statement on the U.S.-Ukraine Strategic Partnership Wales Summit in 2014 Peace in Ukraine Coalition Hassan El-Tayyab is a musician and peace activist, who works as the lead lobbyist on Middle East policy for the Friends Committee on National Legislation. Marcy Winograd is the Coordinator of CODEPINK Congress and a steering committee member of the Peace in Ukraine Coalition. The coalition calls for a ceasefire, peace negotiations and investments in climate, housing, healthcare and jobs, not endless war in Ukraine. To join the coalition, visit peaceinukraine.org This episode of the Scott Horton Show is sponsored by: Tom Woods' Liberty Classroom; ExpandDesigns.com/Scott; and Thc Hemp Spot. Get Scott's interviews before anyone else! Subscribe to the Substack. Shop Libertarian Institute merch or donate to the show through Patreon, PayPal or Bitcoin: 1DZBZNJrxUhQhEzgDh7k8JXHXRjYu5tZiG. Learn more about your ad choices. Visit megaphone.fm/adchoices

 Download Episode. This week on Antiwar Radio, Scott talks with Hassan El-Tayyab about the vote next week to end support for the Saudi war in Yemen. The two give a brief background on the war to explain how we got here before explaining next week's vote. Next Scott talks with Marcy Winograd of CODEPINK. She recently published a letter to the Left, urging leftists to commit to their principles and call for negotiations to bring about peace in Ukraine.   Discussed on the show: 1833stopwar.com  “Letter to the Left on Ukraine” (CODEPINK) Joint Statement on the U.S.-Ukraine Strategic Partnership Wales Summit in 2014 Peace in Ukraine Coalition Hassan El-Tayyab is a musician and peace activist, who works as the lead lobbyist on Middle East policy for the Friends Committee on National Legislation. Marcy Winograd is the Coordinator of CODEPINK Congress and a steering committee member of the Peace in Ukraine Coalition. The coalition calls for a ceasefire, peace negotiations and investments in climate, housing, healthcare and jobs, not endless war in Ukraine. To join the coalition, visit peaceinukraine.org This episode of the Scott Horton Show is sponsored by: Tom Woods' Liberty Classroom; ExpandDesigns.com/Scott; and Thc Hemp Spot. Get Scott's interviews before anyone else! Subscribe to the Substack. Shop Libertarian Institute merch or donate to the show through Patreon, PayPal or Bitcoin: 1DZBZNJrxUhQhEzgDh7k8JXHXRjYu5tZiG.

Jacob Winograd comes back on the show to talk about the Ye/Jones/Fuentes debacle, the nuance in the Judaism/Jewish/Zionist discussion, and if there really is an agenda controlling the media.   SHOW LINKS: https://linktr.ee/ReedCoverdale   GUEST LINKS: https://linktr.ee/BiblicalAnarchy

We were delighted to be joined by Davis Professor at the Sante Fe Insitute, Melanie Mitchell! We chat about our understanding of artificial intelligence, human intelligence, and whether it's reasonable to expect us to be able to build sophisticated human-like automated systems anytime soon. Follow Melanie on twitter @MelMitchell1 and check out her website: https://melaniemitchell.me/ We discuss: - AI hype through the ages - How do we know if machines understand? - Winograd schemas and the "WinoGrande" challenge. - The importance of metaphor and analogies to intelligence - The four fallacies in AI research: - 1. Narrow intelligence is on a continuum with general intelligence - 2. Easy things are easy and hard things are hard - 3. The lure of wishful mnemonics - 4. Intelligence is all in the brain - Whether embodiment is necessary for true intelligence - Douglas Hofstadter's views on AI - Ray Kurzweil and the "singularity" - The fact that Moore's law doesn't hold for software - The difference between symbolic AI and machine learning - What analogies have to teach us about human cognition Errata - Ben mistakenly says that Eliezer Yudkowsky has bet that everyone will die by 2025. It's actually by 2030. You can find the details of the bet here: https://www.econlib.org/archives/2017/01/my_end-of-the-w.html. References: - NY Times reporting on Perceptrons (https://www.nytimes.com/1958/07/13/archives/electronic-brain-teaches-itself.html). - The WinoGrande challenge paper (https://arxiv.org/abs/1907.10641) - Why AI is harder than we think (https://arxiv.org/pdf/2104.12871.pdf) - The Singularity is Near (https://smile.amazon.com/Singularity-Near-Humans-Transcend-Biology/dp/0143037889?sa-no-redirect=1), by Ray Kurzweil Contact us - Follow us on Twitter at @IncrementsPod, @BennyChugg, @VadenMasrani - Check us out on youtube at https://www.youtube.com/channel/UC_4wZzQyoW4s4ZuE4FY9DQQ - Come join our discord server! DM us on twitter or send us an email to get a supersecret link Eliezer was more scared than Douglas about AI, so he wrote a blog post about it. Who wrote the blog post, Eliezer or Douglas? Tell us at over at incrementspodcast@gmail.com. Special Guest: Melanie Mitchell.

Jacob Daniel Winograd of the "Daniel 3 Biblical Anarchy Podcast" joins us to discuss his transition from Bernie Sanders leftism to Austro/Anarcho-libertarianism, including his approach to interpreting Romans 13 in light of Romans 12. Audio Production by Podsworth Media.

Sarah Winograd's journey as a “professional volunteer” led to the realization that poverty was a driving factor in the child removals in the families she was working with. See how Sarah mobilized her community to address family poverty to help families stay together. Through the Together for Families program, Sarah lives out her vision of supporting families by helping them meet basic needs. Show Notes00:00:30  | Matt opens this episode reflecting on the child welfare system's association between poverty and neglect00:01:10  | Meet Sarah Winograd: Program Manager for Together for Families, Advocates for Children, and an adoptive mom. 00:05:05  | Back in the U.S. as an adult, Sarah dedicated herself to volunteering where she would begin spending a lot of her time working with youth formerly in foster care in New York and later in Georgia. 00:06:45  | Sarah talks about the first case she worked on as a CASA volunteer in Georgia. 00:10:37  | Who was representing and supporting the mom in Sarah's case?  00:11:28  | Sarah explains the “ah-ha” moment she experienced while talking to one of the children in the family. This helped her fill in gaps that were missing from the family's case file. 00:15:20  | Matt and Sarah discuss a shift in thinking around the reason Sarah became a CASA volunteer—from helping kids to helping the whole family.  00:17:38  | Sarah explains some of the support she provided to the family while staying within the boundaries of her role as a CASA volunteer.00:20:37  | Sarah shares how she received the reputation for the "resource queen” by helping families not on her case load meet their basic needs and stay in-tact. 00:22:59  | Sarah shares her findings  on poverty as a driver of child welfare involvements, as well as how her colleagues felt about the realities of the families they served. 00:25:10  | Sarah talks about the conversation with her CASA supervisor.00:30:54  | Matt reflects on the punitive structure of the child welfare system and Sarah's approach to seeing families for their strengths and with empathy, rather than defining them by their circumstances. 00:32:14  | What's next for Sarah after CASA?  00:41:14  | Sarah's vision of what's next for the Together for Families program. 00:44:40  | Advice for people seeing the same issues in their community who want to address the needs of families. 00:45:53  | Final thoughts from Matt Anderson. ResourcesTogether for Families | Advocates for Children Georgia ranks 38th in the Nation for Child and Family Well-Being | Georgia Family Connection Partnership One promise became a lifelong mission for this Atlanta family advocate | CBS46 Cobb County, GA Child Welfare Stats | Fostering Court Improvement A Key Connection: Economic Stability and Family Well-being | Chapin Hall at the University of Chicago Child Welfare: Purposes, Federal Programs, and Funding | Congressional Research Service Reports Child Welfare Financing SFY 2018: A survey of federal, state, and local expenditures | Child Trends 

Today we welcome Daniel Winograd to the show. Daniel is the host of the Daniel 3 Biblical Anarchy podcast, which you've heard advertised on this show. John and Daniel talk about his path to liberty and his life growing up as a pastor's son. They dive into the biblical text and how it relates to the principle of liberty. Subscribe to John's Finding Freedom Show solo feed. Listen and Subscribe on Apple Podcasts and Spotify. Get 20% off your selection of the AMAZING CBD products over at PalomaVerdeCBD.com and use discount code "ROAR" at checkout! Get access to all of our bonus audio content, livestreams, behind-the-scenes segments and more for as little as $5 per month by joining the Lions of Liberty Pride on Patreon OR support us on Locals! Lions also get 20% off all merchandise at the Lions of Liberty Store, including our hot-off-the-press Hands Up Don't Nuke! T-Shirt!