Podcasts about ssms

Many announcements and interesting releases came out this past month, so we got a lot to talk about in this episode! Relevant links: What's New in SQL Server 2025 - SQL Server | Microsoft Learn SQL Server 2025 - AI ready enterprise database from ground to cloud | Microsoft Community Hub Announcing Public Preview of DiskANN in SQL Server 2025 | Microsoft Community Hub SQL Server 2025: introducing optimized Halloween protection | Microsoft Community Hub SQL Server 2025: introducing tempdb space resource governance | Microsoft Community Hub Unlocking the Power of Regex in SQL Server - Azure SQL Devs' Corner Announcing the General Availability (GA) of JSON data type and JSON aggregates | Microsoft Community Hub Announcing the Public Preview of JSON index in SQL Server 2025 | Microsoft Community Hub ZSTD compression in SQL Server 2025 | Microsoft Community Hub MSSQL Extension for VS Code: GitHub Copilot Preview + UI GA MSSQL Extension for VS Code: Introducing Schema Compare (Preview) - Azure SQL Devs' Corner SQL Server Management Studio (SSMS) 21 is now generally available (GA) | Microsoft Community Hub Copilot in SSMS preview Recently released: Updates to the SqlPackage and the DacFx ecosystem | Microsoft Community Hub Visual Studio 2022 Release Notes | Microsoft Learn Free SQL Managed Instance offer is now generally available Avoid T-SQL anti-patterns with the free T-SQL analysis tool - Azure SQL Devs' Corner

Erin Stellato, a program manager at Microsoft, asked a very interesting question on LinkedIn: "why do you *not* want a Copilot in SSMS?" That got me to stop and think a minute. Why don't we want to use an AI to help us? It's a good question, given the hype and (maybe) potential of the technology. While it might not help you now, or with your specific thing, it might help others, so are there good reasons not to use AI technologies, like the GenAI LLMs? Read the rest of Why Not Use AI?

I don't know how many of you will be disappointed or impacted by this, but Azure Data Studio (ADS) is being retired, as of 6 Feb, 2024. It will be supported for a little over a year, until 28 Feb, 2026. On one hand I'm not surprised, and on the other, I'm a little shocked by this. I have written a number of articles on ADS, and shown how things work, as well as pointed out a number of things that don't work well in the product or its extensions. These pieces have gotten a number of reads, and people have commented on them, so I wonder if there are a lot of you that are upset by this. Is this going to change the way you work? I will say that it will lightly change my work, as I do use ADS to connect to PostgreSQL, but not so much for SQL Server. Read the rest of The End of Azure Data Studio

Guy and Eitan answer an interesting question from the forum regarding the logic of NOT BETWEEN. And also, checking out the SQL Squid Game, discussing SQL Audit, and sharing some tips for SSMS. Relevant links: SQL Squid Game SQL Server Audit (Database Engine) - SQL Server | Microsoft Learn microsoft-dbas-club/Maintenance/SQLVersionUpdates Tell us how you did and whether you survived the SQL Squid Game!

Happy holidays! We'll be sharing snippets from Latent Space LIVE! through the break bringing you the best of 2024! We want to express our deepest appreciation to event sponsors AWS, Daylight Computer, Thoth.ai, StrongCompute, Notable Capital, and most of all all our LS supporters who helped fund the gorgeous venue and A/V production!For NeurIPS last year we did our standard conference podcast coverage interviewing selected papers (that we have now also done for ICLR and ICML), however we felt that we could be doing more to help AI Engineers 1) get more industry-relevant content, and 2) recap 2024 year in review from experts. As a result, we organized the first Latent Space LIVE!, our first in person miniconference, at NeurIPS 2024 in Vancouver.Of perennial interest, particularly at academic conferences, is scaled-up architecture research as people hunt for the next Attention Is All You Need. We have many names for them: “efficient models”, “retentive networks”, “subquadratic attention” or “linear attention” but some of them don't even have any lineage with attention - one of the best papers of this NeurIPS was Sepp Hochreiter's xLSTM, which has a particularly poetic significance as one of the creators of the LSTM returning to update and challenge the OG language model architecture:So, for lack of a better term, we decided to call this segment “the State of Post-Transformers” and fortunately everyone rolled with it.We are fortunate to have two powerful friends of the pod to give us an update here:* Together AI: with CEO Vipul Ved Prakash and CTO Ce Zhang joining us to talk about how they are building Together together as a quote unquote full stack AI startup, from the lowest level kernel and systems programming to the highest level mathematical abstractions driving new model architectures and inference algorithms, with notable industry contributions from RedPajama v2, Flash Attention 3, Mamba 2, Mixture of Agents, BASED, Sequoia, Evo, Dragonfly, Dan Fu's ThunderKittens and many more research projects this year* Recursal AI: with CEO Eugene Cheah who has helped lead the independent RWKV project while also running Featherless AI. This year, the team has shipped RWKV v5, codenamed Eagle, to 1.5 billion Windows 10 and Windows 11 machines worldwide, to support Microsoft's on-device, energy-usage-sensitive Windows Copilot usecases, and has launched the first updates on RWKV v6, codenamed Finch and GoldFinch. On the morning of Latent Space Live, they also announced QRWKV6, a Qwen 32B model modified with RWKV linear attention layers. We were looking to host a debate between our speakers, but given that both of them were working on post-transformers alternativesFull Talk on YoutubePlease like and subscribe!LinksAll the models and papers they picked:* Earlier Cited Work* Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention* Hungry hungry hippos: Towards language modeling with state space models* Hyena hierarchy: Towards larger convolutional language models* Mamba: Linear-Time Sequence Modeling with Selective State Spaces* S4: Efficiently Modeling Long Sequences with Structured State Spaces* Just Read Twice (Arora et al)* Recurrent large language models that compete with Transformers in language modeling perplexity are emerging at a rapid rate (e.g., Mamba, RWKV). Excitingly, these architectures use a constant amount of memory during inference. However, due to the limited memory, recurrent LMs cannot recall and use all the information in long contexts leading to brittle in-context learning (ICL) quality. A key challenge for efficient LMs is selecting what information to store versus discard. In this work, we observe the order in which information is shown to the LM impacts the selection difficulty. * To formalize this, we show that the hardness of information recall reduces to the hardness of a problem called set disjointness (SD), a quintessential problem in communication complexity that requires a streaming algorithm (e.g., recurrent model) to decide whether inputted sets are disjoint. We empirically and theoretically show that the recurrent memory required to solve SD changes with set order, i.e., whether the smaller set appears first in-context. * Our analysis suggests, to mitigate the reliance on data order, we can put information in the right order in-context or process prompts non-causally. Towards that end, we propose: (1) JRT-Prompt, where context gets repeated multiple times in the prompt, effectively showing the model all data orders. This gives 11.0±1.3 points of improvement, averaged across 16 recurrent LMs and the 6 ICL tasks, with 11.9× higher throughput than FlashAttention-2 for generation prefill (length 32k, batch size 16, NVidia H100). We then propose (2) JRT-RNN, which uses non-causal prefix-linear-attention to process prompts and provides 99% of Transformer quality at 360M params., 30B tokens and 96% at 1.3B params., 50B tokens on average across the tasks, with 19.2× higher throughput for prefill than FA2.* Jamba: A 52B Hybrid Transformer-Mamba Language Model* We present Jamba, a new base large language model based on a novel hybrid Transformer-Mamba mixture-of-experts (MoE) architecture. * Specifically, Jamba interleaves blocks of Transformer and Mamba layers, enjoying the benefits of both model families. MoE is added in some of these layers to increase model capacity while keeping active parameter usage manageable. * This flexible architecture allows resource- and objective-specific configurations. In the particular configuration we have implemented, we end up with a powerful model that fits in a single 80GB GPU.* Built at large scale, Jamba provides high throughput and small memory footprint compared to vanilla Transformers, and at the same time state-of-the-art performance on standard language model benchmarks and long-context evaluations. Remarkably, the model presents strong results for up to 256K tokens context length. * We study various architectural decisions, such as how to combine Transformer and Mamba layers, and how to mix experts, and show that some of them are crucial in large scale modeling. We also describe several interesting properties of these architectures which the training and evaluation of Jamba have revealed, and plan to release checkpoints from various ablation runs, to encourage further exploration of this novel architecture. We make the weights of our implementation of Jamba publicly available under a permissive license.* SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers* We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096×4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU. Core designs include: * (1) Deep compression autoencoder: unlike traditional AEs, which compress images only 8×, we trained an AE that can compress images 32×, effectively reducing the number of latent tokens. * (2) Linear DiT: we replace all vanilla attention in DiT with linear attention, which is more efficient at high resolutions without sacrificing quality. * (3) Decoder-only text encoder: we replaced T5 with modern decoder-only small LLM as the text encoder and designed complex human instruction with in-context learning to enhance the image-text alignment. * (4) Efficient training and sampling: we propose Flow-DPM-Solver to reduce sampling steps, with efficient caption labeling and selection to accelerate convergence. * As a result, Sana-0.6B is very competitive with modern giant diffusion model (e.g. Flux-12B), being 20 times smaller and 100+ times faster in measured throughput. Moreover, Sana-0.6B can be deployed on a 16GB laptop GPU, taking less than 1 second to generate a 1024×1024 resolution image. Sana enables content creation at low cost. * RWKV: Reinventing RNNs for the Transformer Era* Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence length. In contrast, recurrent neural networks (RNNs) exhibit linear scaling in memory and computational requirements but struggle to match the same performance as Transformers due to limitations in parallelization and scalability. * We propose a novel model architecture, Receptance Weighted Key Value (RWKV), that combines the efficient parallelizable training of transformers with the efficient inference of RNNs.* Our approach leverages a linear attention mechanism and allows us to formulate the model as either a Transformer or an RNN, thus parallelizing computations during training and maintains constant computational and memory complexity during inference. * We scale our models as large as 14 billion parameters, by far the largest dense RNN ever trained, and find RWKV performs on par with similarly sized Transformers, suggesting future work can leverage this architecture to create more efficient models. This work presents a significant step towards reconciling trade-offs between computational efficiency and model performance in sequence processing tasks.* LoLCATs: On Low-Rank Linearizing of Large Language Models* Recent works show we can linearize large language models (LLMs) -- swapping the quadratic attentions of popular Transformer-based LLMs with subquadratic analogs, such as linear attention -- avoiding the expensive pretraining costs. However, linearizing LLMs often significantly degrades model quality, still requires training over billions of tokens, and remains limited to smaller 1.3B to 7B LLMs. * We thus propose Low-rank Linear Conversion via Attention Transfer (LoLCATs), a simple two-step method that improves LLM linearizing quality with orders of magnitudes less memory and compute. * We base these steps on two findings. * First, we can replace an LLM's softmax attentions with closely-approximating linear attentions, simply by training the linear attentions to match their softmax counterparts with an output MSE loss ("attention transfer").* Then, this enables adjusting for approximation errors and recovering LLM quality simply with low-rank adaptation (LoRA). * LoLCATs significantly improves linearizing quality, training efficiency, and scalability. We significantly reduce the linearizing quality gap and produce state-of-the-art subquadratic LLMs from Llama 3 8B and Mistral 7B v0.1, leading to 20+ points of improvement on 5-shot MMLU. * Furthermore, LoLCATs does so with only 0.2% of past methods' model parameters and 0.4% of their training tokens. * Finally, we apply LoLCATs to create the first linearized 70B and 405B LLMs (50x larger than prior work). * When compared with prior approaches under the same compute budgets, LoLCATs significantly improves linearizing quality, closing the gap between linearized and original Llama 3.1 70B and 405B LLMs by 77.8% and 78.1% on 5-shot MMLU.Timestamps* [00:02:27] Intros* [00:03:16] Why Scale Context Lengths? or work on Efficient Models* [00:06:07] The Story of SSMs* [00:09:33] Idea 1: Approximation -> Principled Modeling* [00:12:14] Idea 3: Selection* [00:15:07] Just Read Twice* [00:16:51] Idea 4: Test Time Compute* [00:17:32] Idea 2: Hardware & Kernel Support* [00:19:49] RWKV vs SSMs* [00:24:24] RWKV Arch* [00:26:15] QWRKWv6 launch* [00:30:00] What's next* [00:33:21] Hot Takes - does anyone really need long context?Transcript[00:00:00] AI Charlie: We're back at Latent Space Live, our first mini conference held at NeurIPS 2024 in Vancouver. This is Charlie, your AI co host. As a special treat this week, we're recapping the best of 2024 going domain by domain. We sent out a survey to the over 900 of you who told us what you wanted, and then invited the best speakers in the Latent Space Network to cover each field.[00:00:24] AI Charlie: 200 of you joined us in person throughout the day, with over 2200 watching live online. Thanks Our next keynote covers the State of Transformers alternative architectures, with a special joint presentation with Dan Fu of Together AI and Eugene Chia of Recursal AI and Featherless AI. We've featured both Together and Recursal on the pod before, with CEO Veepal Vedprakash introducing them.[00:00:49] AI Charlie: And CTO CE Zhang joining us to talk about how they are building together together as a quote unquote full stack AI startup from the lowest level kernel and systems [00:01:00] programming to the highest level mathematical abstractions driving new model architectures and inference algorithms with notable industry contributions from Red Pajama V2, Flash Attention 3, Mamba 2, Mixture of Agents.[00:01:15] AI Charlie: Based, Sequoia, Evo, Dragonfly, Danfoo's Thunder Kittens, and many more research projects this year. As for Recursal and Featherless, we were the first podcast to feature RWKV last year, and this year the team has shipped RWKV v5, codenamed Eagle, to 1. 5 billion Windows 10 and Windows 11 machines worldwide to support Microsoft's on device, end Energy Usage Sensitive Windows Copilot Use Cases and has launched the first updates on RWKV v6, codenamed Finch and Goldfinch.[00:01:53] AI Charlie: On the morning of Latent Space Live, they also announced QRdata UKv6, a QEN32B model [00:02:00] modified with RDWKV linear attention layers. Eugene has also written the most single most popular guest post on the Latent Space blog this year. Yes, we do take guest posts on what he has discovered about the H100 GPU inference NeoCloud market since the successful launch of Featherless AI this year.[00:02:20] AI Charlie: As always, don't forget to check the show notes for the YouTube link to their talk as well as their slides. Watch out and take care.[00:02:27] Intros[00:02:27] Dan Fu: Yeah, so thanks so much for having us. So this is going to be a little bit of a two part presentation. My name is Dan. I'm at Together AI, and I'll be joining UCSD as faculty in about a year. And Eugene, you want to introduce yourself?[00:02:46] Eugene Cheah: Eugene, I lead the art activity team, and I, I'm CEO of Featherless, and we both work on this new post transformer architecture space.[00:02:55] Dan Fu: Yeah, so yeah, so today we're really excited to talk to you a little bit [00:03:00] about that. So first I'm going to give a broad overview of kind of the last few years of progress in non post transformer architectures. And then afterwards Eugene will tell us a little bit about the latest and the greatest and the latest frontier models in this space.[00:03:16] Why Scale Context Lengths? or work on Efficient Models[00:03:16] Dan Fu: So, the story starts with Scaling. So this is probably a figure or something like this that you've seen very recently. Over the last five to six years, we've seen models really scale up in parameter size, and that's brought with it a bunch of new capabilities, like the ability to talk to you and tell you sometimes how to use your Colab screens.[00:03:35] Dan Fu: But another place where we've seen scaling especially recently is scaling in context length. So this can mean Having more text inputs for your models, but it can also mean things like taking a lot of visual token inputs image inputs to your models or generating lots of outputs. And one thing that's been really exciting over the last few months or so is that we're, we're seeing scaling, not only during training time, but also [00:04:00] during test time.[00:04:00] Dan Fu: So this is one of the, the, this is the iconic image from the OpenAI 01 release. Not only are we starting to scale train time compute, but we're also starting to scale test time compute. Now if you're familiar with our attention and our transformer architectures today, this graph on the right might look a little bit scary.[00:04:19] Dan Fu: And one of the reasons is that the implications are a little bit Interesting. So what does it mean if we want to continue having smarter and smarter models? Do we just need to start building bigger, bigger data centers, spending more flops? Is this this little Dolly 3, we need more flops, guys? Is this going to be the future of all of AI?[00:04:39] Dan Fu: Or is there a better way, another path forward? Maybe we can get the same capabilities that we've gotten used to, But for a lot less compute, a lot less flops. And one of the things that we're going to talk about today is specifically looking at that core attention operator in some of these models.[00:04:57] Dan Fu: And the reason is that so this is just some, some [00:05:00] basic you know, scaling curves, but attention has compute that scales quadratically in the context length. So that means that if you're doing something like test time compute and you want to spend a bunch of tokens thinking about what comes next, the longer that that goes the, the, the more tokens you spend on that, that compute grows quadratically in that.[00:05:19] Dan Fu: One of the questions that we're interested in is, can we take that basic sequence model, that basic sequence primitive at the bottom, and get it to scale better? Can we scale in, let's say, n to the 3 halves or n log n? So in, in the first part of the talk, so we just went over the introduction. What I'm gonna do over the next few slides is just talk about some of the key advances and ideas that have shown over the past few years since maybe early 2020 to, to now that shown promise that this might actually be possible.[00:05:48] Dan Fu: That you can actually get potentially the same quality that we want while scale, while scaling better. So to do that, we're and, and basically the, the story that we're gonna look is we're gonna start to see [00:06:00] how. So this is a basic graph of just the past couple years of progress of perplexity where that blue line, that dotted blue line, is attention.[00:06:07] The Story of SSMs[00:06:07] Dan Fu: It's your basic transformer, full dense attention. And then the dots coming down are some of the methods that you'll see in this presentation today. We're going to turn the clock back all the way to 2020. So this, this, this question of can we make attention subquadratic? Basically, as soon as we said attention is all you need, People started asking this question.[00:06:28] Dan Fu: So we have this quadratic attention operator. Can we do better? I'll briefly talk about why attention is quadratic. And the basic thing that happens, if you're not familiar, is that you have these inputs, these keys and queries. And what you do in this attention matrix, this S matrix over here, is that you're using, you're comparing every token in your input to every other token.[00:06:49] Dan Fu: So when I try to do something like upload a whole book to Gemini, what happens beyond the Maybe not Gemini, because we don't necessarily know what architecture is. But let's say we upload it to LLAMA, what happens beyond [00:07:00] the scenes, behind the scenes, is that it's going to take every single word in that book and compare it to every other word.[00:07:05] Dan Fu: And this has been a really, it's, it's led to some pretty impressive things. But it's kind of a brute forcing of the way that you would try to interpret a interpret something. And what attention does in particular is the, and then what attention, sorry, don't want to. Okay, no, no laser pointer. What, what attention does afterwards is that instead of always operating in this quadratic thing, it takes a row wise softmax over this matrix, and then multiplies it by this values matrix.[00:07:32] Dan Fu: So, one of the key points to notice is that the output size is always going to be the same as the inputs, at least in standard self attention. So one of the first things that folks tried to do around 2020 is this thing called linear attention, which is just, just noticing that if we take out this softmax from here, if we take out this non linearity in the middle of the attention operation, and then if you compute the keys and the values operation first, you actually never hit this quadratic bottleneck.[00:07:57] Dan Fu: So that, that's potentially a way [00:08:00] to get a lot more computationally efficient. And there are various ways to do this by basically using feature maps or try to approximate this overall attention computation. But some of this work sort of started to hit a wall in 2020. And the basic challenges were, were two.[00:08:16] Dan Fu: So one was quality. It was back then, it was kind of hard to, to get good quality with these linear attention operators. The other one was actually hardware efficiency. So these, this feature map that was just shown by a simplify simplify here. Actually ends up being quite computationally expensive if you just implement it naively.[00:08:34] Dan Fu: So you started having these operators that not only were you sure, you're not really sure if they have the same quality, but also they're actually just wall clock slower. So you kind of end up getting the worst of both worlds. So this was the the stage. So that kind of sets the stage for four years ago.[00:08:49] Dan Fu: Keep this in mind because linear attention is actually going to come back in a few years once we have a better understanding. But one of the works that started kicking off this, this [00:09:00] mini revolution in post transformer architectures was this idea called states based model. So here the seminal work is, is one about our work queue in 2022.[00:09:09] Dan Fu: And this, this piece of work really brought together a few ideas from, from some long running research research lines of work. The first one was, and this is really one of the keys to, to closing the gap in quality was just using things that, that if you talk to a, a, an electrical engineer off the street, they might know off, off the, like the back of their hand.[00:09:33] Idea 1: Approximation -> Principled Modeling[00:09:33] Dan Fu: But taking some of those properties with how we model dynamical systems in signal processing and then using those ideas to model the inputs, the, the text tokens in, for example a transformer like Next Token Prediction Architecture. So some of those early states-based model papers were looking at this relatively, relatively simple recurrent update model that comes from maybe chapter one of a signal processing class.[00:09:59] Dan Fu: But then using [00:10:00] some principle theory about how you should do that recurrent update in order to really get the most that you can out of your hidden state, out of your out of your sequence. So that, that was one key idea for quality and. When this was eventually realized, you started to see a bunch of benchmarks that were pretty sticky for a few years.[00:10:20] Dan Fu: Things like long range arena, some long sequence evaluation benchmarks, There was stuff in time series, time series analysis. They started to, you started to see the quality tick up in meaningful ways. But the other key thing that What's so influential about these states based models is that they also had a key idea about how you can compute these things efficiently.[00:10:45] Dan Fu: So if you go back to your machine learning 101 class where you learned about RNNs, one thing that you may have learned is that they don't paralyze as well as detention, because if you just run them naively, you have to do this kind of sequential update to process new tokens, [00:11:00] whereas in attention, you can process all the tokens in parallel at one time.[00:11:04] Dan Fu: One of the key insights behind the S4 paper was that these recurrent models, you could take them and you could also formulate them as a convolution. And in particular, with a convolution, you could, instead of using a PyTorch conv1d operation, you can compute that with the FFT. And that would give you n log n compute in the in the sequence length n with an operator that was relatively well optimized for modern hardware.[00:11:28] Dan Fu: So those are really, I'd say, the two key ideas in 2022 that started allowing these breakthroughs to happen in these non transformer architectures. So, these ideas about how to principally model sorry, how to model the recurrent updates of a mo of, of a sequence in a principled way, and also these key ideas in how you can compute it efficiently by turning it into a convolution and then scaling it up with the FFT.[00:11:53] Dan Fu: Along those same lines, so afterwards we started putting out some work on specialized kernels, so just [00:12:00] like we have flash attention for transformers, we also have works like flash fft conf, and if you look at these lines of work oftentimes when, whenever you see a new architecture, you see a new primitive one of the, one of the table stakes now is, do you have an efficient kernel so that you can actually get wall clock speed up?[00:12:14] Idea 3: Selection[00:12:14] Dan Fu: So by 2022, We are starting to have these models that had promising quality primitives, but and, and also promising wall clocks. So you could actually see regimes where they were better than transformers in meaningful ways. That being said, there were, there's still sometimes a quality gap, particularly for language modeling.[00:12:33] Dan Fu: And because languages, It's so core to what we do in sequence modeling these days the, the next, the next key idea that I'm going to talk about is this idea of selection mechanisms. And this is basically an idea of, so you have this recurrent state that you're keeping around that just summarizes everything that, that came before.[00:12:50] Dan Fu: And to get a good sequence model, one of the things that you really need to be able to do is have the model learn what's the best way to pick out pieces from that recurrent [00:13:00] state. So one of the, one of the major ideas here in a line of work called H3, Hungry Hungry Hippos, and also these hyena models were One way you can do this is by just adding some simple element wise gates.[00:13:13] Dan Fu: So versions of these ideas have been around for decades. If you squint at the LSTM paper you, you can probably find, find this gating mechanism. But turns out you can take those old ideas, add them into these new. state space models, and then you can see quality start to pick up. If you've heard of the Mamba model, this also takes the selection to the next level by actually making some changes in that fundamental recurrent state space.[00:13:40] Dan Fu: So, it's not only just this gating that happens around the SSM layer, but also you can actually make The ABCD matrices of your state space model, you can make them data dependent, which will allow you to even better select out different pieces from your hidden state depending on what you're seeing. I'll also point out if you look at the [00:14:00] bottom right of this figure, there's this little triangle with a GPU SRAM, GPU HBM, and this, this is just continuing that trend of when you have a new architecture you, you, you also release it with a kernel to, to, to show that it is hardware efficient, that it, that it can be hardware efficient on modern hardware.[00:14:17] Dan Fu: The, the, one of the next cool things that happened is once we had this understanding of these are the basic pieces, these are the basic principles behind some of the sequence models linear attention actually started to come back. So in earlier this year, there was a model called BASED the, from Simran Arora and, and some other folks, that combined a more principled version of linear attention that basically the, the, the, the two second summary is that it used a Taylor approximation of the softmax attention, combined that with a simple sliding window attention and was starting to able, starting to be able to expand the Pareto frontier of how much data can you recall from your sequence, versus how small is your recurrent state size.[00:14:58] Dan Fu: So those orange dots [00:15:00] are, at the top there, are just showing smaller sequences that can recall more memory.[00:15:07] Just Read Twice[00:15:07] Dan Fu: And the last major idea I think that has been influential in this line of work and is very relatively late breaking just a few months ago, is just the basic idea that when you have these models that are fundamentally more efficient in the sequence length, you maybe don't want to prompt them or use them in exactly the same way.[00:15:26] Dan Fu: So this was a really cool paper called Just Read Twice, also from Simran. That basically said, hey, all these efficient models can process tokens so much more efficiently than transformers that they can sometimes have unfair advantages compared to a simple transformer token. So, or sorry, a simple transformer model.[00:15:44] Dan Fu: So take, for example the standard, the standard use case of you have some long document, you're going to pass it in as input, and then you're going to ask some question about it. One problem you might imagine for a recurrent model where you have a fixed state size is, let's say that [00:16:00] you're. Article is very long, and you're trying to ask about some really niche thing.[00:16:04] Dan Fu: You can imagine it might be hard for the model to know ahead of time what information to put into the hidden state. But these, these, these models are so much more efficient that you can do something really stupid, like, you can just put the document write down the document, write down the question, write down the document again, and then write down the question again, and then this time, the second time that you go over that document, you know exactly what to look for.[00:16:25] Dan Fu: And the cool thing about this is, so this is, And this this results in better quality, especially on these recall intensive tasks. But the other interesting thing is it really takes advantage of the more efficient architectures that, that we're having here. So one of the other, I think, influential ideas in this line of work is if you change the fundamental compute capabilities of your model and the way that it scales, you can actually start to query it at test time differently.[00:16:51] Idea 4: Test Time Compute[00:16:51] Dan Fu: And this actually, of course, goes back to those slides on test time compute. So while everybody's looking at, say, test time compute for big transformer models, [00:17:00] I think potentially a really interesting research question is, how can you take those and how does it change with this new next generation of models?[00:17:09] Dan Fu: So the, I'll just briefly summarize what some of those key ideas were and then talk and then show you briefly kind of what the state of the art is today. So, so the four key ideas are instead of just doing a simple linear attention approximation, instead take ideas that we know from other fields like signal processing, do a more principled approach to your modeling of the sequence.[00:17:32] Idea 2: Hardware & Kernel Support[00:17:32] Dan Fu: Another key idea throughout all these lines of work is you really want. Hardware and kernel support from day one. So, so even if your model is theoretically more efficient if somebody goes and runs it and it's two times slower one of the things that, that we've learned is that if, if you're in that situation, it's, it's just gonna be dead on arrival.[00:17:49] Dan Fu: So you want to be designing your architectures one of the key, key machine learning ideas that has been important for the quality is just making sure that you encode different ways that you can [00:18:00] select from your hidden state and, and really focus on that as a key decider of quality. And finally, I think one of the, the, the emerging new, new things for, for this line of work and something that's quite interesting is, What are the right test time paradigms for these models?[00:18:15] Dan Fu: How do they change relative to relative to what you might do for a standard transformer? I'll briefly end this section. So I've labeled this slide where we are yesterday because Eugene is going to talk about some new models that he released literally this morning. But as of yesterday, some of the really cool results out of the, these efficient alternative models were so AI2 trained this hybrid MOE called Jamba.[00:18:40] Dan Fu: That, that, that seems, that is currently the state of the art for these non transformer architectures. There's this NVIDIA and MIT put out this new diffusion model called SANA recently that one of their key key observations is that you can take a standard diffusion transformer diffusion model, replace the layers with linear [00:19:00] attention, and then that lets you scale to much larger much larger images, much, much Much larger sequences more efficiently.[00:19:07] Dan Fu: And and one thing that I don't think anybody would have called when a few years ago is that one of those gated SSM, gated states based models ended up on the cover of Science because a great group of folks went and trained some DNA models. So that's Michael Polley, Eric Yuen from from Stanford and the Arc Institute.[00:19:26] Dan Fu: So it's, we're really at an exciting time in 2024 where these non transformer, post transformer architectures are showing promise across a wide range. Across a wide range of, of modalities, of applications, and, and of tasks. And with that, I'll pass it on to Eugene, who can tell you a little bit about the latest and greatest with RWKV.[00:19:49] RWKV vs SSMs[00:19:49] Eugene Cheah: So, that's useful? Yeah. You're talking to here. Oh, I'm talking to here. Okay. So, yeah, two streams. Yeah. So, I think one common questions that we tend to get asked, right, is what's the difference between [00:20:00] RWKV and state space? So I think one of the key things to really understand, right the difference between the two groups, right, is that we are actually more like an open source, random internet meets academia kind of situation.[00:20:11] Eugene Cheah: Like, most of us never wrote any paper, but we, we basically look at RNNs and linear intention when intention is all you need came out, and then we decided to like, hey there is a quadratic scaling problem. Why don't we try fixing that instead? So, so, so we end up developing our own branch, but we end up sharing ideas back and forth.[00:20:30] Eugene Cheah: So, and, and we do all this actively in Discord, GitHub, etc. This was so bad for a few years, right, that basically, the average group's H index was so close to zero, right, Illuter. ai actually came in and helped us write our first paper. Great, now our H index is now three, apparently. So, so, so, but, but the thing is, like, a lot of these experiments led to results, and, and, essentially, essentially, we we took the same ideas from linear attention, [00:21:00] and we built on it.[00:21:01] Eugene Cheah: So, to take a step back into, like, how does RWKB handle its own attention mechanic and achieve the same goals of, like, O and compute, respectively, and in focus of our overall goal to make AI accessible to everyone, regardless of language, nation, or compute, that's our goal. We actually train our models primarily on over a hundred languages, which is another topic altogether.[00:21:23] Eugene Cheah: And our goal is to train to even 200 languages to cover all languages in the world. But at the same time, we work on this architecture, To lower the compute cost so that people can run it on Raspberry Pis and on anything. So, how did RWKB break the dependency of LSTM token flow? Because I think to understand architecture, right, it's probably easier to understand it from the RNN lens.[00:21:46] Eugene Cheah: Because that's where we built on. We all, we all state space kind of like try to, try to start anew and took lessons from that and say, So there's a little bit of divergence there. And AKA, this our version of linear attention. So to take step back [00:22:00] all foundation models, be it transformers or non transformers at a very high level, right?[00:22:05] Eugene Cheah: Pumps in the token. I mean, text that things into embeddings and go through a lot of layers. Generate a lot of states where the QKV cache or be iron in states or RW KB states. And outputs and embedding, they are not the same thing. And we just take more layers and more embeddings. And somehow that magically works.[00:22:23] Eugene Cheah: So, if you, if you remember your ancient RNN lessons which we, which we, which we we call best learning these days the general idea is that you have the embedding information flowing all the way up, and when, and you take that information and you flow it back down, and then you process it as part of your LSTM layers.[00:22:41] Eugene Cheah: So, this is how it generally works. Kapati is quoted saying that RNNs are actually unreasonably effective. The problem is this is not scalable. To start doing work on the second token, you need to wait for the first token. And then you need to, and likewise for the third token and fourth token, yada yada.[00:22:55] Eugene Cheah: That is CPU land, not GPU land. So, so, so, you [00:23:00] can have a H100 and you can't even use 1 percent of it. So, so that's kind of why RNNs didn't really take off in the direction that we wanted, like, billions of parameters when it comes to training. So, what did RDAP KV version 0 do? Boom. We just did the dumbest, lamest thing.[00:23:13] Eugene Cheah: Sorry, this is the bottleneck for RNN. We did the dumb thing of removing that line. And it kind of worked. It trained. It sucked, but it kind of worked. Then we were like, hey, then no one cared because the loss was crap, but how do we improve that? And that's essentially where we move forward, because if you see this kind of flow, right, you can actually get your GPU saturated quickly, where it essentially cascades respectively.[00:23:41] Eugene Cheah: So I'm just waiting for this to loop again. So it's like, once you get your first layer, your token to be computed finish. You start to cascade your compute all the way until you are, Hey, I'm using 100 percent of the GPU. So we, we worked on it, and we started going along the principle of that as long as we keep this general architecture [00:24:00] where, where we can cascade and, and be highly efficient with our architecture, nothing is sacred in our architecture.[00:24:06] Eugene Cheah: And we have done some crazy ideas. In fact, you ask us, if you ask me to explain some things in the paper, right, officially in the paper, I'll say we had this idea and we wrote it this way. The reality is someone came with a code, we tested it, it worked, and then we rationalized later. So, so the general[00:24:24] RWKV Arch[00:24:24] Eugene Cheah: The idea behind rwkbr is that we generally have two major blocks that we do.[00:24:30] Eugene Cheah: We call time mix and channel mix. And time mix generally handles handles long term memory states, where essentially, where essentially where we apply the matrix multiplication and Cilu activation functions into processing an input embedding and an output embedding. I'm oversimplifying it because this, This calculation changed every version and we have, like, version 7 right now.[00:24:50] Eugene Cheah: ChannelMix is similar to Base in the sense that it does shorter term attention, where it just looks at the sister token, or the token before it, because [00:25:00] there's a shift in the token shift matrix. I don't really want to go too much into the papers itself, because, like, we do have three papers on this.[00:25:09] Eugene Cheah: Basically, RWKB, RNN for the transformer, ERA, Ego and Pinch, RWKB, Matrix Value State. This is the updated version 5, version 6. And Goldfinch is our, is, is, is, is our hybrid model respectively. We are writing the paper already for V seven and which is, which is for R wk V seven. Called, named Goose, or architectures are named by Bird.[00:25:30] Eugene Cheah: And, I'm going to cover as well, qrwkb, and mama100k, and rwkb, and Where did that lead to? Great! Because we are all GPU poor and to be clear, like, most of this research is done, like, only on a handful H100s, which I had one Google researcher told me that was, like, his experiment budget for a single researcher.[00:25:48] Eugene Cheah: So, our entire organization has less compute than a single researcher in Google. So We, we, one of the things that we explored into was to how do we convert transformer models instead? Because [00:26:00] someone already paid that billion dollars, a million dollars onto training, so why don't we take advantage of those weights?[00:26:05] Eugene Cheah: And, and to, I believe, together AI worked on the lockets for, for the Lambda side of things, and, and we took some ideas from there as well, and we essentially did that for RWKB.[00:26:15] QWRKWv6 launch[00:26:15] Eugene Cheah: And that led to, Q RWKB6, which we just dropped today, a 32 bit instruct preview model, where we took the Quen 32 bit instruct model, freeze the feedforward layer, remove the QKB attention layer, and replace it with RWKB linear layers.[00:26:32] Eugene Cheah: So to be clear, this means we do not have the rwkv channel mix layer, we only have the time mix layer. But but once we do that, we train the rwkv layer. Important is that the feedforward layer needs to be frozen, so the new attention can be learned. And then we unfreeze the feedforward layer, and train all the layers together with a custom learning rate schedule, so that they can learn how to work together.[00:26:54] Eugene Cheah: The end result, surprisingly, And, to be honest, to the frustration of the R. W. [00:27:00] KV MOE team, which ended up releasing the model on the same day, was that, with just a few hours of training on two nodes, we managed to get it to be on par, kind of, with the original QUAN32B model. So, in fact, when the first run, right, that completely confused us, it was like, and I was telling Daniel Goldstein, Smirky, who kind of leads most of our research coordination, When you pitched me this idea, you told me at best you'll get the same level of performance.[00:27:26] Eugene Cheah: You didn't tell me the challenge and score and Winograd score will shoot up. I don't know what's happening there. But it did. MMLU score dropping, that was expected. Because if you think about it, when we were training all the layers, right, we were essentially Like, Frankenstein this thing, and we did brain damage to the feedforward network layer 2 with the new RWKB layers.[00:27:47] Eugene Cheah: But, 76%, hey, somehow it's retained, and we can probably further train this. We didn't even spend more than 3 days training this, so there's a lot more that can be done, hence the preview. This brings up [00:28:00] a big question, because We are already now in the process of converting to 7TB. We are now, this is actually extremely compute efficient to test our attention mechanic.[00:28:10] Eugene Cheah: It's like, it becomes a shortcut. We can, we are already planning to do our version 7 and our hybrid architecture for it. Because we don't need to train from scratch. And we get a really good model out of it. And the other thing that is uncomfortable to say is that because we are doing right now on the 70b is that if this scales correctly to 128k context length, I'm not even talking about a million 128, majority of enterprise workload today is just on 70b at under 32k context length.[00:28:41] Eugene Cheah: That means if this works and the benchmark matches it, It means we can replace the vast majority of current AI workload, unless you want super long context. And then sorry, can someone give us more GPUs? Because we do need the VRAM for super long context, sadly. So yeah, that's what we are working on, and essentially, [00:29:00] we are excited about this to just push it further.[00:29:02] Eugene Cheah: And this conversion process, to be clear, I don't think it's going to be exclusive to RWKB. It probably will work for Mamba as well, I don't see why not. And we will probably see more ideas, or more experiments, or more hybrids, or Yeah, like, one of the weirdest things that I wanted to say outright, and I confirmed this with the Black Mamba team and the Jamba team, which because we did the GoFinch hybrid model, is that none of us understand why a hard hybrid with a state based model to be R.[00:29:28] Eugene Cheah: QA state space and transformer performs better when, than the baseline of both. It's like, it's like when you train one, you expect, and then you replace, you expect the same results. That's our pitch. That's our claim. But somehow when we jam both together, it outperforms both. And that's like one area of emulation that, like, we only have four experiments, plus four teams, that a lot more needs to be done.[00:29:51] Eugene Cheah: But, but these are things that excite me, essentially, because that is what it's potentially we can move ahead for. Which brings us to what comes next.[00:30:00] What's next[00:30:00] [00:30:00][00:30:00] Dan Fu: So, this part is kind of just some, where we'll talk a little bit about stuff that, that we're excited about. Maybe have some wild speculation on, on what, what's, what's coming next.[00:30:12] Dan Fu: And, of course this is also the part that will be more open to questions. So, a couple things that, that I'm excited about is continued hardware model co design for, for these models. So one of the things that we've put out recently is this library called ThunderKittens. It's a CUDA library.[00:30:29] Dan Fu: And one of the things that, that we found frustrating is every time that we built one of these new architectures, and I'm sure you had the exact same experience, we'd have to go and spend two months in CUDA land, like writing these, these new efficient things. And. If we decided to change one thing in PyTorch, like one line of PyTorch code is like a week of CUDA code at least.[00:30:47] Dan Fu: So one of our goals with, with a library like Thunderkitten, so we, we just broke down what are the key principles, what are the key hardware things what are the key, Compute pieces that you get from the hardware. So for example on [00:31:00] H100 everything is really revolves around a warp group matrix multiply operation.[00:31:06] Dan Fu: So you really want your operation to be able to split into relatively small matrix, matrix multiply operations. So like multiplying two 64 by 64 matrices, for example. And so if you know that ahead of time when you're designing your model, that probably gives you you know, some information about how you set the state sizes, how you set the update, how you set the update function.[00:31:27] Dan Fu: So with Thunderkittens we basically built a whole library just around this basic idea that all your basic compute primitives should not be a float, but it should be a matrix, and everything should just be matrix compute. And we've been using that to, to try to both re implement some existing architectures, and also start to design code.[00:31:44] Dan Fu: Some new ones that are really designed with this core with a tensor core primitive in mind. Another thing that that we're, that at least I'm excited about is we, over the last four or five years, we've really been looking at language models as the next thing. But if you've been paying [00:32:00] attention to Twitter there's been a bunch of new next generation models that are coming out.[00:32:04] Dan Fu: So there, there are. So, video generation models that can run real time, that are supported by your mouse and your keyboard, that I'm told if you play with them that, you know, that they only have a few seconds of memory. Can we take that model, can we give it a very long context length so that you could actually maybe generate an entire game state at a time?[00:32:25] Dan Fu: What does that look like for the model? You're certainly not going to do a giant quadratic attention computation to try to run that. Maybe, maybe use some of these new models, or some of these new video generation models that came out. So Sora came out I don't know, two days ago now. But with super long queue times and super long generation times.[00:32:43] Dan Fu: So that's probably a quadratic attention operation at the, at the bottom of it. What if we could remove that and get the same quality, but a lot faster generation time? Or some of the demos that we saw from Paige earlier today. You know, if I have a super long conversation with my [00:33:00] Gemini bot, what if I wanted to remember everything that it's seen in the last week?[00:33:06] Dan Fu: I mean, maybe you don't for personal reasons, but what if I did, you know? What does that mean for the architecture? And I think, you know, that's certainly something I'm pretty excited about. I'm sure you're excited about it too. So, I think we were supposed to have some hot takes, but I honestly don't remember what our hot takes were.[00:33:21] Hot Takes - does anyone really need long context?[00:33:21] Eugene Cheah: Yeah, including the next slide. Hot takes, yes, these are our[00:33:25] Dan Fu: hot takes.[00:33:25] Eugene Cheah: I think the big one on Twitter that we saw, that we shared, was the question is like, is RAG relevant? In the case of, like, the future of, like, state based models?[00:33:38] Dan Fu: Let's see, I haven't played too much with RAG. But when I have. I'll say I found it was a little bit challenging to do research on it because we had this experience over and over again, where you could have any, an embedding model of any quality, so you could have a really, really bad embedding model, or you could have a really, really [00:34:00] good one, By any measure of good.[00:34:03] Dan Fu: And for the final RAG application, it kind of didn't matter. That's what I'll say about RAG while I'm being recorded. I know it doesn't actually answer the question, but[00:34:13] Eugene Cheah: Yeah, so I think a lot of folks are like, extremely excited of the idea of RWKB or State Space potentially having infinite context.[00:34:21] Eugene Cheah: But I think the reality is that when we say infinite context, we just mean a different kind of infinite context, or you, or as it's previously covered, you need to test the model differently. So, think of it more along the lines of the human. Like, I don't remember what I ate for breakfast yesterday.[00:34:37] Eugene Cheah: Yeah, that's the statement that I'll say. And And we humans are not quadratic transformers. If we did, if let's say we increased our brain size for every second we live, we would have exploded by the time we are 5 years old or something like that. And, and I think, I think basically fundamentally for us, right, be it whether we, regardless of whether RWKB, statespace, XLSTM, [00:35:00] etc, our general idea is that instead of that expanding state, that increase in computational cost, what if we have a fixed state size?[00:35:08] Eugene Cheah: And Information theory detects that that fixed state size will have a limit. Just how big of a limit is a question, like, we, like, RWKB is running at 40 megabytes for, for its state. Its future version might run into 400 megabytes. That is like millions of tokens in, if you're talking about mathematically, the maximum possibility.[00:35:29] Eugene Cheah: It's just that I guess we were all more inefficient about it, so maybe we hit 100, 000. And that's kind of like the work we are doing, trying to like push it and maximize it. And that's where the models will start differing, because it will choose to forget things, it will choose to remember things. And that's why I think that there might be some element of right, but it may not be the same right.[00:35:49] Eugene Cheah: It may be the model learn things, and it's like, hmm, I can't remember that, that article. Let me do a database search, to search. Just like us humans, when we can't remember the article in the company. We do a search on Notion. [00:36:00][00:36:00] Dan Fu: I think something that would be really interesting is if you could have facts that are, so right now, the one intuition about language models is that all those parameters are around just to store random facts about the world.[00:36:14] Dan Fu: And this intuition comes from the observation that if you take a really small language model, it can do things like talk to you, or kind of has like the The style of conversation, it can learn that, but where it will usually fall over compared to a much larger one is it'll just be a lot less factual about things that it knows or that it can do.[00:36:32] Dan Fu: But that points to all those weights that we're spending, all that SGD that we're spending to train these models are just being used to store facts. And we have things like databases that are pretty good at storing facts. So I think one thing that would be really interesting is if we could actually have some sort of outside data store that a language model can can look at that that maybe is you know, has has some sort of gradient descent in it, but but would be quite interesting.[00:36:58] Dan Fu: And then maybe you could edit it, delete [00:37:00] facts, you know, change who's president so that it doesn't, it doesn't get lost.[00:37:04] Vibhu: Can we open up Q& A and hot takes for the audience? I have a hot take Q& A. Do these scale? When, when 405B state space model, RAG exists, no one does long context, who's throwing in 2 million token questions, hot takes?[00:37:24] Dan Fu: The, the who's throwing in 2 million token question, I think, is, is a really good question. So I actually, I was going to offer that as a hot take. I mean, my hot take was going to be that long context doesn't matter. I know I just gave a whole talk about it, but you know, what, what's the point of doing research if you can't, you know, play both sides.[00:37:40] Dan Fu: But I think one of the, so I think for both of us, the reason that we first got into this was just from the first principled questions of there's this quadratic thing. Clearly intelligence doesn't need to be quadratic. What is going on? Can we understand it better? You know, since then it's kind of turned into a race, which has [00:38:00] been exciting to watch, like, how much context you can take in.[00:38:03] Dan Fu: But I think it's right. Nobody is actually putting in a two million context prompt into these models. And, and, you know, if they are, maybe we can go, go You know, design a better model to do that particular thing. Yeah, what do you think about that? So you've also been working on this. Do you think long context matters?[00:38:19] Eugene Cheah: So I'm going to burn a bit. How many of you remember the news of Google Gemini supporting 3 million contacts, right? Raise your hand.[00:38:28] Vibhu: Yeah, 2 million.[00:38:29] Eugene Cheah: Oh, it's 2 million.[00:38:31] Eugene Cheah: Yeah, how many of you actually tried that? See?[00:38:34] Vibhu: I use it a lot. You? You work for MindsTV. I use it a lot.[00:38:41] Eugene Cheah: So, for some people that has used, and I think, I think that's the, that's might be, like, this is where my opinion starts to differ, because I think the big labs may have a bigger role in this, because Like, even for RWKB, even when we train non contacts, the reason why I say VRAM is a problem is that because when we did the, we need to backprop [00:39:00] against the states, we actually need to maintain the state in between the tokens by the token length.[00:39:05] Eugene Cheah: So that means we need to actually roll out the whole 1 million contacts if we are actually training 1 million. Which is the same for transformers, actually, but it just means we don't magically reuse the VRAM consumption in the training time space. So that is one of the VRAM bottlenecks, and I'm neither OpenAI nor Google, so donate GPUs if you have too much of them.[00:39:27] Eugene Cheah: But then, putting it back to another paradigm, right, is that I think O1 style reasoning might be actually pushing that direction downwards. In my opinion, this is my partial hot take is that if, let's say you have a super big model, And let's say you have a 70B model that may take double the tokens, but gets the same result.[00:39:51] Eugene Cheah: Strictly speaking, a 70B, and this is even for transformer or non transformer, right? We we'll take less less resources than that 400 B [00:40:00] model, even if it did double the amount thinking. And if that's the case, and we are still all trying to figure this out, maybe the direction for us is really getting the sub 200 B to be as fast as efficient as possible.[00:40:11] Eugene Cheah: We a very efficient architecture that some folks happen to be working on to, to just reason it out over larger and larger context thing.[00:40:20] Question: Yeah. One thing I'm super interested in is. Models that can watch forever? Obviously you cannot train something on infinite context length. How are y'all thinking about that, where you run on a much longer context length than is possible to train on?[00:40:38] Dan Fu: Yeah, it's a, it's a great question. So I think when I think you guys probably had tweets along these lines, too. When we first started doing these things, because these are all recurrent models in theory you could just run it forever. You could just run it forever. And at the very least it won't, it won't like error out on your crash.[00:40:57] Dan Fu: There's another question of whether it can actually [00:41:00] use what it's seen in that infinite context. And I think there, so one place where probably the research and architectures ran faster Then another research is actually the benchmarks for long context. So you turn it on forever. You want to do everything or watch everything.[00:41:16] Dan Fu: What is it that you actually wanted to do? Can we actually build some benchmarks for that? Then measure what's happening. And then ask the question, can the models do it? Is there something else that they need? Yeah, I think that if I were to turn back the clock to 2022, that's probably one of the things I would have done differently, which would have been actually get some long context benchmarks out at the same time as we started pushing context length on all these models.[00:41:41] Eugene Cheah: I will also say the use case. So like, I think we both agree that there's no Infinite memory and the model needs to be able to learn and decide. I think what we have observed for, I think this also fits the state space model, is that one of the key advantages of this alternate attention mechanic that is not based on token position is that the model don't suddenly become crazy when you go past the [00:42:00] 8k training context tank, or a million context tank.[00:42:03] Eugene Cheah: It's actually still stable. It's still able to run, it's still able to rationalize. It just starts forgetting things. But some of these things are still there in latent memory. Some of these things are still somewhat there. That's the whole point of why reading twice works. Things like that. And one of the biggest pushes in this direction is that I think both Statespace and RWKB have Separate papers by other researchers where they use this architecture for time series data.[00:42:26] Eugene Cheah: Weather modeling. So, you are not asking what was the weather five days ago. You're asking what's the weather tomorrow based on the infinite length that we, as long as this Earth and the computer will keep running. So, so, and they found that it is like, better than existing, like, transformer or existing architecture in modeling this weather data.[00:42:47] Eugene Cheah: Control for the param size and stuff. I'm quite sure there are people with larger models. So, so there are things that, that in this case, right, there is future applications if your question is just what's next and not what's 10 years ago.[00:42:59] Dan Fu: Thanks so [00:43:00] much for having us. Get full access to Latent Space at www.latent.space/subscribe

Guy and Eitan go over the most popular sessions released from the SQL PASS archive of 2023, and discuss some other topics. Relevant links: Announcing Microsoft SQL Server 2025: Enterprise AI-ready database from ground to cloud - Microsoft SQL Server Blog Sessions Archive for 2023 - PASS Data Community Summit Resolving and Preventing Deadlocks in SQL Server SSMS v21: Dark Mode, Git, Vertical Tabs, and More - Brent Ozar Unlimited® Copilot in SSMS is Kinda Like IE in SSMS - Brent Ozar Unlimited®

I've been very pleased with the direction of SSMS the last few years. As it's been separated from SQL Server releases and gets updated more often, I think the changes from v17 though v20 have been improvements. There are still issues, but it's been better. Now we finally have SSMS moving to a modern shell with the v21 preview and I'm excited to see how this changes the future of our tooling. However, the PM for SSMS, Erin Stellato, posted a note on LinkedIn recently asking why people don't read documentation. She also asks what you want to see in 21, so respond if you think there are holes in the SSMS docs. I think this post came about because of many responses that came from people who clearly hadn't read some documentation. Read the rest of Doing a Little Research

What's happening with SQL Server Management Studio? Richard chats with Erin Stellato, now at Microsoft, about the big jump coming for SSMS. Erin talks about how folks felt SSMS was a bit neglected when the reality is that there was a push to catch up with its parent codebase in Visual Studio. However, the next version of SSMS makes that jump, which opens the door to some excellent extension models. The conversation dives into the role of the Copilots in SQL Server through SSMS - helping you understand databases, write queries, and diagnose problems - eventually!LinksSQL Server Management StudioAzure SQL DatabaseSQL Server Integration ServicesSQL Server Data ToolsSQL FormatterRecorded September 26, 2024

Hey folks, Alex here from Weights & Biases, and this week has been absolutely bonkers. From robots walking among us to rockets landing on chopsticks (well, almost), the future is feeling palpably closer. And if real-world robots and reusable spaceship boosters weren't enough, the open-source AI community has been cooking, dropping new models and techniques faster than a Starship launch. So buckle up, grab your space helmet and noise-canceling headphones (we'll get to why those are important!), and let's blast off into this week's AI adventures!TL;DR and show-notes + links at the end of the post

On my new laptop, I only use containers as database servers. I made the decision not to install SQL Server or PostgreSQL and instead work on containers only. I've written lightly about this, but I set up docker-compose files to load different instances of SQL Server and PostgreSQL (and others) and batch files to start and stop them. I've also set dedicated places on my disk where I can drop backup files and access them from the host. It's 2024. I moved to containers on my laptop exclusively for databases for the first time this year. This is despite the fact that I like containers, am comfortable with them, and find them handy. Moving from installed database server software to containers took a conscious effort, and it took time to configure everything. Really, it took me a bit of time to think about how I'd want to configure my system so that my work in SSMS went smoothly. Read the rest of Container Development Work

Nathan hosts Albert Gu, assistant professor at CMU and co-founder of Cartesia AI, to discuss the groundbreaking Mamba architecture. In this episode of The Cognitive Revolution, we explore the state space model revolution, diving into the technical details of Mamba and Mamba 2. Join us for an insightful conversation on the future of AI architectures and their potential to transform the field. Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj RECOMMENDED PODCAST: Byrne Hobart, the writer of The Diff, is revered in Silicon Valley. You can get an hour with him each week. See for yourself how his thinking can upgrade yours. Spotify: https://open.spotify.com/show/6rANlV54GCARLgMOtpkzKt Apple: https://podcasts.apple.com/us/podcast/the-riff-with-byrne-hobart-and-erik-torenberg/id1716646486 SPONSORS: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist. CHAPTERS: (00:00:00) About the Show (00:05:39) State Space Models (00:13:05) Intuition and inspiration (00:18:27) Surprises (00:22:33) Sponsors: Oracle | Brave (00:24:41) Biological inspiration (00:25:19) MAMBA breakthrough (00:30:59) How does the state work? (00:36:44) What is the size of the state? (00:39:05) Training vs. Inference (Part 1) (00:42:04) Sponsors: Omneky | Squad (00:43:51) Training vs. Inference (Part 2) (00:43:51) Sequence Models (00:49:20) Mamba inference (00:57:53) Mamba2 vs Mamba1 (01:16:05) Overtraining and the future of SSMs (01:17:44) Training efficiency vs inference efficiency (01:20:52) Hybrid models (01:25:04) Scaling Attention Layers (01:30:23) Optimizing State (01:34:09) The extrapolation abilities of the SSMs (01:36:37) Sequence parallelism with Mamba 2 (01:39:20) Why are you publishing all this? (01:40:46) Cartesia and Together (01:41:54) Outro

This week on No Priors, Sarah Guo and Elad Gil sit down with Karan Goel and Albert Gu from Cartesia. Karan and Albert first met as Stanford AI Lab PhDs, where their lab invented Space Models or SSMs, a fundamental new primitive for training large-scale foundation models. In 2023, they Founded Cartesia to build real-time intelligence for every device. One year later, Cartesia released Sonic which generates high quality and lifelike speech with a model latency of 135ms—the fastest for a model of this class. Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @krandiash | @_albertgu Show Notes:  (0:00) Introduction (0:28) Use Cases for Cartesia and Sonic  (1:32) Karan Goel & Albert Gu's professional backgrounds (5:06) Steady State Models (SSMs) versus Transformer Based Architectures  (11:51) Domain Applications for Hybrid Approaches  (13:10) Text to Speech and Voice (17:29) Data, Size of Models and Efficiency  (20:34) Recent Launch of Text to Speech Product (25:01) Multimodality & Building Blocks (25:54) What's Next at Cartesia?  (28:28) Latency in Text to Speech (29:30) Choosing Research Problems Based on Aesthetic  (31:23) Product Demo (32:48) Cartesia Team & Hiring

Hey everyone, Alex here! Can you believe it's already end of May? And that 2 huge AI companies conferences are behind us (Google IO, MSFT Build) and Apple's WWDC is just ahead in 10 days! Exciting! I was really looking forward to today's show, had quite a few guests today, I'll add all their socials below the TL;DR so please give them a follow and if you're only in reading mode of the newsletter, why don't you give the podcast a try

Speakers for AI Engineer World's Fair have been announced! See our Microsoft episode for more info and buy now with code LATENTSPACE — we've been studying the best ML research conferences so we can make the best AI industry conf! Note that this year there are 4 main tracks per day and dozens of workshops/expo sessions; the free livestream will air much less than half of the content this time.Apply for free/discounted Diversity Program and Scholarship tickets here. We hope to make this the definitive technical conference for ALL AI engineers.ICLR 2024 took place from May 6-11 in Vienna, Austria. Just like we did for our extremely popular NeurIPS 2023 coverage, we decided to pay the $900 ticket (thanks to all of you paying supporters!) and brave the 18 hour flight and 5 day grind to go on behalf of all of you. We now present the results of that work!This ICLR was the biggest one by far, with a marked change in the excitement trajectory for the conference:Of the 2260 accepted papers (31% acceptance rate), of the subset of those relevant to our shortlist of AI Engineering Topics, we found many, many LLM reasoning and agent related papers, which we will cover in the next episode. We will spend this episode with 14 papers covering other relevant ICLR topics, as below.As we did last year, we'll start with the Best Paper Awards. Unlike last year, we now group our paper selections by subjective topic area, and mix in both Outstanding Paper talks as well as editorially selected poster sessions. Where we were able to do a poster session interview, please scroll to the relevant show notes for images of their poster for discussion. To cap things off, Chris Ré's spot from last year now goes to Sasha Rush for the obligatory last word on the development and applications of State Space Models.We had a blast at ICLR 2024 and you can bet that we'll be back in 2025

Adobe is introducing new AI-powered tools to their video editing software, including the ability to extend video clips, add or remove objects from scenes, and generate B-roll footage using prompts. Amazon's Bedrock platform is adding all three versions of Anthropic's Claude 3 AI model, enhancing the ability of customers to rapidly test, build, and deploy generative AI applications across their organizations. "The Illusion of State in State-Space Models" challenges the assumption that SSMs are inherently better at state tracking than transformers. "Megalodon" proposes a new neural architecture for efficient sequence modeling, allowing for unlimited context length and better efficiency than Transformers. Contact:  sergi@earkind.com Timestamps: 00:34 Introduction 01:35 Adobe previews AI video features 02:56 Amazon Puts All Three Claude AI Models on Bedrock 05:07 Automating Complex Business Workflows with Cohere: Multi-Step Tool Use in Action 07:02 Fake sponsor 09:02 The Illusion of State in State-Space Models 10:58 Generative Information Retrieval Evaluation 12:49 Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length 14:26 Outro

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ophiology (or, how the Mamba architecture works), published by Danielle Ensign on April 9, 2024 on LessWrong. The following post was made as part of Danielle's MATS work on doing circuit-based mech interp on Mamba, mentored by Adrià Garriga-Alonso. It's the first in a sequence of posts about finding an IOI circuit in Mamba/applying ACDC to Mamba. This introductory post was also made in collaboration with Gonçalo Paulo. A new challenger arrives! Why Mamba? Promising Scaling Mamba [1] is a type of recurrent neural network based on state-space models, and is being proposed as an alternative architecture to transformers. It is the result of years of capability research [2] [3] [4] and likely not the final iteration of architectures based on state-space models. In its current form, Mamba has been scaled up to 2.8B parameters on The Pile and on Slimpj, having similar scaling laws when compared to Llama-like architectures. Scaling curves from Mamba paper: Mamba scaling compared to Llama (Transformer++), previous state space models (S3++), convolutions (Hyena), and a transformer inspired RNN (RWKV) More recently, ai21labs [5] trained a 52B parameter MOE Mamba-Transformer hybrid called Jamba. At inference, this model has 12B active parameters and has benchmark scores comparable to Llama-2 70B and Mixtral. Jamba benchmark scores, from Jamba paper [5:1] Efficient Inference One advantage of RNNs, and in particular of Mamba, is that the memory required to store the context length is constant, as you only need to store the past state of the SSM and of the convolution layers, while it grows linearly for transformers. The same happens with the generation time, where predicting each token scales as O(1) instead of O(context length). Jamba throughput (tokens/second), from Jamba paper[5:2] What are State-space models? The inspiration for Mamba (and similar models) is an established technique used in control theory called state space models (SSM). SSMs are normally used to represent linear systems that have p inputs, q outputs and n state variables. To keep the notation concise, we will consider the input as E-dimensional vector x(t)RE, an E-dimensional output y(t)RE and a N-dimensional latent space hRN. In the following, we will note the dimensions of new variables using the notation [X,Y]. In particular, in Mamba 2.8b, E=5120 and N=16. Specifically, we have the following: [N]h(t)=[N,N]A[N]h(t)+[N,E]B[E]x(t) [E]y(t)=[E,N]C[N]h(t)+[E,E]D[E]x(t) This is an ordinary differential equation (ODE), where h(t) is the derivative of h(t) with respect to time, t. This ODE can be solved in various ways, which will be described below. In state space models, A is called the state matrix, B is called the input matrix, C is called the output matrix, and D is called the feedthrough matrix. Solving the ODE We can write the ODE from above as a recurrence, using discrete timesteps: [N]ht=[N,N]A[N]ht1+[N,E]B[E]xt [E]yt=[E,N]C[N]ht+[E,E]D[E]xt where A and B are our discretization matrices. Different ways of integrating the original ODE will give different A and B, but will still preserve this overall form. In the above, t corresponds to discrete time. In language modeling, t refers to the token position. Euler method The simplest way to numerically integrate an ODE is by using the Euler method, which consists in approximating the derivative by considering the ratio between a small variation in h and a small variation in time, h=dhdtΔhΔt. This allows us to write: ht+1htΔt=Aht+Bxt ht+1=Δt(Aht+Bxt)+ht Where the index t, of ht, represents the discretized time. This is the same thing that is done when considering a character's position and velocity in a video game, for instance. If a character has a velocity v and a position x0, to find the position after Δt time we can do x1=Δtv+x0. In general: xt=Δtvt+xt1 xt=(...

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ophiology (or, how the Mamba architecture works), published by Danielle Ensign on April 9, 2024 on LessWrong. The following post was made as part of Danielle's MATS work on doing circuit-based mech interp on Mamba, mentored by Adrià Garriga-Alonso. It's the first in a sequence of posts about finding an IOI circuit in Mamba/applying ACDC to Mamba. This introductory post was also made in collaboration with Gonçalo Paulo. A new challenger arrives! Why Mamba? Promising Scaling Mamba [1] is a type of recurrent neural network based on state-space models, and is being proposed as an alternative architecture to transformers. It is the result of years of capability research [2] [3] [4] and likely not the final iteration of architectures based on state-space models. In its current form, Mamba has been scaled up to 2.8B parameters on The Pile and on Slimpj, having similar scaling laws when compared to Llama-like architectures. Scaling curves from Mamba paper: Mamba scaling compared to Llama (Transformer++), previous state space models (S3++), convolutions (Hyena), and a transformer inspired RNN (RWKV) More recently, ai21labs [5] trained a 52B parameter MOE Mamba-Transformer hybrid called Jamba. At inference, this model has 12B active parameters and has benchmark scores comparable to Llama-2 70B and Mixtral. Jamba benchmark scores, from Jamba paper [5:1] Efficient Inference One advantage of RNNs, and in particular of Mamba, is that the memory required to store the context length is constant, as you only need to store the past state of the SSM and of the convolution layers, while it grows linearly for transformers. The same happens with the generation time, where predicting each token scales as O(1) instead of O(context length). Jamba throughput (tokens/second), from Jamba paper[5:2] What are State-space models? The inspiration for Mamba (and similar models) is an established technique used in control theory called state space models (SSM). SSMs are normally used to represent linear systems that have p inputs, q outputs and n state variables. To keep the notation concise, we will consider the input as E-dimensional vector x(t)RE, an E-dimensional output y(t)RE and a N-dimensional latent space hRN. In the following, we will note the dimensions of new variables using the notation [X,Y]. In particular, in Mamba 2.8b, E=5120 and N=16. Specifically, we have the following: [N]h(t)=[N,N]A[N]h(t)+[N,E]B[E]x(t) [E]y(t)=[E,N]C[N]h(t)+[E,E]D[E]x(t) This is an ordinary differential equation (ODE), where h(t) is the derivative of h(t) with respect to time, t. This ODE can be solved in various ways, which will be described below. In state space models, A is called the state matrix, B is called the input matrix, C is called the output matrix, and D is called the feedthrough matrix. Solving the ODE We can write the ODE from above as a recurrence, using discrete timesteps: [N]ht=[N,N]A[N]ht1+[N,E]B[E]xt [E]yt=[E,N]C[N]ht+[E,E]D[E]xt where A and B are our discretization matrices. Different ways of integrating the original ODE will give different A and B, but will still preserve this overall form. In the above, t corresponds to discrete time. In language modeling, t refers to the token position. Euler method The simplest way to numerically integrate an ODE is by using the Euler method, which consists in approximating the derivative by considering the ratio between a small variation in h and a small variation in time, h=dhdtΔhΔt. This allows us to write: ht+1htΔt=Aht+Bxt ht+1=Δt(Aht+Bxt)+ht Where the index t, of ht, represents the discretized time. This is the same thing that is done when considering a character's position and velocity in a video game, for instance. If a character has a velocity v and a position x0, to find the position after Δt time we can do x1=Δtv+x0. In general: xt=Δtvt+xt1 xt=(...

In this episode Michael, Sarah and Mark talk with guest Ryan Munsch about the newly released Copilot for Security. We also discuss Azure Security news about Azure SQL DB, SSMS 20, Change Actor, Copilot for Azure SQL DB, Azure Container Apps, AI Prompt Shields, AI Groundedness Detection and BlueHat India and Israel.New tab (azsecuritypodcast.net)

In this second part of a two episode series, Nathan and AI scout Jason Meaux provide a sweeping overview of the first 90 days of Mamba-inspired research. They discusstMamba's application to computer vision, experiments in extending effective context length, the potential problem of internal rotting states, and the use of hybrid SSMs in biology.Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api LINKS: Show Notes and Paper Links: https://docs.google.com/document/d/1NK_a3deVL_aczORmSRw8LyujNPotpO7Kd90sIRj9Qx0/edit?usp=sharing Nathan's original Mamba Deep Dive Spotify: https://open.spotify.com/episode/3b3zeDVU6fE2yV9vdNVLqX?si=dc7a885d601d4a58 Apple: https://podcasts.apple.com/us/podcast/emergency-pod-mamba-memory-and-the-ssm-moment/id1669813431?i=1000639397566 X/SOCIAL: @labenz (Nathan) SPONSORS: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, instead of...does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api ODF is where top founders get their start. Apply to join the next cohort and go from idea to conviction-fast. ODF has helped over 1000 companies like Traba, Levels and Finch get their start. Is it your turn? Go to http://beondeck.com/revolution to learn more. TIMESTAMPS: (00:00) - Episode Start (00:01:14) - Nathan and Jason's Bet (00:04:51) - U-Mamba (00:15:10) - Sponsors: Oracle | Omneky (00:06:08) - Swin U-Mamba (00:18:59) - Vision Mamba (00:24:30 - VMamba (00:27:26) - VM-UNet (00:30:37) - Sponsors: Brave | On Deck (00:35:15) - SegMamba (00:36:59) - Vivim (00:37:31) - Mamba Morph (00:39:01) - Graph Mamba  (00:43:15) - Long Mamba (00:59:54) - Evo and StripedHyena The Cognitive Revolution is produced by Turpentine: a media network covering technology, business, and culture. Producer: Vivian Meng Editor: Graham Bessellieu For sponsor or guest inquiries, email: vivian@turpentine.co 

In episode 113 of The Gradient Podcast, Daniel Bashir speaks to Professor Sasha Rush.Professor Rush is an Associate Professor at Cornell University and a Researcher at HuggingFace. His research aims to develop natural language processing systems that are safe, fast, and controllable. His group is interested primarily in tasks that involve text generation, and they study data-driven probabilistic methods that combine deep-learning based models with probabilistic controls. He is also interested in open-source NLP and deep learning, and develops projects to make deep learning systems safer, clearer, and easier to use.Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pubSubscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on TwitterOutline:* (00:00) Intro* (01:47) Professor Rush's background* (03:23) Professor Rush's reflections on prior work—importance of learning and inference* (04:58) How much engineering matters in deep learning, the Rush vs. Frankle Bet* (07:12) On encouraging and incubating good research* (10:50) Features of good research environments* (12:36) 5% bets in Professor Rush's research: State-Space Models (SSMs) as an alternative to Transformers* (15:58) SSMs vs. Transformers* (18:53) Probabilistic Context-Free Grammars—are (P)CFGs worth paying attention to?* (20:53) Sequence-level knowledge distillation: approximating sequence-level distributions* (25:08) Pruning and knowledge distillation — orthogonality of efficiency techniques* (26:33) Broader thoughts on efficiency* (28:31) Works on prompting* (28:58) Prompting and In-Context Learning* (30:05) Thoughts on mechanistic interpretability* (31:25) Multitask prompted training enables zero-shot task generalization* (33:48) How many data points is a prompt worth? * (35:13) Directions for controllability in LLMs* (39:11) Controllability and safety* (41:23) Open-source work, deep learning libraries* (42:08) A story about Professor Rush's post-doc at FAIR* (43:51) The impact of PyTorch* (46:08) More thoughts on deep learning libraries* (48:48) Levels of abstraction, PyTorch as an interface to motivate research* (50:23) Empiricism and research commitments* (53:32) OutroLinks:* Research* Early work / PhD* Dual Decomposition and LP Relaxations* Vine Pruning for Efficient Multi-Pass Dependency Parsing* Improved Parsing and POS Tagging Using Inter-Sentence Dependency Constraints* Research — interpretable and controllable natural language generation* Compound Probabilistic Context-Free Grammars for Grammar Induction* Multitask prompted training enables zero-shot task generalization* Research — deep generative models* A Neural Attention Model for Abstractive Sentence Summarization* Learning Neural Templates for Text Generation* How many data points is a prompt worth?* Research — efficient algorithms and hardware for speech, translation, dialogue* Sequence-Level Knowledge Distillation* Open-source work* NamedTensor* Torch Struct Get full access to The Gradient at thegradientpub.substack.com/subscribe

I got a message recently that SSM S19.3 is out. I am wary of major versions, especially with a few add-in tools, but I have tended to try and update SSMS regularly when it patches, which is about once a quarter. As I checked my desktop, I saw I was still on 19.1 (my laptop was 19.2), so I downloaded and updated both machines. I wonder what the rest of you do. I know many of you have corporate rules and restrictions and you may not be able to update regularly, but in many places, I've worked, once software was installed, I could update it. Are you doing the same thing out there? Let us know in the comments, or better yet, include the version you are running. Read the rest of How Often Do You Update SSMS?

Recently the state space models (SSMs) with efficient hardware-aware designs, i.e., Mamba, have shown great potential for long sequence modeling. Building efficient and generic vision backbones purely upon SSMs is an appealing direction. However, representing visual data is challenging for SSMs due to the position-sensitivity of visual data and the requirement of global context for visual understanding. In this paper, we show that the reliance of visual representation learning on self-attention is not necessary and propose a new generic vision backbone with bidirectional Mamba blocks (Vim), which marks the image sequences with position embeddings and compresses the visual representation with bidirectional state space models. On ImageNet classification, COCO object detection, and ADE20k semantic segmentation tasks, Vim achieves higher performance compared to well-established vision transformers like DeiT, while also demonstrating significantly improved computation&memory efficiency. For example, Vim is 2.8$times$ faster than DeiT and saves 86.8% GPU memory when performing batch inference to extract features on images with a resolution of 1248$times$1248. The results demonstrate that Vim is capable of overcoming the computation&memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to become the next-generation backbone for vision foundation models. Code is available at https://github.com/hustvl/Vim. 2024: Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, Xinggang Wang https://arxiv.org/pdf/2401.09417v1.pdf

State Space Models (SSMs) have become serious contenders in the field of sequential modeling, challenging the dominance of Transformers. At the same time, Mixture of Experts (MoE) has significantly improved Transformer-based LLMs, including recent state-of-the-art open-source models. We propose that to unlock the potential of SSMs for scaling, they should be combined with MoE. We showcase this on Mamba, a recent SSM-based model that achieves remarkable, Transformer-like performance. Our model, MoE-Mamba, outperforms both Mamba and Transformer-MoE. In particular, MoE-Mamba reaches the same performance as Mamba in 2.2x less training steps while preserving the inference performance gains of Mamba against the Transformer. 2024: Maciej Pi'oro, Kamil Ciebiera, Krystian Kr'ol, Jan Ludziejewski, Sebastian Jaszczur https://arxiv.org/pdf/2401.04081.pdf

We are running an end of year listener survey! Please let us know any feedback you have, what episodes resonated with you, and guest requests for 2024! Survey link here.NeurIPS 2023 took place from Dec 10–16 in New Orleans. The Latent Space crew was onsite for as many of the talks and workshops as we could attend (and more importantly, hosted cocktails and parties after hours)!Picking from the 3586 papers accepted to the conference (available online, full schedule here) is an impossible task, but we did our best to present an audio guide with brief commentary on each. We also recommend MLContests.com NeurIPS recap and Seb Ruder's NeurIPS primer. We also found the VizHub guide useful for a t-SNE clustering of papers.We'll start with the NeurIPS Best Paper Awards, and then go to a selection of non-awarded but highly influential papers, and then arbitrary personal picks to round out the selection. Where we were able to do a poster session interview, please scroll to the relevant show notes for images of their poster for discussion. We give Chris Ré the last word due to the Mamba and StripedHyena state space models drawing particular excitement but still being too early to assess impact. Timestamps* [0:01:19] Word2Vec (Jeff Dean, Greg Corrado)* [0:15:28] Emergence Mirage (Rylan Schaeffer)* [0:28:48] DPO (Rafael Rafailov)* [0:41:36] DPO Poster Session (Archit Sharma)* [0:52:03] Datablations (Niklas Muennighoff)* [1:00:50] QLoRA (Tim Dettmers)* [1:12:23] DataComp (Samir Gadre)* [1:25:38] DataComp Poster Session (Samir Gadre, Alex Dimakis)* [1:35:25] LLaVA (Haotian Liu)* [1:47:21] LLaVA Poster Session (Haotian Liu)* [1:59:19] Tree of Thought (Shunyu Yao)* [2:11:27] Tree of Thought Poster Session (Shunyu Yao)* [2:20:09] Toolformer (Jane Dwivedi-Yu)* [2:32:26] Voyager (Guanzhi Wang)* [2:45:14] CogEval (Ida Momennejad)* [2:59:41] State Space Models (Chris Ré)Papers covered* Distributed Representations of Words and Phrases and their Compositionality (Word2Vec) Tomas Mikolov · Ilya Sutskever · Kai Chen · Greg Corrado · Jeff Dean. The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several improvements that make the Skip-gram model more expressive and enable it to learn higher quality vectors more rapidly. We show that by subsampling frequent words we obtain significant speedup, and also learn higher quality representations as measured by our tasks. We also introduce Negative Sampling, a simplified variant of Noise Contrastive Estimation (NCE) that learns more accurate vectors for frequent words compared to the hierarchical softmax. An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of Canada'' and "Air'' cannot be easily combined to obtain "Air Canada''. Motivated by this example, we present a simple and efficient method for finding phrases, and show that their vector representations can be accurately learned by the Skip-gram model.* Are Emergent Abilities of Large Language Models a Mirage? (Schaeffer et al.). Emergent abilities are abilities that are present in large-scale models but not in smaller models and are hard to predict. Rather than being a product of models' scaling behavior, this paper argues that emergent abilities are mainly an artifact of the choice of metric used to evaluate them. Specifically, nonlinear and discontinuous metrics can lead to sharp and unpredictable changes in model performance. Indeed, the authors find that when accuracy is changed to a continuous metric for arithmetic tasks where emergent behavior was previously observed, performance improves smoothly instead. So while emergent abilities may still exist, they should be properly controlled and researchers should consider how the chosen metric interacts with the model.* Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al.)* While large-scale unsupervised language models (LMs) learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult due to the completely unsupervised nature of their training. Existing methods for gaining such steerability collect human labels of the relative quality of model generations and fine-tune the unsupervised LM to align with these preferences, often with reinforcement learning from human feedback (RLHF). However, RLHF is a complex and often unstable procedure, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model. * In this paper, we leverage a mapping between reward functions and optimal policies to show that this constrained reward maximization problem can be optimized exactly with a single stage of policy training, essentially solving a classification problem on the human preference data. The resulting algorithm, which we call Direct Preference Optimization (DPO), is stable, performant, and computationally lightweight, eliminating the need for fitting a reward model, sampling from the LM during fine-tuning, or performing significant hyperparameter tuning. * Our experiments show that DPO can fine-tune LMs to align with human preferences as well as or better than existing methods. Notably, fine-tuning with DPO exceeds RLHF's ability to control sentiment of generations and improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train.* Scaling Data-Constrained Language Models (Muennighoff et al.)* The current trend of scaling language models involves increasing both parameter count and training dataset size. Extrapolating this trend suggests that training dataset size may soon be limited by the amount of text data available on the internet. Motivated by this limit, we investigate scaling language models in data-constrained regimes. Specifically, we run a large set of experiments varying the extent of data repetition and compute budget, ranging up to 900 billion training tokens and 9 billion parameter models. We find that with constrained data for a fixed compute budget, training with up to 4 epochs of repeated data yields negligible changes to loss compared to having unique data. However, with more repetition, the value of adding compute eventually decays to zero. We propose and empirically validate a scaling law for compute optimality that accounts for the decreasing value of repeated tokens and excess parameters. Finally, we experiment with approaches mitigating data scarcity, including augmenting the training dataset with code data or removing commonly used filters. Models and datasets from our 400 training runs are freely available at https://github.com/huggingface/datablations.* QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al.). * This paper proposes QLoRA, a more memory-efficient (but slower) version of LoRA that uses several optimization tricks to save memory. They train a new model, Guanaco, that is fine-tuned only on a single GPU for 24h and outperforms previous models on the Vicuna benchmark. Overall, QLoRA enables using much fewer GPU memory for fine-tuning LLMs. Concurrently, other methods such as 4-bit LoRA quantization have been developed that achieve similar results.* DataComp: In search of the next generation of multimodal datasets (Gadre et al.)* Multimodal datasets are a critical component in recent breakthroughs such as CLIP, Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model architectures or training algorithms. To address this shortcoming in the machine learning ecosystem, we introduce DataComp, a testbed for dataset experiments centered around a new candidate pool of 12.8 billion image-text pairs from Common Crawl. Participants in our benchmark design new filtering techniques or curate new data sources and then evaluate their new dataset by running our standardized CLIP training code and testing the resulting model on 38 downstream test sets. * Our benchmark consists of multiple compute scales spanning four orders of magnitude, which enables the study of scaling trends and makes the benchmark accessible to researchers with varying resources. Our baseline experiments show that the DataComp workflow leads to better training sets. Our best baseline, DataComp-1B, enables training a CLIP ViT-L/14 from scratch to 79.2% zero-shot accuracy on ImageNet, outperforming OpenAI's CLIP ViT-L/14 by 3.7 percentage points while using the same training procedure and compute. We release datanet and all accompanying code at www.datacomp.ai.* Visual Instruction Tuning (Liu et al)* Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field. In this paper, we present the first attempt to use language-only GPT-4 to generate multimodal language-image instruction-following data. * By instruction tuning on such generated data, we introduce LLaVA: Large Language and Vision Assistant, an end-to-end trained large multimodal model that connects a vision encoder and LLM for general-purpose visual and language understanding.* Our early experiments show that LLaVA demonstrates impressive multimodel chat abilities, sometimes exhibiting the behaviors of multimodal GPT-4 on unseen images/instructions, and yields a 85.1% relative score compared with GPT-4 on a synthetic multimodal instruction-following dataset. When fine-tuned on Science QA, the synergy of LLaVA and GPT-4 achieves a new state-of-the-art accuracy of 92.53%. We make GPT-4 generated visual instruction tuning data, our model and code base publicly available.* Tree of Thoughts: Deliberate Problem Solving with Large Language Models (Yao et al)* Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. * To surmount these challenges, we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving. * ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices.* Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. * Code repo with all prompts: https://github.com/princeton-nlp/tree-of-thought-llm.* Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al)* LMs exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. They also, paradoxically, struggle with basic functionality, such as arithmetic or factual lookup, where much simpler and smaller specialized models excel. * In this paper, we show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. * We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction. * This is done in a self-supervised way, requiring nothing more than a handful of demonstrations for each API. We incorporate a range of tools, including a calculator, a Q&A system, a search engine, a translation system, and a calendar. * Toolformer achieves substantially improved zero-shot performance across a variety of downstream tasks, often competitive with much larger models, without sacrificing its core language modeling abilities.* Voyager: An Open-Ended Embodied Agent with Large Language Models (Wang et al)* We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: * 1) an automatic curriculum that maximizes exploration, * 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and * 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. * Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent's abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize.Voyager discovers new Minecraft items and skills continually by self-driven exploration, significantly outperforming the baselines.* Evaluating Cognitive Maps and Planning in Large Language Models with CogEval (Momennejad et al)* Recently an influx of studies claims emergent cognitive abilities in large language models (LLMs). Yet, most rely on anecdotes, overlook contamination of training sets, or lack systematic Evaluation involving multiple tasks, control conditions, multiple iterations, and statistical robustness tests. Here we make two major contributions. * First, we propose CogEval, a cognitive science-inspired protocol for the systematic evaluation of cognitive capacities in LLMs. The CogEval protocol can be followed for the evaluation of various abilities. * * Second, here we follow CogEval to systematically evaluate cognitive maps and planning ability across eight LLMs (OpenAI GPT-4, GPT-3.5-turbo-175B, davinci-003-175B, Google Bard, Cohere-xlarge-52.4B, Anthropic Claude-1-52B, LLaMA-13B, and Alpaca-7B). We base our task prompts on human experiments, which offer both established construct validity for evaluating planning, and are absent from LLM training sets.* * We find that, while LLMs show apparent competence in a few planning tasks with simpler structures, systematic evaluation reveals striking failure modes in planning tasks, including hallucinations of invalid trajectories and falling in loops. These findings do not support the idea of emergent out-of-the-box planning ability in LLMs. This could be because LLMs do not understand the latent relational structures underlying planning problems, known as cognitive maps, and fail at unrolling goal-directed trajectories based on the underlying structure. Implications for application and future directions are discussed.* Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Albert Gu, Tri Dao)* Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers' computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. * First, simply letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information along the sequence length dimension depending on the current token. * Second, even though this change prevents the use of efficient convolutions, we design a hardware-aware parallel algorithm in recurrent mode. We integrate these selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba). * Mamba enjoys fast inference (5x higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences. As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics. On language modeling, our Mamba-1.4B model outperforms Transformers of the same size and matches Transformers twice its size, both in pretraining and downstream evaluation.* Get full access to Latent Space at www.latent.space/subscribe

Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers' computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. First, simply letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information along the sequence length dimension depending on the current token. Second, even though this change prevents the use of efficient convolutions, we design a hardware-aware parallel algorithm in recurrent mode. We integrate these selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba). Mamba enjoys fast inference (5$times$ higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences. As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics. On language modeling, our Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size, both in pretraining and downstream evaluation. 2023: Albert Gu, Tri Dao https://arxiv.org/pdf/2312.00752v1.pdf

One of the things that has been interesting to watch over time is how the SQL Server platform has expanded the amount of information that we get back about the performance of the query optimizer and query processor. While it's not perfect, and there is room for improvement, the advances made with intelligent query processing are helping many systems run faster. Not all queries, but some. As I've done a little work on other platforms, there are ways to look for potential missing indexes in PostgreSQL and MySQL, but these aren't built into tools, nor are they easily accessible to developers or DBAs. There's work to be done on many platforms, though I'm not sure if there is more work than required in SQL Server. On all these platforms, you need to dig into queries and understand why they are slow, though the tooling for SQL Server, with graphical plans in SSMS (or with Plan Explorer) can make the job easier. Read the rest of Be Careful with Missing Index Requests

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Interpretability Externalities Case Study - Hungry Hungry Hippos, published by Magdalena Wache on September 20, 2023 on LessWrong. Some people worry about interpretability research being useful for AI capabilities and potentially net-negative. As far as I was aware of, this worry has mostly been theoretical, but now there is a real world example: The hungry hungry hippos (H3) paper. Tl;dr: The H3 paper Proposes an architecture for sequence modeling which can handle larger context windows than transformers Was inspired by interpretability work. (Note that the H3 paper is from December 2022, and it was briefly mentioned in this discussion about publishing interpretability research. But I wasn't aware of it until recently and I haven't seen the paper discussed here on the forum.) Larger Context Windows The H3 paper proposes a way to use state space models (SSMs) for language models instead of attention. With an SSM it's possible to model longer sequences. Using attention, the compute for context window length n scales with O(n2). Using the SSM based architecture, the compute scales with O(nlog(n)). Inspired by Interpretability Work The paper mentions that the work was inspired by Anthropic's In-context learning and induction heads paper. E.g. they write We provide an informal sketch of a two-layer attention model that can solve the associative recall task, inspired by the construction of [In-context learning and induction heads paper]. There is also the "Hyena paper" which builds on the H3 paper, and was also inspired by the induction heads paper: This work would not have been possible without [...] inspiring research on mechanistic understanding of Transformers (Olsson et al. 2022; Power et al. 2022; Nanda et al. 2023). My Takes These two papers in particular will probably not shorten AI timelines much. It seems unlikely that this type of architecture ends up being the state of the art. However, the example makes me take the downsides of publishing interpretability research more seriously. Even if this work itself is not a key capability milestone, it shows that there is truth in the argument "If we understand systems better, it will not just be useful for safety but also lead to capability advancements" Capabilities externalities are a strong argument that most (good) interpretability research should not be published There are alternative ways to distribute research which are less risky than publishing. We can probably learn something by studying military research practices which have a similar use case of "make research accessible to other researchers while preventing it from becoming public" The constraints are less strict than with military research because there is not an adversary force trying really hard to get access. Maybe this is already relatively common (I would not know of most unpublished research) On the other hand, interpretability research is probably crucial for AI alignment. I think it is possible but unlikely that we get alignment without extremely good interpretability. The cost of keeping interpretability research private is really high. Publishing is a great driver of scientific progress. Overall, publishing interpretability research seems both pretty risky, and extremely valuable, and it's not clear to me if it is worth it. Your Takes? I would be really interested to see a discussion about this! How big a deal are the H3 and Hyena papers? Does this example change your mind about whether publishing (or even doing) interpretability research is a good idea? Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Interpretability Externalities Case Study - Hungry Hungry Hippos, published by Magdalena Wache on September 20, 2023 on The AI Alignment Forum. Some people worry about interpretability research being useful for AI capabilities and potentially net-negative. As far as I was aware of, this worry has mostly been theoretical, but now there is a real world example: The hungry hungry hippos (H3) paper. Tl;dr: The H3 paper Proposes an architecture for sequence modeling which can handle larger context windows than transformers Was inspired by interpretability work. (Note that the H3 paper is from December 2022, and it was briefly mentioned in this discussion about publishing interpretability research. But I wasn't aware of it until recently and I haven't seen the paper discussed here on the forum.) Larger Context Windows The H3 paper proposes a way to use state space models (SSMs) for language models instead of attention. With an SSM it's possible to model longer sequences. Using attention, the compute for context window length n scales with O(n2). Using the SSM based architecture, the compute scales with O(nlog(n)). Inspired by Interpretability Work The paper mentions that the work was inspired by Anthropic's In-context learning and induction heads paper. E.g. they write We provide an informal sketch of a two-layer attention model that can solve the associative recall task, inspired by the construction of [In-context learning and induction heads paper]. There is also the "Hyena paper" which builds on the H3 paper, and was also inspired by the induction heads paper: This work would not have been possible without [...] inspiring research on mechanistic understanding of Transformers (Olsson et al. 2022; Power et al. 2022; Nanda et al. 2023). My Takes These two papers in particular will probably not shorten AI timelines much. It seems unlikely that this type of architecture ends up being the state of the art. However, the example makes me take the downsides of publishing interpretability research more seriously. Even if this work itself is not a key capability milestone, it shows that there is truth in the argument "If we understand systems better, it will not just be useful for safety but also lead to capability advancements" Capabilities externalities are a strong argument that most (good) interpretability research should not be published There are alternative ways to distribute research which are less risky than publishing. We can probably learn something by studying military research practices which have a similar use case of "make research accessible to other researchers while preventing it from becoming public" The constraints are less strict than with military research because there is not an adversary force trying really hard to get access. Maybe this is already relatively common (I would not know of most unpublished research) On the other hand, interpretability research is probably crucial for AI alignment. I think it is possible but unlikely that we get alignment without extremely good interpretability. The cost of keeping interpretability research private is really high. Publishing is a great driver of scientific progress. Overall, publishing interpretability research seems both pretty risky, and extremely valuable, and it's not clear to me if it is worth it. Your Takes? I would be really interested to see a discussion about this! How big a deal are the H3 and Hyena papers? Does this example change your mind about whether publishing (or even doing) interpretability research is a good idea? Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Interpretability Externalities Case Study - Hungry Hungry Hippos, published by Magdalena Wache on September 20, 2023 on LessWrong. Some people worry about interpretability research being useful for AI capabilities and potentially net-negative. As far as I was aware of, this worry has mostly been theoretical, but now there is a real world example: The hungry hungry hippos (H3) paper. Tl;dr: The H3 paper Proposes an architecture for sequence modeling which can handle larger context windows than transformers Was inspired by interpretability work. (Note that the H3 paper is from December 2022, and it was briefly mentioned in this discussion about publishing interpretability research. But I wasn't aware of it until recently and I haven't seen the paper discussed here on the forum.) Larger Context Windows The H3 paper proposes a way to use state space models (SSMs) for language models instead of attention. With an SSM it's possible to model longer sequences. Using attention, the compute for context window length n scales with O(n2). Using the SSM based architecture, the compute scales with O(nlog(n)). Inspired by Interpretability Work The paper mentions that the work was inspired by Anthropic's In-context learning and induction heads paper. E.g. they write We provide an informal sketch of a two-layer attention model that can solve the associative recall task, inspired by the construction of [In-context learning and induction heads paper]. There is also the "Hyena paper" which builds on the H3 paper, and was also inspired by the induction heads paper: This work would not have been possible without [...] inspiring research on mechanistic understanding of Transformers (Olsson et al. 2022; Power et al. 2022; Nanda et al. 2023). My Takes These two papers in particular will probably not shorten AI timelines much. It seems unlikely that this type of architecture ends up being the state of the art. However, the example makes me take the downsides of publishing interpretability research more seriously. Even if this work itself is not a key capability milestone, it shows that there is truth in the argument "If we understand systems better, it will not just be useful for safety but also lead to capability advancements" Capabilities externalities are a strong argument that most (good) interpretability research should not be published There are alternative ways to distribute research which are less risky than publishing. We can probably learn something by studying military research practices which have a similar use case of "make research accessible to other researchers while preventing it from becoming public" The constraints are less strict than with military research because there is not an adversary force trying really hard to get access. Maybe this is already relatively common (I would not know of most unpublished research) On the other hand, interpretability research is probably crucial for AI alignment. I think it is possible but unlikely that we get alignment without extremely good interpretability. The cost of keeping interpretability research private is really high. Publishing is a great driver of scientific progress. Overall, publishing interpretability research seems both pretty risky, and extremely valuable, and it's not clear to me if it is worth it. Your Takes? I would be really interested to see a discussion about this! How big a deal are the H3 and Hyena papers? Does this example change your mind about whether publishing (or even doing) interpretability research is a good idea? Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

I've used a number of clients with SQL Server across the last 30 years. I worked early on with the command line and isql (now SQLCMD and soon Go-SQLCMD). That was handy with DOS and Windows 3.1. However, I soon moved to GUI tools and spent years in ISQL/W,  which was how I ran Query Analyzer. Enterprise Manager was the next evolution, though I used Rapid SQL for awhile to get offline query work while on a plane. This was handy for me in documenting and commenting code without a server. Amazing to think I used to work without a SQL instance on my laptop. SSMS (Management Studio) has become the de facto way I've written queries for over a decade. Microsoft has tried to push people to Azure Data Studio (ADS), but I don't see a lot of people moving to it. It's fast, but also, not attractive, at least not to me. Read the rest of The Importance of Thick Clients

Sebelum Pasar Buka: Laporan Makro AS dan Hijau-nya pasar global. Jatuhnya harga komoditas. Analisis #IHSG #GOTO #SSMS #BYAN dan #GDST. Efek Laporan Jobs market AS ke IHSG Kita briefing pagi sebelum pasar buka di hari Senin, tanggal 5 Juni 2023 Hari senin adalah hari untuk briefing apa yang akan terjadi satu minggu ke depan. Di #podcast kali ini: Laporan makroekonomi AS yang menunjukan besarnya lapangan pekerjaan yang tersedia di AS. Hal ini mengindikasikan bahwa inflasi masih akan membandel, dan tentu saja ini menjadi indikasi kenaikan suku bunga. Lalu, harga komoditas juga berjatuhan, dan kita bisa lihat bagaimana efeknya ke emiten emiten energy di Indonesia. Podcast kali ini juga bahas bagaimana dampak laporan makroekonomi tersebut ke mood pasar IHSG senin nanti? Bagaimana analisis teknikal #IHSG #GOTO #SSMS #BYAN dan #GDST? Episode kali ini berdurasi 13 menit! #Sebelumpasarbuka

Transformers have been essential to pretraining success in NLP. While other architectures have been used, downstream accuracy is either significantly worse, or requires attention layers to match standard benchmarks such as GLUE. This work explores pretraining without attention by using recent advances in sequence routing based on state-space models (SSMs). Our proposed model, Bidirectional Gated SSM (BiGS), combines SSM layers with a multiplicative gating architecture that has been effective in simplified sequence modeling architectures. The model learns static layers that do not consider pair-wise interactions. Even so, BiGS is able to match BERT pretraining accuracy on GLUE and can be extended to long-form pretraining of 4096 tokens without approximation. Analysis shows that while the models have similar average accuracy, the approach has different inductive biases than BERT in terms of interactions and syntactic representations. 2022: Junxiong Wang, J. Yan, Albert Gu, Alexander M. Rush https://arxiv.org/pdf/2212.10544v2.pdf

On this episode, we're joined by Aidan Gomez, Co-Founder and CEO at Cohere. Cohere develops and releases a range of innovative AI-powered tools and solutions for a variety of NLP use cases.We discuss:- What “attention” means in the context of ML.- Aidan's role in the “Attention Is All You Need” paper.- What state-space models (SSMs) are, and how they could be an alternative to transformers. - What it means for an ML architecture to saturate compute.- Details around data constraints for when LLMs scale.- Challenges of measuring LLM performance.- How Cohere is positioned within the LLM development space.- Insights around scaling down an LLM into a more domain-specific one.- Concerns around synthetic content and AI changing public discourse.- The importance of raising money at healthy milestones for AI development.Aidan Gomez - https://www.linkedin.com/in/aidangomez/Cohere - https://www.linkedin.com/company/cohere-ai/Thanks for listening to the Gradient Dissent podcast, brought to you by Weights & Biases. If you enjoyed this episode, please leave a review to help get the word out about the show. And be sure to subscribe so you never miss another insightful conversation.Resources:- https://cohere.ai/- “Attention Is All You Need”#OCR #DeepLearning #AI #Modeling #ML

There was a Slack thread at Redgate recently where a developer was showing some code where they decided to use the "extra" column from the information_schema.columns view. They were making decisions on how to detect certain metadata about a column based on the data in this column. Apparently, the data in here is overloaded for different options that might be set on a table. This caught my eye because I had no idea there was a column named "extra" in this view. I flipped over to SSMS and decided to check what was being stored in here. To my surprise, there was no "extra" column. As I dug in a little deeper in the thread, I realized the developer was talking about Information_schema.columns in a MySQL database. Read the rest of Information Schema Strangeness

When we couldn't confirm a guest speaker by recording time, Eugene, Kevin, Amanda, and I decided the show must go on, so we went into the ideas vault and pulled out a couple of listener questions we hadn't answered.  In this episode, we talk about how to create your own documentation, SSMS installs, SOS_Scheduler_Yield and Eugene gives us an update on SQL Bits and the conference crud he picked up along the way. The show notes and video for today's episode can be found at https://sqldatapartners.com/2023/03/29/episode-263-sql-server-this-n-that. Have fun on the SQL Trail!

A Middle Tennessee Business PodcastThere's no doubt that teaching is a passion-driven job. They don't get into it for the money. In Ashley Harris' case, she is driven by creating a life that purely fits who she is. Her desire to pour into our kids here in Spring Hill, Tennessee is off the charts!For anyone who knows Ashely, it comes as no surprise that she was selected by her peers as the 22-23 Teacher of the Year for SSMS. Music runs in the family for Ashley as her husband Erick is also a Band Director at the adjacent high school. Plus he's the spitting image of the late great Eddie Van Halen. From Ashley's Facebook page:“I'm a wife, mother, band director & entrepreneur. I love all things Disney and positive motivation.”*****SUBSCRIBE/RATE/FOLLOW What's Your Problem? PODCAST:www.whatsyourproblempodcast.comwww.instagram.com/whatsyourproblempodwww.instagram.com/jimmccarthyvosTiktok: @jimmccarthyvos __________________________________________________________Random 5 sponsored by IT'S YOUR SHOW.CO!www.itsyourshow.coYou know you have a lot to share with the world, but how? What's your source? At It's Your Show.co, your source is a podcast that we help you create to ultimately extract micro-content for your brand! With our 20+ years of experience in podcast, video and radio production, we can churn out a bunch of content for your social platforms...from just one episode! Get started today:www.itsyourshow.co____________________________________________________________****You hear Jim mention it on almost every episode, ME vs. WE and how 2023 will be 1943 all over again….order “PENDULUM:How Past Generations Shape Our Present and Predict Our Future”:https://a.co/d/7oKK7Ip__________________________________________________________________________The co-author of Pendulum wrote a myriad of other books and started a non-profit 21st Century Non-Traditional Business School that you should really check out: Wizard Academy - www.wizardacademy.org__________________________________________________________________________Talking about the real problems (and possible solutions) of everyday business owners and professionals in and around Middle Tennessee and beyond...this is the What's Your Problem Podcast!WYP Intro VO: @johndavidwells#podcasts #podcast #podcasting #tennessee #nashville #podcastproduction #sales #selling #business #marketing #radio #production #voiceover #vo #fitness #bni #networking #nft #nonfungibletokens #crypto #cryptocurrency #applepodcasts #businesscoach #businessideas #businesslife #businesspassion #businesstips #countrymusic #entrepreneurlife #entrepreneurship #marketing #mindset #nashvegas #nashville #nashvillelife #nashvillemusic #nashvillescene #nashvilletennessee #nashvilletn #newpodcast #podcastaddict #podcasters #podcastersofinstagram #podcastlife #podcastlove #podcasts #podcastshow #tennessee #bradlea #therealbradlea #droppingbombspodcast #droppingbombs #lightspeedvt #virtualtraining #grantcardone #garyvaynerchuk #drummer #drummerworld #drumporn #drums #drumsdaily #fashion #guitarist #instadrums #jasonaldean #love #lukebryan #midland #mirandalambert #morganwallen #musicfestival #nashville #nashvillelife #nashvillemusic #nashvillescene #nashvilletennessee #nashvilletn #nature #opry #percussion #realcountrymusic #rock #singer #tennessee #timmcgraw #womenofcountry

Guy and Eitan finally celebrate the generally available release of SQL Server 2022! And also discuss other announcements from PASS Data Community Summit, and also various interesting news and features of Azure SQL announced in November 2022. Relevant links: SQL Server 2022 is now generally available Consumption-based PAYG billing model for SQL Server 2022 on-prem PASS Data Community Summit Hug a DBA song by Killa DBA SQL Server 2022 song by Killa DBA The SQL Server Native Client has been removed from SQL Server 2022 (16.x) and SQL Server Management Studio 19 (SSMS). Switch to the new Microsoft OLE DB Driver (MSOLEDBSQL) for SQL Server or the latest Microsoft ODBC Driver for SQL Server going forward. Mid-November 2022 Public Preview updates to Azure SQL Azure SQL database external REST endpoints integration in public preview Mid-November 2022 General Availability updates to Azure SQL Public Preview Azure SQL Managed Instance feature wave for November 2022 General Availability Azure SQL Managed Instance feature wave for November 2022 Learn more about the November 2022 Feature Wave for Azure SQL Managed Instance Azure SQL - General availability updates for late November 2022 Azure SQL - Public preview updates for late November 2022 DR Secondary licensing is free with Azure VM and SQL Managed Instance in PAYG model

American Invents Act Has Destroyed Innovation - Cops want to keep mass surveillance app secret; privacy advocates refused - Hackers Hide Malware in Stunning Images Taken by James Webb Space Telescope - TikShock: Don't get caught out by these 5 TikTok scams - Ukrainian Police Bust Crypto Fraud Call Centers Well, the birds are coming home to roost. Well, not the chickens in this case, but this is called the death warrant for American ingenuity. We'll start by talking through this great article from this week's newsletter. [Automated transcript follows.] Well, I hate to say this, but in reality, we are looking at some very, very bad times for inventors, and I've had some of these problems myself before, but last September, there were scores of patent holders who demonstrated in six cities across the US. [00:00:34] They had on these black t-shirts that said homo sapiens, inventor. Endangered species. They were protesting America's decade of stolen dreams. Great article here in the American thinker. It was in my newsletter this year, or excuse me this week, but, uh, but here here's weirdly what happened here. Back in 2011, president Obama pushed through Congress and signed into law. [00:01:04] What they called the America invents act. Now just like the inflation reduction act is going to increase inflation, right? It's all double speak. Isn't it? The American invents act turned over the patent process basically to the biggest Democrat party donors. Big business billionaires, right? Because that's who really is funding them, the Hollywood millionaires, these massive billionaires, Zuckerbergs and, and others. [00:01:35] And what happened here? Is they changed the whole patent law and the basis for it. They flipped the table here, basically. Here's the idea behind the patent law that we've had in place in the United States for well, over a century and patents that are guaranteed in the cons. It used to be that you, if you were first to invent something, if you could show that you were first to invent something, you could file a patent and gain that patent. [00:02:14] Well, what happened is because of all of the donations that went into the Democrats in 2011, from these big, big companies that were lobbying. A, and this is part of the reason I have a huge problem with all this money going to Washington DC, frankly, because it just attracts rodents like these big companies that want to use the law to control you, to gain profit for them. [00:02:39] And really in this case, squash. Potential patent holders. You see there have been piracy for years in the patent field. And this happened to me. I spent a year of my life designing some software, writing some software that emulated an older computer system and allowed you to take. Any of that software and run it on the new system. [00:03:05] And it would run exactly the same way. And a lease on the new hardware was cheaper than just a maintenance contract on the old stuff. Plus it was faster, used less electricity, had more options, et cetera. Right. It was, it was really something, frankly, and I was invited to their headquarters to show them a little bit about. [00:03:27] Did, and, and I was so excited because they wanted to start selling it, right. So they need to understand a little bit better. So I went to the headquarters and met with them, you know, of course paid my own way. Flew down there, stayed in the hotel, rented a car, you know, all the stuff that you have to do. [00:03:43] And then nothing happened afterwards. Wouldn't return phone calls. It just, all of a sudden went silent. And then about a year and a half later Tata, they had an alternative product out on the. . Yeah, and they tried to emulate what I had done, but they did a very, very poor job at it. That's patent theft, that's piracy in this particular case, uh, if you are an inventor, you've probably experienced that sort of thing before, you know, you can put employees all of the non-disclosure agreements you want to have in place, but in reality, good luck enforcing those, especially against a big company. [00:04:25] Well, piracy went on steroids because of president Obama's America and events act. They, as part of that established something, they called the patent trial and appeal board. And it's just gone downhill ever since. So a professor that has more than 40 patents, I'm gonna read a little quote of his, this includes some inventions used in the space shuttles, by the way, which by the way, my invention was used with the space shuttle. [00:04:57] Um, so Dan brown invented something called the bionic wrench. I have one of those. I bought one of those some years back, this is a one size fits all wrench that does not strip bolt corners like it does if you're trying to use vice scripts or some pair of pliers, right. Because you're just too lazy to go and get the right socket size or box wrench or whatever it is. [00:05:22] That's the right size. It very, very. And professor brown says that Sears stole his idea for this bionic wrench right down to the marketing pitch. And then Sears, according to him, went out and hired a Chinese company to make it. And all of a sudden now, what kind of invention does he have? How's he gonna battle somebody like that? [00:05:49] I know a guy who is, uh, completely unethical. You know, I've done many shows from the consumer electronic show and it's really kind of cool, cuz I would get in depth with the inventors and, and explain what they were doing on the air. It was really neat all the way around. It was just a whole lot of fun. [00:06:08] And I met a guy there who was going to the consumer electronic show to find cool new consumer electronics. He thought might be popular. And then he'd go and talk to the people who were exhibiting that wonderful new electronics and say, Hey, I'm interested in, in selling your stuff. I have, you know, retail space and, uh, you know, kiosks in the mall. [00:06:33] What can, uh, what kind of deal can we work out here? Well, you know, first I, can I, let me get a, I, I need a copy of, uh, of your device here. I want a copy of it so I can mess with it and see, see if we really wanna follow through on. Oh, and I, I don't want to carry it around the floor of the consumer electronic show. [00:06:51] So I need you to ship it to me. So they'd ship 'em off. They might be a little speaker. They might be a charger. They might be who knows what? And consumer electronics is pretty broad. And if he liked it, he wouldn't buy it from them. He sent them over to his contacts in China. And had them reverse engineered and make the same thing with his brand label on it. [00:07:16] And he'd sell it in the stores. Now, when it comes to software and a lot of consumer electronics patents, aren't really a big deal because things. Changed so quickly. Right? And if you're a small guy, it's very hard to file a patent. And that's how president Obama sold this American Bens act to us. I remember this very, very clearly where he said, Hey, listen, this is gonna make the patent process way more streamlined, way easier for the small guys to be able to get patents, uh, not only applied for, but actually get them out to market. [00:07:52] And it's just gonna be an absolutely wonderful. It, it isn't because what happens now? Is big companies are not investing in research and development. That is true across the board. Now you might say, Hey Craig, well, how about big companies? How about Tesla? That's R and D. How about SpaceX? That's R and D. [00:08:14] Yes, but they are R and D companies. They're not big companies out there like Facebook, does Facebook try and come up with this or that new invention? Well, yeah, they kind of do from time to time, but most of the time what's been happening is corporate America looks for a winner. And then tries to buy the winner. [00:08:35] Microsoft has been doing that forever. Microsoft in court has lost cases because of what they did to inventors. And now it's been codified in law for over 10 years. So our American ingenuity, which is what we rely on in order to grow our economy, the ingenuity, the, the brain skills, the science, the true science that we have gives us a major competitive advantage because that particular, uh, type of intellectual property has a much higher profit margin than something like manufacturing a widget. [00:09:14] When you get right down to it, that's where the real money is. so a very interesting article and I would suggest you take a little bit of time to read it. If you've ever thought about patenting something, if you had a great idea, it used to be, you know, this is kind of the, the, uh, old wives tale. If you will, if you've got a great idea, you think you might wanna patent it, write it all out, take all of your notes, do it in a, a, a workbook that you can. [00:09:43] Alter right. You can't tear out pages or things. Uh, mail it to yourself in a Manila envelope and make sure you put stamps on it. And then the post office is going to date, stamp it for you or send it to your attorney even better. Right. And your attorney's gonna go ahead and keep that on file. And then when it's time to file the patents, you can say, Hey, look, it here's the proof. [00:10:06] I invented this in April of 2019. It doesn't matter because if some other company sees what you're doing or comes up with a similar or the same idea, and that company has the money to have the lawyers that know patent law inside out and backwards and can go ahead and file that patent claim. You've lost it. [00:10:31] you know, as early as the constitutional convention of 18 or 1787, our founding fathers recognize the need to promote innovation and we have to be promoting it. We've gotta get rid of this Obama era law. Absolutely. We've gotta go from first to file, which is what it has been for a decade. The first person to file you. [00:10:54] And move back to the way it was intended, the way it worked for well over a hundred years where it is a first to invent, it's very, very important for all of us, for economy, et cetera. The, the third law of Congress was a patent act of 1790. It it's just man, have we come a long way, stick around. We'll be right back online. [00:11:19] Craig peterson.com. [00:11:22] You know, we've had firewalls in our cars for a very long time for a very good reason. Right? You wanna keep the engine stuff out of the passenger compartment? The same thing is true. When we're talking about our networks, we're using firewalls to keep things out. [00:11:39] Firewalls are there to keep things out. And we have firewalls in our homes. [00:11:44] If you've got an internet service provider, you've probably got a firewall right there. Something that you don't even think about, right. It's just, there's gonna protect. You, it might, it's providing some services. You might be familiar with them. It's obviously doing a network address translation for you in this day and age. [00:12:06] Pretty much everything is especially with the internet transition that's been going on for years now from, um, IP four to IP six, but, uh, the firewall. is critical for every person and every business out there. But when we get into the configurations of firewalls, frankly, they are really a touchy subject. [00:12:29] You know, every network security professional has their own preferred hardware and software, uh, use Cisco. As a rule, Cisco has some great stuff. What I like the best about the Cisco equipment that we use in software and install at our clients is it is one pane of glass. It's a single vendor that covers everything from endpoint security. [00:12:54] In other words, security on your desktop, through the network itself, the switches, the firewalls, the email filters Absolut. Everything is there and is taken care of by all of the Cisco gear. It it's really quite something to look. I saw, in fact, a survey just last week at businesses who are trying to consolidate, there's just too many vendors in there selling this piece of endpoint, that piece of endpoint. [00:13:25] And, you know, that's part of the problem that I see happen pretty frequently, which is people look at Gartner report. Gartner, of course, a research company. They've got a lot of great research out there that I've used before. I've had Gartner on the radio show before, as well as some of their competitors talking about trends. [00:13:44] Well, There is something known as the upper right quadrant in those Gartner reports where they are rating various vendors for various pieces of software. So there might be for instance, a report on firewalls and the upper right hand cor quadrant is kind of what you want, cuz it's new, it's innovative. It, it innovative. [00:14:06] It's uh, really cool and wonderful. And it's the best. Since life spread. So they go out and they buy that cuz it's upper, right. Gartner quadrant. And then man, they find out, uh, okay, so now we need desk desktop, desktop. Okay. So they find the or buy actually the Gartner report for five to 10 grand. That's like a page long is crazy how expensive these things are. [00:14:32] They then look at that and say, okay, so the best desktop is vendor Y so let me see, we got X for the firewall. We've got Y for the endpoint and then, oh, they need switches. So let's go to the Gartner report. Who's in the upper right quadrant here for switches. Oh, it's uh, vendor Z. Okay. So we got Z. So now all of a sudden. [00:14:51] You end up with all of these different pieces of hardware, different pieces of software that have limited offerability at best interoperability at best. Right? So the, this day and age, when we're talking about cybersecurity, There are so many legitimate attacks every day. I mean, thousands of attacks going on even against a single business. [00:15:18] And there are hundreds potentially of false alarms every day. So how do you deal with that? That that's a good question. So, uh, a lot of businesses turn to companies like mine now, you know, full disclosure, I've been doing internet security work for businesses since, uh, early 1990s. So whew, 30 years now. [00:15:40] And I've been doing internet work for even longer than that, helping to develop it. So they'll go and they'll say, Hey, we need a managed security services provider. Uh, there's a big problem with that. And I, I was watching, uh, Yellowstone that TV show and I, it was a great little example of what we're seeing in the world today. [00:16:05] And Frank, frankly, we've seen forever obviously. And that is if there's a demand for something, all of a sudden, a lot of people will be hanging up shingles. and if they know, if that vendor knows more than you do, or is able to kind of turn, twist your ear and convince you to buy from them, you'll buy from them. [00:16:26] We saw that man around the year, 2000, all of the people who were trying to sell web services that had no idea what they're doing now, we're seeing all kinds of people trying to sell network services, security services that have little idea of what they're doing. We support. These companies that call themselves manage security services providers, where we actually go in, we design the system, we build the system and we implement the system. [00:16:53] We run the system and the third party here builds the client. Right. Cuz it's their client. And you know, that's all fine. It's so well and good, but what should you be looking. Particularly if you are a business, if you want to have a managed firewall, which is, I think important again, it's kind of a long tail thing to have a firewall vendor and, uh, this vendor and a managed vendor, and now it can get to be a headache pretty quickly. [00:17:23] But if you're going to focus on one thing, It's probably the firewall and your end points. Right? So maybe it's two things. So here's what a managed firewall service provider should be able to offer you. First of all, firewall system health and alerting. Software life cycle management, which means your updates, your patches, service, and incident management. [00:17:48] Whenever there's an alarm, they should know about it and they should be handling it. Security policy implementation your reporting, your analysis, your remediation, some of that is required by these various regulations and laws that are out there. You. To do it, uh, you know, without getting in a lot of detail right now, um, network monitoring, uh, the traffic monitoring, you know, the idea here behind any kind of managed service is to bring in a true expert rather than just completely outsourcing. [00:18:24] So you're partnering with someone. One of the things I've, I've bated my head against the wall for, for decades now, is that the it department. Thinks that they're up to snuff to be able to do something, or maybe they just want to do it because it's gonna be wonderful for them on the resume for the next job. [00:18:45] Right. Uh, man, I've seen that a lot of times when, when you are looking at all of this stuff and you've got an it department, maybe you're better off bringing in a very narrow expert to support your it department rather than fight against your it depart. good questions here. Uh, bottom line, they should have better expertise than what you have. [00:19:11] And you've got to read between the lines between your it staff that are currently doing it and the other vendors reducing the burden on your staff. So that maybe what they can do is. Focus more on things that are, uh, revenue generating that are more important to your business. You'll get faster incident response with any luck here. [00:19:33] With service level agreement, proactive security from the managed security services providers, or just regular service providers. Your burden on updates is going to be lower, improved manufacturer support. Because a lot of times, like we do my company mainstream, we have direct connections to the manufacturer. [00:19:56] Our case is usually Cisco because of the volume or services that we have and the equipment that we buy from them, uh, easier to scale there. There's a whole bunch of things, right. Uh, But be careful. One of the things you gotta watch out for too is where are their service people, their support people physically located, and are they us citizens? [00:20:20] A lot of the regulations. In fact, pretty much everyone. I can't think of an exception require us persons to be the ones in control of your network and data. So lots to consider. But keep that all in mind. I think it's an important thing to understand. Stick around. We'll be right back. And in the meantime, visit me online. [00:20:42] Craig peterson.com and sign up for my free newsletter. [00:20:49] The best way to secure a system is something, you know, and something you have, well, many systems have been securing themselves with your phone, right? They send you a text message, but it turns out that that isn't working well. [00:21:05] Having an SMS message sent to you in order to authenticate who you are, has turned out to be well, a problem we've seen over the last few years, people who have things like cryptocurrency who have a cryptocurrency wallet who are keeping their money, if you will, in this wallet and are using. [00:21:30] SMS to verify who they are. So here's how that works. You log into a website using a username or perhaps an email address. Again, it should not be asking for an email address for a login because you probably use the same email address or maybe two or three. And. Have for what? 50, a hundred different sites, maybe a thousand, I've got 3000 records, uh, logins on my one password account. [00:22:02] Okay. So there's a lot of them. They really should be letting you set up your own username so that it can be unique. For every single website that you go to. So, but anyway, that aside, you've got your username, which may be your email address. You've got a password and we've talked about passwords before. [00:22:21] Hopefully you're following the current guidelines, which are, don't worry about random characters, make sure it is long. And that means. A past phrase. So you string three or four words together. You put some digits, some special characters in between the words, maybe, you know, one word is all upper case. You, you play with it a little bit, but it's easy to remember. [00:22:48] So if someone then gets your email address and they get your password, they can potentially log into a website. Correct. And that website might be your bank account. It might be your work account. We've had a lot of problems lately. The FBI is saying that about every 12 hours, they're filing a new report of a company that got their intellectual property stolen. [00:23:22] one of the ways the bad guys steal it is they'll log to your RDP server, your Microsoft remote desktop server, using your credentials that you used at another website. It's that easy. It really is. They might be trying to log in via a VPN again, the same thing. So how do you secure this? How do you secure this? [00:23:47] Well, how to secure this properly? That's where the something you have comes into play. We all have a smartphone of some sort, even if it's not considered a smartphone, it can still receive text messages. So what a lot of these companies did is they asked their underpaid it people to set it up so that when you enter in your username and your password, it then sends you a text message. [00:24:16] Usually with a six digit text message and you then have to type that into the website as well. Seems pretty good. Doesn't it? Well, and, and in 30 it is pretty good. There are however, a few problems. Those people I mentioned who have cryptocurrency accounts and have been using this SMS methodology, which is SMS, of course, text messages have found that sometimes their phones have been hijack. [00:24:48] easy enough to do. And if they know you have a fair amount of cryptocurrency, it's probably worth their effort to spend a few hours to try and get into your account. And they have been getting into your account and people notice, Hey, wait a minute, I'm a kid. They do phone calls or text messages. What's what's with that. [00:25:07] And you found out that they have dismissed you, they have stolen your. Your, uh, SIM card, basically, even though they don't have to physically have hold of it. And there's a number of ways that they do that there's a new scam or newer scam that's out right now that the fishers are using. And that is they're sending out these SSMS, these text messages that are trying to get people to respond. [00:25:34] So how do they get people to respond? Well, In this case, they're primarily going after this company called Octo Octo post. And, uh, there's a number of different types of Octos out there, but anyways, they are trying to get you to. Do something you shouldn't do let me just put it that way. Right. So what they're trying to do is get you to, uh, enter in your username and your password. [00:26:04] Okay. Well, that's been around for a long time. Craig, you're telling me we've had fake bank account, uh, bank website. So they'll send you an email and in it, they'll say, Hey, I need you to go right now. to our bank page and, uh, authorize this $2,000 transaction that wasn't you. And so now you're freaking out, you click on the link, you go to the bank, you try and log in and the login doesn't work well. [00:26:31] That can be because what the fishers did is a made a webpage that looked like the bank's web page. And when you went there and entered in your username and password, you just gave it to the crooks. That's happened a lot. Well, there's a company called Octa O K T a. That is an authentication company. And what the bad guys have done is they have registered domains similar to a company. [00:26:59] So for instance, they went after CloudFlare, which is a huge, um, company they're number one, I think they have like 80% of all of the protection for denial of service and caching a business on the internet. It's just amazing. Cloudflare's huge. And I've used them and continue to use them for some customer. [00:27:19] So, what they did is they found a whole bunch of people that worked for CloudFlare sent them a message. And, and here's what it said. It said alert, your CloudFlare schedule has been updated. Please tap cloudflare-okta.com to view. The changes. So you go there, it looks like a regular Okta login page and they go ahead and ask user name and password, but CloudFlare is smart. [00:27:47] They're using Okta. So they're sending an SMS message to the user to make sure it's really, them turns out what was really happening is yeah, it was sending that guy a text message and it was using telegram. To relay that his response back to the hackers. So now the hackers have your username, they have your password and they have your six digit login key. [00:28:15] That's supposedly unique that supposedly went to you. And in this case, they didn't even have to bother a hijacking your SIM card. In this case, they just sent you that text message. So it's been causing some serious problems. They've been going after all kinds of different companies out there, uh, food service company, DoorDash you've heard of them. [00:28:37] Right? August 25th, they said that there was a sophisticated fishing attack on a third party vendor that allowed a attackers to gain access to some, a door dashes internal company. Tools DoorDash said, intruders stole information on a small percentage of users that have since been notified, big deal, or what a tech crunch, by the way, reported that the incident was linked to the same fishing campaign that targeted Twilio. [00:29:07] That also, as we just mentioned, targeted cloud. So we have to be careful with this. We cannot be using SMS text messages to authenticate ourselves. Some banks now allow you to use one time passwords from things like one password or others. However, some banks don't turn off the SMS, the text messages for authentication, which they really should be doing. [00:29:36] And the other thing I wanna let you know is I like UBI. Y U B ico.com. Yubico check them out. I'm not making a dime off of this, but they have a physical token. That you either have to plug in or the connects via Bluetooth. That is something you have that authenticates you to all major popular websites out there, and many of the tools. [00:30:03] So if you have any questions, just email me, me@craigpeterson.com gimme a few days, but I'll get back to you. [00:30:12] Have you heard about fog reveal? They it's almost invisible when you search for it online, but it's something that police departments have started using. And they're trying to keep all of this secret. So we're gonna tell you what's happening there and got a few others too. [00:30:29] Great little article that was in the newsletter this week. [00:30:32] Hopefully you got my free newsletter, but it is about fog reveal. This is an ours Technica. Often some of these ours Technica stories are carried in multiple places online. It's kind of interesting because we know to some degree what the federal government's doing to collect information on people, they go to open source. [00:30:57] Sources of information. In other words, things that are put out there publicly online, so they might search you your Facebook information or what you've been saying on Twitter, uh, or more, they go to data brokers that anybody can go to. And those data brokers have more information. They probably. Bought records from the states and they know from each individual state what property you own. [00:31:25] If you have a car, if there's liens on it, any mortgages that you might have, right. Putting all of the stuff together. It's kind of an interesting problem, frankly, but that's a, again, they say it's legitimate. Now the federal government is not allowed to collect this information. So they just go to third party data aggregators. [00:31:45] And remember again, If you have apps on your phone, if you have an Android phone, this does not apply to iPhones. Generally it does apply to iPhone apps. However, but, and this is part of the reason I say never, ever, ever use Android. Okay guys, I, I just. Blows my mind. I, I was talking to an old friend of mine. [00:32:09] Uh, he was the, the CTO in fact for the state of New Hampshire. And he was telling me that, uh, you know, we were talking and telling me, yeah, yeah, I got an Android phone. He says, don't you just love Android? And he knows that I do cybersecurity. He knows I've been in it. He hired my company to do a bunch of different tasks for the state, right over the years, we still do business with the state and he's using Android. [00:32:41] He's probably listening right now. BU get a little note from him, but, uh, it, it, it's a problem to use Android any. Those free apps that you're using, that Google maps app that you're using. And of course you can use that on iOS as well is tracking you. They know where you live because they know where your smartphone stays at night. [00:33:04] They know all of this stuff. How do you think the FBI is able to seize a smartphone at a Hardee's drive through. they know where you are. Well, they have some more access to information as it turns out. Uh, one Marilyn based Sergeant, according to the article wrote in a department, email TDY, the benefit of quote, no court paperwork and quote before purchasing the software. [00:33:37] And the Sergeant said the success lies in secre. interesting. So the electronic frontier foundation, FF, who I have supported over the years and the associated press got together. Now, the associated press won a Pulitzer center for crisis reporting, uh, award, I think. But anyways, the Pulitzer center for crisis reporting also got involved here. [00:34:05] So she had these three different organizations trying to figure out. what could, or what would be considered local places best kept secret. So they went online. They started doing some searching, trying to figure this out. And according to ours, Technica, the reporting revealed the potentially extreme extent of data surveillance of ordinary people being tracked and made vulnerable just for moving about. [00:34:38] Small town America. So it isn't just the big cities where you're tracked anymore. Reports showed how police nearly two dozen agencies. One record shows the total figure could possibly be up to 60. Use Google maps, like technology called frog reveal. now this is licensed by fog. I, I keep saying frog it's fog licensed by fog data science, and it gives state and local police a power to surveil. [00:35:10] Hundreds of billions of records from 250 million mobile devices. And if that doesn't scare you, I don't know what does now FF, the electronic frontier foundation found that fog reveal gets its data from veal. That's the same data source the feds use. neither companies disclosing the nature of their business relationship. [00:35:33] Okay. They fog, reveal. Didn't say what Tel is providing and vice versa, right? Yeah. But it really appears that fog reveal is getting data location services to local police at its steep discount. So it's making it more affordable for smaller police departments and private security companies to access major amounts of data and trace devices across months or even years. [00:36:03] isn't that something. So typically FF found that police agencies license the software annually for costs as low as six grand to nine grand. Some agencies spend even more on this tech to track people as they are moving and exactly where they are. Again, think being in a Hardee's drive through having the FBI show up. [00:36:27] Knowing you're there. Uh, ours reviewed one annual contract in Anaheim, California. That was for more than $40,000. So it took months for these three organizations that are used to digging into this sort of stuff, uh, to figure this out, took more than a hundred public records requests to gather thousands of pages of evidence to trying to compile a picture of how local law enforcement. [00:36:55] Is using and mining the location data. Now, to me, this is scary because we look at abuses of power. Through the years and I it's happened again and again and again, we are smelling more and more like Venezuela than we are free us. It's frankly scary, scary to me, but I'm talking about it cuz I think it's important. [00:37:21] That I bring this to light to everybody else out there. Okay. Now fog data science, managing partner, Matthew Brodrick told the associated press that fog reveal has been critical to police to save time and money on investigations, suggesting police who are under-resourced and investigation suffered from reliance on outdated. [00:37:44] Outdated tech now that's true. Isn't it? But isn't it also true that, uh, that's why we have some of these policies and procedures in place. That's why the Supreme court Miranda decision has some policies and procedures. That's why a warrant, a search warrant is supposed to be specific in what they're looking for and where it is located. [00:38:11] We don't allow these broad warrants that the king used to issue, but we are doing that nowadays. It seems against political enemies and that's where it starts really, really scaring me. It isn't that I think that the, the current administration it, or even the next administration in Washington, DC, is going to be rounding up its enemies and putting them up against a. [00:38:38] But when would it happen? Well, it would happen if everything were in place for it to happen. What's one of the most important things for fastest regime. It's to have a citizenry where they know everything about everyone. It, it reminds me of the Soviet era. Show me the man. I'll show you the crime. There's a great book out there right now. [00:39:04] I think it's called, um, three felonies. a day, I think is what the name of it is. But the, it points out how every last one of the people that call ourselves Americans in the United States of America, every one of us commits at least three felonies a day. Now a lot of these things are just absolutely crazy. [00:39:26] You know, there's been a lot of jokes about, oh, did you chair the label off of that pillow? Well, you can cuz you're the consumer, right? It's. The people that are selling it that are in distribution chain that cannot tear that off by law. Okay. But in reality, there is a lot of stuff that could be used against you. [00:39:46] So it it's like when they say, uh, you know, give me this, or why don't you answer that question? It's none of their business. You have a right to be secure in your papers right now, if they have a warrant that's specific, then you need to surrender it. But hopefully the warrant's actually issued by real court. [00:40:08] Some of these agencies now, uh, like the IRS have their own courts that are paid for by the agency. The judges are working for the agency. So you really think they're gonna be fair. I wonder, I wonder. Okay. Couple more things. Next up these pictures taken by the James web space telescope. Have you seen these? [00:40:35] It is amazing. I've seen them side by side with our latest or, you know, our previous high tech pictures. And we're seeing what maybe galaxies that we never could see before. It's just absolutely crazy. Well, guess what bad guys have seen them as. And they are embedding malware inside of some of these amazing images taken by the James web space telescope. [00:41:05] If you can believe this, by the way, they're writing them in go. Uh, so the Phish and emails, they've got a Microsoft office attachment. That's the entry point for the attack chain when you open it, it retrieves and obfuscated, VBA, macro, which in. Auto executed. all of a sudden there is a macro that is de obfuscated and run on your computer. [00:41:34] So be careful careful with that again. And good news. Microsoft is now turning off the execution of macros by default. Double check your machine, making sure that macros are blocked by default. So, yay. Okay. So they are, by the way, changing campaigns to rogue link and ISO files because of the blocked macros. [00:41:56] But, uh, it's good that Microsoft is doing that. Thank goodness. And you Ukraine, the police busted a crypto fraud call center. In fact, more than. And they're also shattering two more Russian bot farms. So we shouldn't be getting as many of those, uh, phone messages from the, uh, the bad guys scammers as we used to get. [00:42:20] Thank you, Ukraine. All right. Online Craig peterson.com. Get that newsletter and stay on top. [00:42:29] Well, we got some election news here from our friends at Google and at Twitter, they are taking opposite directions about exactly how they're gonna handle news postings about the elections. This is an interesting thing. [00:42:46] The federal election commission is the branch of the us government that monitors elections. [00:42:53] It does things like impose fines for misuse of funds. It sets some of the standards for funds and for their use. And. and one of the things it looks at is what are called in kind contributions. This is where someone might, uh, for instance, run a whole bunch of ads on behalf of a candidate. And those ads are coordinated with the campaign and that is illegal. [00:43:24] You're not supposed to do that. And because it's illegal, you know, they try and stop it. But most of the time they end up finding after the fact. And that's part of the reason they want campaigns to be filing their financial reports fairly frequently so they can catch it quite quickly. Well, There have been many complaints from the G O P about what has happened with some of the campaign finance stuff, where you have someone like Facebook or Twitter or Google, who seems to be meddling with the election. [00:44:02] They are running ads for your competition. They are really screening the results from people's searches. And from that those results they're, they're benefiting. There was a study down in orange county here a few years back where they looked at. Google results that were related to the elections going on in orange county and found that the Google results were tainted in such a way that it dramatically favored the Democrats that were running in those districts in orange county, California. [00:44:39] Pretty interesting when you get right down to it. So the GLP says, wait a minute, now that sort of thing is worth millions, tens of millions of dollars, because if they were going to run TV ads, for instance, to get as many eyeballs, to get as much attention to convince people that this is the way they should vote, that would cost them tens of millions of dollars. [00:45:02] So how much is it worth? Where do you go to really straighten things out in order to ultimately make fairness work and well, you know, that's kind of what the federal election commission's supposed to do. Well, here's, what's happening with the next elections. The federal elections commission has decided that Google. [00:45:28] Getting rid of their anti spam measures for. Candidates does not violate a ban on contributions on inkind contributions. So this is an interesting approach because Google's saying, Hey, listen, we want to allow pretty much any political message to come right through to Google Gmail users, inboxes, and not filter those. [00:45:59] Which I frankly think is a smart move on their part. Now some of these campaigns get pretty crazy. They're sending money requests all of the time. It it's been crazy to watch both sides do this and both sides complain about the other side, doing it. But at least by getting rid of these spam rules for the politicians, their messages are gonna get through. [00:46:24] I think that's ultimately a very good thing. So what kind of messages are gonna get through how and why? Well, ultimately they're saying we're gonna let all of them through. and what that means for you. If you already get some of these messages from the politicians, it means your mailbox. At least if it's a Google Gmail box, you are going to be seen even more during elections. [00:46:51] And I think this is gonna go on for very long time. Because Google doesn't want to get caught in the middle. When we're talking about these in kind contributions. If this were to be done for the Republicans or were to be done just for the Democrat, can you imagine the noise that would be made? By both sides and in kind contributions where the Republicans tens of millions of dollars Googled get dull tied up in some of these, uh, you know, lawsuits that would really be inevitable. [00:47:23] Bottom line. Well, Republicans have accused. Google of giving Democrats an advantage in its algorithms. And, and as I said, there have been studies on that that have proved that they have. The big question is why. And there's an article in ours, Technica talking about a meeting that happened in may 20. 22 between Senate Republicans and Google's chief legal officer. [00:47:52] And he said that the most forceful rebuke came from Senator Marco Rubio from Florida who claimed that not a single email from one of his addresses was reaching inboxes. And the Washington post, which of course is a mouthpiece for the Democrat party reported in late July. That the reason it was getting blocked was that a vendor had not enabled an authentication tool that keeps messages from being marked to spam. [00:48:21] Now, if that's true, The Washington post accidentally reported the truth here. And it might be true. I had a company call me up this week. They had their Google ads account banned, and they were trying to figure out the details of why and what happened. And I went in and we solved that problem, and I noticed that they had. [00:48:44] Properly configured their email. There's there's gets technical here. I have a paper we've put together on this, a special report talking about what's called D K I M. These, uh, SPF records DMAR records and how they should all be set up and why I need to use them. So this company was doing marketing. [00:49:04] Obviously they had a Google, Google ad account. They were sending out emails, but because they had not properly and fully configured their email. They were not getting delivered at the rate that they could get delivered. Now that's kind of a very, very big deal when you get right down to it. And the Washington post is saying, well, that's what happened to center to Rubio. [00:49:26] Now there's other things that might happen too. There are. Keywords that are used. There's software called spam assassin. That's very, very common. I have used it since it came out decades ago. I can't even remember how long spam Assassin's been out there, but it looks for certain things in the emails. , it looks for a lot of graphical content, a lot of HTML, even a lot of links and it kind of, it gauges, you know, this is likely spam on this scale. [00:49:56] And typically if the, the score is higher than five or eight, or in some cases, some people said as high as 15, that email is bounced. Well, one of the real big checks as to whether or not this is legitimate email is to check and see. Who is the domain? Does that domain have these special keys that tell us? [00:50:19] Yes, indeed. This did come from us. In other words, in this case did come from Marco Rubio or in the case of my client, it came from their company.com. And is it signed encrypted so that we know that nobody's kind of playing a man in the middle thing, trying to mess things up on us. And they say, okay, well that's a really good score. [00:50:40] So we will, we'll lower that spam score. And, and that's how that game is played. So what by Google doing what it. Talking about doing it's really gonna help out because I have of every company I've checked for email, email deliveries, we've got a, a new customer that is a startup and you know, what do they know? [00:51:02] They they're very narrow. Right? They understand their. Basic technology and their email again, was set up kind of like apparently Senator Rubio's email was set up and, and didn't have these things. And just like this company that I helped this week, they didn't have it set up properly. And, uh, they had experts who supposed experts who had set it up, but both cases, right. [00:51:26] It was outsourced. Yeah. You know how that goes. Now, some Gmail users submitted comments to the federal elections commission and they were criticizing Google's plan cuz they did not want to get more spam. Okay. And there were more than 2,500 comments. You can find them by the way, online, all of the stuff is a matter. [00:51:48] Public record and they call it the docket. And so there's a page out for this particular docket and the commissions through Republicans and Democrat commissioner voted for the order appro Google's plan. I think this is a very, very good deal. And it's really kind of the opposite of what Twitter is planning on doing Twitter has. [00:52:12] essentially announced that it's going to. In the elections. Yeah. So you got Google on the one side saying our hands are clean. We're staying away from this. We don't want anything to do with this. Thank you very much. We love you, but, uh, forget about it. We're just gonna let all the emails. Through, Twitter's saying that it's going to have its wonderful sensors who have been proven right. [00:52:39] Every time he said with his tongue firmly planted in his cheek, and they're gonna have those wonderful sensors that, you know, they're sitting in the basement and, and eating pizza and drinking Coke or red bull. I, I still kinda understand why somebody that's 30, whatever years old needs, energy drinks, you know, come on, come on. [00:53:00] Uh, but anyways, They're they're saying that they, Twitter is going to be the determiner as to whether or not something that is posted on Twitter is correct. Or if it should be censored or if it should be blocked entirely. And they're admitting that they're gonna shadow ban conservative content, they don't like isn't that. [00:53:25] So. Yeah. Uh, that's from the gateway pundit good article. And you'll find it in this week's newsletter. Uh, I think it went out Monday this week and you can follow the link through to these articles on Google and Twitter and the elections or any of the others that we have out there. So stick around, we'll be right back and make sure you sign up. [00:53:46] If you didn't already get that newsletter. Absolutely free. Craig, Peter son.com/subscribe. [00:53:59] I'm not sure a week goes by where I don't hear from a listener saying that somehow Facebook is tracking what they're talking about because all of a sudden ad starts showing up. And they're related to things that they've been talking about. [00:54:16] Meta is the owner of Facebook and Instagram and, and some other things like WhatsApp, which is part of the reason I don't trust WhatsApp, but we've had, I don't know how many complaints from people saying that Facebook is listening in to what they're talking. [00:54:36] And people are kind of wondering, well, wait a minute. Is it listening in on my phone calls? Is it listening when and how? It's a very, very good question. Now Facebook says in a statement that Facebook does not use your phone's microphone to inform ads or to change what in the newsfeed. Some recent articles have suggested that we must be listening to people's conversations in order to show them. [00:55:06] Ads. This is not true. We show ads based on people's interests and other profiled information, not what you're talking out loud about. We only access your microphone if you've given our app permission. And if you are actively using a specific feature that requires audio, this might include recording a video or using in an optional feature. [00:55:30] We introduced two years ago to include music. Or other audio in your status updates. So there it is. There's the official word from our friends over at Facebook. But do you notice there's a little bit of an out in there, right? Facebook does not use your phone microphone to inform ads or change what you see in your news. [00:55:55] Doesn't use your microphone. So there's a study out right now. That is from an X Google engineer. And this article is in the guardian and they are talking about what he found. So, let me explain the background on some of this technology. First, if you are an app developer, if, if you're a developer of any software of any kind you use libraries and these libraries do things like search for a specific set of characters called a string or in search. [00:56:31] Them or move things around or open a connection to another machine. So rather than having implement the whole T C P I P stack and ethernet underneath it and, and all of the operating system work that you'd have to do with all of the interrupts and the buffer fills and reading, toggling. As switches in the hardware, doing all of that sort of stuff. [00:56:52] You just make one library call and say, listen, and you give the port and TA anybody who tries to connect you. It just comes right through. It's all taken care of for you, right? That's what libraries are all about. And they've become much more complex, more recently libraries nowadays can do things like provide you with a full web browser. [00:57:16] Many of the applications that we use on a daily basis, these apps in our phones, particularly, but it's also true with some of the apps on our computers are actually. Just web browsers. They're web browsers that talk to a server out on the internet and yeah, there might be wrapped in various things, but oftentimes if you're trying to pay within an app, it'll go to a third party site. [00:57:44] And part of the beauty of that is. Becomes a, a service to them. They don't have to worry about coding it all up. Right. They don't have to worry about taking your money, keeping everything safe. Am I using really good algorithms here to encrypt it can bad guys hack in? No, no, no. There's, they're just calling this routine that spins up a little web browser. [00:58:07] Inside the application and uses a secure connection to talk to the web server somewhere who cares? Not mine. I'm just the app developer, right? I'm letting you play your farming game or whatever it might be. That makes sense to you guys. So it makes their life much, much easier. Why bother if you've got a website that does everything, why bother coding it all up from scratch in an app? [00:58:34] They don't people don't. Why would. Well, we've seen that again. And again, for instance, look at Microsoft's latest browser out there, edge, not the original edge, but the latest edge, you know how Microsoft is, right. They call it the same thing, even though it's entirely different. Uh, yeah. How many versions of windows where they're like 20 at one point, right? [00:58:56] Different ones or different architectures and just crazy. But now the edge browser is. Built on chromium, which is Google Chrome, which is built on Apple's libraries to manipulate, draw things, et cetera. So you're running your edge browser on your Microsoft windows, computer. You're actually running code libraries. [00:59:21] If you will, from Google and from apple. And that way, if you're developing a browser like edge, you don't have to worry about every little nit bitty thing. That's all taken care of by other programmers who are making a smaller piece of code. Now that's been the whole Unix philosophy forever, by the way. [00:59:42] Instead of having these monolithic applications. That could be just full of bugs and security problems. You just have nice small, easy to maintain, easy to research applications and let other people worry about the little pieces, which is really kind of cool. It's great. Many browsers in fact are based right there on chromium and they modify it around a little bit. [01:00:07] Microsoft added all kinds of spyware to it. Well, it turns out. According to this research from an ex Google engineer that both Facebook and Instagram apps have been taking advantage of this in-app browser technology. And what they're doing is users who click on links inside the Facebook app or inside the Instagram at gram act are actually taken to the webpages. [01:00:39] Using an in-app browser controlled by Facebook or Instagram rather than sending you to your default browser. So if you are using iOS, your default browser might be safari, which is a rather safe. Browser and good for privacy, or you might have decided you wanna use the Chrome browser on iOS or maybe Firefox or brave, or one of dozens of different browsers that are out there. [01:01:10] No, no, it's not gonna use those. It's not gonna use your default browser. It's going to use the in-app browser. And what it's doing with that in-app browser now is here's a quote from him. The Felix Crouse, he's a privacy researcher founded an app development tool that was acquired by Google in 2017. He says, quote, the Instagram app injects their tracking code into. [01:01:37] Website shown, including when clicking on AB ads, enabling them to monitor all user interactions. Like every button that you press, every link you taped, every piece of text that you select or highlight any screenshot you take, any forms, you fill out any user forms, things like passwords addresses, credit card numbers. [01:02:06] Are all seen by the Instagram app? Yes, indeed. So in the statement, of course, uh, medicated that injecting a tracking code, obeyed users preferences on whether or not they allowed apps to follow them. And there was only used to aggregate data before being applied for targeted advertis. Now, this is interesting because according to Crouse, this code injection, uh, was tracked and he was able to look at doing, doing it right for normal browsers. [01:02:42] His test code detected no changes, but for Facebook and Instagram, it finds up to 18 lines of code added by. App into the webpage. So there you go. JavaScript injection and more from our friends at Facebook and Instagram. So they are tracking you, but apparently. They're not listening to your microphone, but they're watching you as you cruise around the web thinking you're using your browser, but no, no. [01:03:18] You're using theirs. Hey, stick around Craig peterson.com. [01:03:24] Cell phone security is something I've talked about for a long time. And you guys know my basics here. If you've been a listener for really any length of time, when it comes to smartphones, we're gonna get into this in more detail, particularly after this raid. [01:03:41] Well, of course everyone's heard, I'm sure about the rate on Trump's property, Mar Lago. [01:03:48] There was something else that happened right. About the same time. And that was representative. Perry Scott Perry was traveling with his in-laws, uh, who are described as elderly. They were on vacation. He's a Republican representative in the house of Congress from Pennsylvania. And he told the Fox news people that three FBI agents approached him, issued him a warrant and demanded he hand over his. [01:04:24] He said they made no attempt to contact my lawyer, who would've made arrangements for them to have my phone, if that was what they wanted. He says I'm outraged. Although not surprised that the FBI. Under the direction of Merrick Garland's DOJ would seize the phone of a sitting member of Congress. My, my phone contains info about my legislative and political activities, personal private discussions with my wife, family constituents, and friends. [01:04:53] None of this is the government's business. Now that's really an interesting point. And, and it brings up the discussion about our smart devices, you know, what should we be doing with our phones and, and what is it frankly, that our phones have in them. Now, just think about that for a minute. Scott Perry rec he, he not recommended. [01:05:21] He mentioned that he had all kinds of records. That were in that phone. You do too. You've got your contacts. Of course. The phone contains information about who you called, where you went, cuz it's got a GPS tracker, but even if GPS is turned off, it's still tracking which cell towers you've connected to. [01:05:43] Uh, we've got all kinds of email in our phones, which are gonna contain business documents, private documents, attorney, client, privilege documents, all kinds of stuff there. And we have the fourth amendment, which protects the right of privacy against unreasonable searches and seizures by the go. Now, in this case, obviously the government got a warrant we could argue about, you know, how legitimate is the warrant and should they have issued it, et cetera. [01:06:16] Right. That that's not what I'm talking about. This is not a political show. In reality. What we're talking about here is the technology. The technology we're using to store this information, this personal information, what should we be using? What shouldn't we be using? How should we use it? Right. All of that sort of stuff. [01:06:38] Well, okay, so we've established that there was not apparently a fourth amendment violation here. There, there might have been, we don't know. We may never know. It doesn't really matter, but if someone gets a hold of your smartphone or your tablet or your computer, what information does it have on there? [01:07:01] And we also have a right under the fifth amendment. against self-incrimination. So if someone's thumbing through our phone, what are they gonna find? People plead the fifth amendment all of the time, because they don't want to get trapped in one of these traps where maybe you don't remember the date. [01:07:24] Right. And all of a sudden you're in a perjury trap because you said something that wasn't true. Well, you know, our, our memories aren't the best, particularly when we're on vacation, we've been drinking a little bit, right. if someone finds your phone, opens it up, someone steals your phone and opens it up. [01:07:44] Someone gets a warrant for your phone and opens it up. What's in there. Now some people have in the past said, okay, what I'll do is I'll just go ahead and I'll wipe my phone remotely and they've done it. Right? The police have had the phone in evidence and in evidence locker and somebody remotely went ahead and wiped their phone. [01:08:04] The police are onto. And what the police have been doing more recently is they put it into a special bag that blocks any sort of signals coming in or out as well as the room. Right. It's kind of a fair date cage anyways, and that way, bad guys, good guys who, if the phones are stolen, they can't remotely wipe them, which is a good thing here, frankly. [01:08:30] But what are we ultimately trying to protect from? That's the question, right? It it's, who's gonna have your phone and what are you trying to protect it from personally? I'm not someone who truly trusts the government. I'm a firm believer in our constitution and our bill of right. Ultimately governments become corrupt. [01:08:52] It happens every time. And even if the whole government isn't corrupt, there's guaranteed to be people within the government, within their bureaucracy, the deep state, if you will, who are out there to get you right. makes sense to you. Makes sense to me. I don't know, but our phones, our smartphones, our computers have a lot of stuff in them. [01:09:14] I've talked on the show before how you should not be taking them to China. If you go to China, because of the evil made. T where they are grabbing your phones. They are duplicating them. Same thing with Russian travelers. Not as much as has been happening in China, but it's happened in Russia, probably a lot now with the whole war thing. [01:09:36] Right. But you shouldn't be taking them because they can be duplicated just like rep Scott. But Scott Perry's phone was duplicated. Now the, the FBI apparently said, well, we're not gonna look through well, why you're duplicating it then. And you know, maybe it's just to preserve evidence. I really don't know, but the bad guys can get at your phone employers if they own your phone can get at your phone and they can get a lot of data out of that. [01:10:06] What do you do? Well, bottom line, if you are traveling internationally, you're gonna wanna make sure to wipe your phone and just bring along maybe a, a basic little flip phone. Uh, cetera. Now there is software that we use. For instance, we use one password and duo in order to keep track of all of our stuff, right. [01:10:31] Our personal information. And. That's the two factor authentication stuff that we use, and we can tell it, Hey, we're traveling out of the country and we will only need these passwords. And it goes ahead and wipes out the password database so that we're not carrying a whole bunch of stuff with us that might be compromised by, uh, a government agency right within what is it? [01:10:54] The USS 50 miles of the border. They can confiscate and examine anything that you have, even if you're not trying to cross the border. and they'll do that at airports. They'll do that at a whole bunch of places. And then you've got the employer side and then you've got the bad guy side. Look at what happened to Khai with the Saudis right here. [01:11:16] He was, uh, you know, a journalist. We could argue that I suppose, but he's a journalist. He is abducted and he is murdered by the Saudis. They get their hands on the phone and they decrypt the. this has happened and it'll happen again. So Apple's done something here that I think is a good step in the right direction. [01:11:40] Apple, of course I've recommended for a long time. Never, ever, ever, ever, ever use Android. Okay. Don't. Use it, Google's using it to track you. You're losing your privacy and the security. Isn't very good. Particularly if your phone's more than three years old, apple has come up with this new lockdown mode on their phones and the lockdown mode is meant for. [01:12:09] People who are really under thumb, you know, people living in Russia or Ukraine, or you name it, Iran, all of these countries that are really out to get their citizens and it it's coming out in iOS. You'll see it there. You probably don't want to use it as a regular person, cuz it does block some of the things you can do, but it also locks it down against these Israeli based companies that have been selling software and hardware to break into cell phones. [01:12:44] So consider iPhones. And if you are one of these people, who's at a high risk consider lockdown mode. [01:12:51] I warned last week about using the ring camera as well as Google's camera. We've got some more news about that today. I was right. A major breakthrough in nuclear fusion and a new toolkit released. Talk about it all now. [01:13:08] Well, quite, quite a time, you know, I, I remember when I first started doing the radio show, uh, 22 years ago, now it started right there year 2000 Y two K and I, I was, uh, wondering, you know, am I gonna have enough stuff to talk about? [01:13:27] and my wife, who was just the most amazing person had been helping me and we subscribed to a bunch of newspapers. Yeah. There used to be newspapers back then. And she went through and was clipping articles that we thought might be good, that people might want to, uh, to hear about. And so she had all. Files. [01:13:49] And we, we subscribe to like four or five different newspapers, including the trashy ones like USA today, just so we knew what was going on out there. We had the financial times and the London times and New York times, and we got just files and files worth of stuff. And didn't take us long to realize, Hey, wait a minute. [01:14:14] There is so much tech news out there and stuff to talk about, uh, that weren't, we don't have to worry about that. So we canceled our subscriptions to all of these different things. I, I have actually a subscription to the New York times still, cuz they gave me a buck a week, which is not a bad deal for the online version because the old gray lady still does have some good text stories. [01:14:39] Some of the other stuff obviously is a problem, but, uh, yeah, tech stories anyways. Now we do a lot of this stuff online, the research, and I put it together and send it out in my newsletter every week. And man, did we have a lot of you guys reading it on Monday was the most, most, uh, red newsletter of mine. [01:15:01] The insider show notes newsletter. Of any of them ever. It was really great. It was like I had a, almost a 50% open rate there within the first day. So that's cool. Thank you guys. And obviously you really value it or you would not have opened that newsletter and click through you. See what I do? Is, uh, you probably know, I appear on radio stations all over the place and I I'm also of course have my own radio show here and elsewhere, and my podcasts, which are on every major podcast platform out there. [01:15:40] And I've been doing this for so long this week. What am I at here? Show? Number, I think it's like 1700. I'm trying to remember weeks. Okay. That's weeks of shows and, uh, we, we have never hit the same stuff twice, which is really rather cool. One of the things I brought up and this was in, uh, a recent show is about. [01:16:09] These ring cameras. And I warned everyone not to use ring and went through the whys. So if you have my newsletter from. A few weeks back, you can just probably search your email box

SQL Server Management Studio (SSMS) version 18 was introduced to support SQL Server 2019 and in this episode of the podcast, we discuss some of the new updates available in SSMS and Azure Data Studio.  We talk about scenarios where we use these tools and give some feedback on the new Azure services they support. The show notes and video for today's episode can be found at https://sqldatapartners.com/2022/04/20/episode-245-tooling-updates. Have fun on the SQL Trail!

Why would you use SSMS/T-SQL over PowerShell (PoSh)? When is T-SQL directly a better option than PoSh? That's a question I ask myself regularly as I see articles and blogs that discuss how to accomplish a particular task using one tool or the other. There is plenty of overlap in the capabilities for each language when it comes to working with SQL Server, so this is a decision I think about regularly. This is especially true if you use dbatools. There also appears to be a bias towards one tool or the other for each individual. Many people traditionally have used T-SQL to accomplish most database tasks, and they tend to always look for a solution with a script in SSMS. Others are excited by PoSh and I have seen plenty of questions on the SQL Server Central forums asking how to structure their code in that language. In both cases, there is no shortage of people that argue that you should use T-SQL instead of PoSh or vice versa. Read the rest of Using T-SQL over PoSh

גיא ואיתן מדברים על כמה הודעות חדשותיות חשובות מאוד שלא קשורות בכלל לפוליטיקה! (וגם לומדים קצת ענייני SQL על הדרך שזה תמיד כיף) קישורים רלוונטיים: מיטאפים בריאקטור בתל-אביב הרצאות בנושא הסמכת Azure Data Engineer Associate Big Data Clusters and Polybase Scale-Out Nodes are Being Retired - The path forward for SQL Server analytics פוסטים של בוב דור: How It Works: Session/SPID (–2) for DTC Transactions How it Works: XEL Health Session and Shutdown SQL Server Mysteries: The Case of the Suspended, Awaiting Command, and Blocked Session SQL Feedback Item: Azure Data Studio - Feature Parity with SSMS so that SSMS can be replaced

Most of us that work with SQL Server likely use either the Windows authentication or a user name and password when connecting to an instance in SSMS or ADS. It's how we've operated for years, and likely will for some time to come. If you connect to Azure cloud resources, perhaps you use some multi-factor authentication (MFA), but that's a minority of us. If this article is a picture of the real world, far too few people are using authentication beyond passwords for many services. While plenty are using fingerprints, patterns, or face recognition on a mobile device, that's usually the extent to which they actually go beyond a password. I've actually started to see people using PINs on laptops instead of a password, which feels like a step backward. Read the rest of Getting Beyond Passwords

Harga CPO mencapai harga tertinggi sejak 2019, bahkan memiliki kemiripan dengan trend di 2011. Tentu saja dengan meningkatnya harga kelapa sawit akan membawa kabar baik untuk emiten perkebunan seperti AALI, LSIP, SSMS, atau DSNG. Apakah hal ini membawa sentimen yang baik untuk harga saham perusahaan perkebunan? Apakah strategi "earnings play" boleh diterapkan? Hal ini kami diskusikan di podcast Academic meets practitioners episode 4.

Always Encrypted in SQL Server 2019 is now in its second iteration which has added the ability to do pattern matching searches on encrypted data with the use of a technology called secure enclaves. In this episode with Mladen Prajdić, we'll take a short look at what's needed to make this work, how it works, and how it can benefit your organization for more secure data practices.[00:36] Introduction[01:49] What always encryption is important[03:29] Always Encrypted v1 overview[05:00] Always Encrypted v2 overview[05:35] What is a secure enclave[06:45] Always Encrypted V2[11:05] Always Encrypted V2 - do in-place encryption[14:31] Always Encrypted V2 - do rich computations, pattern matching[16:26] What is supportedAbout Mladen Prajdić:Mladen Prajdić is a Data Platform MVP from Slovenia. He's been programming for 20 years, developing different types of applications in .Net (C#) and SQL Server, ranging from standard line-of-business, image-processing applications to high performance and IoT applications. He's a regular speaker at various conferences and user group meetings, really likes to optimize slow SQL statements, analyze performance, and find unconventional solutions to difficult SQL Server problems. In his free time, he also develops a very popular add-in for SSMS, called the SSMS Tools Pack (www.ssmstoolspack.com).About MVPs:Microsoft Most Valuable Professionals, or MVPs, are technology experts who passionately share their knowledge with the community. They are always on the "bleeding edge" and have an unstoppable urge to get their hands on new, exciting technologies. They have very deep knowledge of Microsoft products and services, while also being able to bring together diverse platforms, products and solutions, to solve real world problems. MVPs make up a global community of over 4,000 technical experts and community leaders across 90 countries/regions and are driven by their passion, community spirit, and quest for knowledge. Above all and in addition to their amazing technical abilities, MVPs are always willing to help others - that's what sets them apart. Learn more: https://aka.ms/mvpprogram

Stotřiatřicátý díl seriálu, který každý měsíc mapuje to nejzajímavější, co se událo v kosmonautice. Obsah dílu: 0:00 - Představení témat 1:04 - Start mise SSMS 3:49 - Statický zážeh FSB-1 motoru na tuhé pohonné látky pro SLS 5:50 - Kosmonautika v kostce 17:16 - Start mise Starlink 1-11 18:47 - Dění na Boca Chica (skok Starship SN6, destrukce SN7.1, příprava SN8) 22:30 - Poděkování za sledování a rozloučení

Successo del vettore Vega dell’Esa, che ha inviato in orbita 53 piccoli satelliti inaugurando il trasporto europeo di cubesat in rideshare con il nuovo sistema SSMS

Monthly blog updatesThe Premier Field Engineer team for Dynamics 365 published a monthly update to our blog.PFE Dynamics Monthly Updates On-demand sessions from BizApps Summit and Microsoft BuildBusiness Applications Summit 2020 Microsoft Build Dynamics 365 Field Service mobile appOffline mode with sync statusView and manage bookings and work ordersCapture service notes including photos, video, audio, signatureCreate time off requestsDynamics 365 Field Service mobile app Manage Portals from Power Platform admin centerManage Portals from PPAC Power Virtual Agents - WebinarCall Q&A Maker from Power Virtual AgentsAAD authenticationAccess Graph APIPower Virtual Agents Webinar Enhanced Calendar ControlFind records quicklyManage records easilySupports personalization settingsFile and Image data types Dynamics 365 Connector is deprecatedDeprecated for Flows, Logic Apps, Canvas AppsUse Common Data Service (Current Environment) insteadIf not possible due to constraints, use Common Data Service connectorDynamics 365 Connector is deprecated CoE Starter KitMonitor (oversight of tenant resources)Govern (drive actions through insights)Nurture (learn about your community)CoE Starter Kit TDS Protocol endpoint for CDSRead-only access to CDS for reporting and analyticsCurrent support for SSMS and Power BizAppsRun TSQL queries against CDS dataTDS Protocol endpoint for CDSUpdates and ReleasesDynamics 365 Online Updates Online Updates Dynamics 365 On-Prem Updates OnPrem Updates Portal Updates Portals

Vicky Harp joins Scott Hanselman to show how Azure Data Studio combined the simple and robust SQL query editing experience of tools like SSMS with the flexibility and collaboration of Jupyter Notebooks. The November 2019 release of Azure Data Studio included SQL Server 2019 Guide as a Jupyter Book, which provides a richer troubleshooting experience.[0:04:00] - DemoAnnouncing Jupyter Book support blog postDownload and install Azure Data Studiomicrosoft/azuredatastudio GitHub repoLatest news on the SQL Server BlogCreate a free account (Azure)

Vicky Harp joins Scott Hanselman to show how Azure Data Studio combined the simple and robust SQL query editing experience of tools like SSMS with the flexibility and collaboration of Jupyter Notebooks. The November 2019 release of Azure Data Studio included SQL Server 2019 Guide as a Jupyter Book, which provides a richer troubleshooting experience.[0:04:00] - DemoAnnouncing Jupyter Book support blog postDownload and install Azure Data Studiomicrosoft/azuredatastudio GitHub repoLatest news on the SQL Server BlogCreate a free account (Azure)

Vicky Harp joins Scott Hanselman to show how Azure Data Studio combined the simple and robust SQL query editing experience of tools like SSMS with the flexibility and collaboration of Jupyter Notebooks. The November 2019 release of Azure Data Studio included SQL Server 2019 Guide as a Jupyter Book, which provides a richer troubleshooting experience.[0:04:00] - DemoAnnouncing Jupyter Book support blog postDownload and install Azure Data Studiomicrosoft/azuredatastudio GitHub repoLatest news on the SQL Server BlogCreate a free account (Azure)

Vicky Harp joins Scott Hanselman to show how Azure Data Studio combined the simple and robust SQL query editing experience of tools like SSMS with the flexibility and collaboration of Jupyter Notebooks. The November 2019 release of Azure Data Studio included SQL Server 2019 Guide as a Jupyter Book, which provides a richer troubleshooting experience.[0:04:00] - DemoAnnouncing Jupyter Book support blog postDownload and install Azure Data Studiomicrosoft/azuredatastudio GitHub repoLatest news on the SQL Server BlogCreate a free account (Azure)

Vicky Harp joins Scott Hanselman to show how Azure Data Studio combined the simple and robust SQL query editing experience of tools like SSMS with the flexibility and collaboration of Jupyter Notebooks. The November 2019 release of Azure Data Studio included SQL Server 2019 Guide as a Jupyter Book, which provides a richer troubleshooting experience.[0:04:00] - DemoAnnouncing Jupyter Book support blog postDownload and install Azure Data Studiomicrosoft/azuredatastudio GitHub repoLatest news on the SQL Server BlogCreate a free account (Azure)

Do you feel like you have the knowledge needed to achieve your success, but can't seem to actually get rolling or sustain your progress? There's a good chance the missing link is learning how to properly manage your energy. If your emotional and psychological energy gets too out of whack, you're not going to be able to think well, and you're not going to be able to act well. There's a bunch of practical common sense stuff you can do to aid in this, but how do you get yourself to consistently do THOSE things? This is where willpower and energy work comes into play. Until you can internally manage your own energy like how you can manage your own musculature, you are not in control of yourself or your outcomes. Watch this video and fix that. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Making sure you've got a clear plan around your internet, computer, and phone usage can have a major impact on your success with quitting porn. The trick is to make sure that while you employ some safety measures to prevent poor split second decisions, you don't ultimately RELY on these things to be successful. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

The reality is that for most guys, quitting porn is a process that involves relapse. The key is to make sure that with every single relapse you grow stronger. You should not be failing in the same way twice. Watch this video and learn how to turn any slip into an experience that makes you stronger. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Trying to figure out the best way to quit porn while in a relationship can be very tricky. Often, I see guys go about this process in a way that ultimately just causes more pain for their partner and more failure on their end. If you want to learn the best way to reboot your brain from porn while in a relationship, then this video is for you. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

In order to maximize your chances at reboot success, you must learn how to cultivate and utilize BOTH positive AND negative emotion. If you want to create sustainable change, not only do you need the pull of positive desires, but you ALSO need the push of appropriately aligned /negative/ emotion. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

We've talked repeatedly about the importance of self-talk in the process of quitting porn. However, if you don't understand the psychology behind your emotional reactions, then it can be very difficult to figure out what to actually say to yourself. By understanding the link between lust, your identity, and your limbic brain's hunger for "status," you'll gain access to an incredibly powerful self understanding which you can use to master yourself and quit porn. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Trying to quit porn from a place of pure rationality has serious limits. If you want to take things one step deeper and unlock an new level of psychological motivation, then you need to learn how to quit porn from a place of identity. Doing so helps you overcome the psychological battle between emotionality and rationality. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

The puppet-master of lust attaches strings of tension to your eyes, mind, emotions, and body - looking to push and pull you into acting out in inauthentic ways. If you wish to break free, you must learn how to cut these strands of tension and establish new intentional patterns of action. In this video I teach you the practical internal steps for making this happen. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Guys tend to get stuck at different points in their reboot journey. Whether you are struggling to just string a few clean days together or are trying to figure out how to make a long streak a permanent one, I'll tell you what you need to do to reach the next level in this video. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

I believe the old-school mindset of spiritual warfare that saw angels and demons constantly among us was an extremely powerful filter for modifying and aligning our behavior. Sadly, there were some issues with this approach that has caused us to mostly abandon this perspective. In this video I update the mindset of spiritual warfare for the modern mind so you can tap into a level of discipline and aggression that will allow you to finally break free from the demons in your life. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

A properly kept journal is one of the most powerful self-development tools that exists. The problem is that people either write a ton without ever writing the things that will create personal change, or they don't know what to write so they don't write at all. I'll tell you exactly what to do in this video to get the most out of a journal. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Think porn is healthy for you? Think again. It's holding you back more than you could have ever imagined. Learn why you should be kicking this habit and developing sexual self-mastery. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

You can't just quit porn and do nothing else. You need to engage in habit replacement. However, since porn is such a potent coping mechanism, you can't just replace it with just ONE thing. Instead you need to develop a holistic lifestyle shift that allows you to live a life you DON'T want to escape from! Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Using porn is an act of escapism. You either escape because you aren't living a life you want to be engaged in, or you don't know how to properly manage yourself (or both). This episode I talk about how to fix that problem. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

If you want to maximize your ability to permanently quit porn and maximize your mental rewiring, then you'll want to do a "hard mode" reboot. But since hard mode is well... hard, you might need some convincing. That's what this episode sets out to do. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

Porn itself is a deception designed to trick you into thinking you are having a real sexual experience when you are not. Once you recognize it's bad for you and want to quit, you must deceive yourself on an even deeper level to continue to use it. The solution is to learn how to speak to yourself honestly. Download the Reboot Regimen http://www.universalman.com/the-reboo... Join the Universal Man Vanguard community through Patreon http://www.patreon.com/universalman Follow Universal Man on social media Twitter: http://www.twitter.com/aUniversalMan Instagram: http://www.instagram.com/aUniversalMan Facebook: http://www.facebook.com/aUniversalMan

אז מה היה לנו? באג ב-SSMS 18.1 יצא SSMS 18.2 (אולי הבאג נפתר…) אם היית SQL Server, איזה חיה היית? רשימת הבנקים ועוזריהם טיפים לניצחון מאריה מליניאק דרך קיצור לכתוב פקודת MERGE מה קורה כשמשתמשים בפרמטר עם LIKE?

How many people do live in our guest's country? What can give you a valuable perspective? How can you convince your boss to upgrade SQL Server to a newer/latest version on enterprise platform?Mladen Prajdić is a Data Platform MVP from Slovenia. He's been professionally programming for over 15 years, developing different types of applications in .Net (C#) and SQL Server, ranging from standard line-of-business, image-processing applications to high performance and IoT applications. He's a regular speaker at various conferences and user-group meetings, really likes to optimize slow SQL statements, analyze performance, and find unconventional solutions to difficult SQL Server problems. In his free time, he also develops a very popular add-in for SSMS, called the SSMS Tools Pack (www.ssmstoolspack.com).This talk has taken place during SQL Saturday #782 Slovenia conference in Ljubljana, on 8th December 2018 (Saturday).Interviewers: Kamil Nowinski & Michal Sadowski.

How many people do live in our guest's country? What can give you a valuable perspective? How can you convince your boss to upgrade SQL Server to a newer/latest version on enterprise platform?Mladen Prajdić is a Data Platform MVP from Slovenia. He's been professionally programming for over 15 years, developing different types of applications in .Net (C#) and SQL Server, ranging from standard line-of-business, image-processing applications to high performance and IoT applications. He's a regular speaker at various conferences and user-group meetings, really likes to optimize slow SQL statements, analyze performance, and find unconventional solutions to difficult SQL Server problems. In his free time, he also develops a very popular add-in for SSMS, called the SSMS Tools Pack (www.ssmstoolspack.com).This talk has taken place during SQL Saturday #782 Slovenia conference in Ljubljana, on 8th December 2018 (Saturday).Interviewers: Kamil Nowinski & Michal Sadowski.

SSRS is great -- it comes with SQL Server, provides quite a bit of functionality, but sometimes doesn't always perform well. In this episode we welcome back Bert Wagner as he chats with us about some of the ways he has improved his under-performing reports. Our conversation focuses on SSRS itself -- we are going to assume you can tune the queries in SSMS from previous episodes or the interwebs. What do you think of our list? Did we miss one? Let us know! The show notes for today's episode can be found at http://sqldatapartners.com/2018/09/27/episode-148-high-performance-ssrs. Have fun on the SQL Trail!

While the ability to shrink data files is available in SSMS, the conventional wisdom is don't do it.  I was extremely curious when I read an email from our guest suggesting the opposite.  In this episode we explain the 'real world' and discuss why you might consider shrinking files and the how it will affect you.  Nigel has a great sense of humor and I know you will enjoy today's episode. The show notes for today's episode can be found at http://sqldatapartners.com/2018/05/02/episode-133-shrinking-files/. Have fun on the SQL Trail!

Today in the show:   SSMS Unuseful Tips Why Database Continuous Integration is Challenging? Monitoring Page Splits

People have strong feelings about SQL Server Management Studio: they love it AND they hate it. In this week's episode, I talk about why people have such conflicting feelings about SSMS, and how to work it all out.

The twentieth episode is up, a.k.a the “hammer and sauce” episode! We talk about he death of the SCCM model, a new Data Management Gateway release, a new Data Migration Assistant release, that Airwatch now can do Chromebook management, expectations of Windows Server 1709 at Ignite, the new SSMS 17.2 release and data masking shenanigans. A shock-full episode indeed!We are now on iTunes, or, sort of, we’re still ironing out the quirks.As always we gladly accept tips and criticism, as well as ideas for content for us to cover. Just tweet me (@arcticdba) or Simon (@bindertech)! See acast.com/privacy for privacy and opt-out information.

In this show, we talk about: Our sessions at the GroupBy conference Another difference between the old and new Cardinality Estimators Lightweight Profiling in SQL Server 2016 SP1 Sign Matan's wife is married to a DBA SSDT 2017 A cool SSMS shortcut

HoH Radio LIVE Broadcast!!!Single Sisters let's talk!!! March 2nd - Brookland Baptist - Northeast - Come learn and share with women of all ages and backgrounds... Everything you wanted to know about being single and didn't know who to ask! Plus live Music from TiffanyJ...food and networking! First Thursday Single Sessions 6:30-9pmIf you can't attend, watch Facebook Live on @Spiritually Minded Singles Ministry at Brookland Baptist Church or the House of Harley Radio page!!!!https://play.google.com/store/apps/details…#hohradio #sms #ftss #hohproductions #brooklandbaptistne

HoH Radio LIVE Broadcast!!!Single Sisters let's talk!!! March 2nd - Brookland Baptist - Northeast - Come learn and share with women of all ages and backgrounds... Everything you wanted to know about being single and didn't know who to ask! Plus live Music from TiffanyJ...food and networking! First Thursday Single Sessions 6:30-9pmIf you can't attend, watch Facebook Live on @Spiritually Minded Singles Ministry at Brookland Baptist Church or the House of Harley Radio page!!!!https://play.google.com/store/apps/details…#hohradio #sms #ftss #hohproductions #brooklandbaptistne

Jeffrey Snover from Microsoft on open source PowerShell   Slack Chatroom We are the live-podcast channel   Interview The announcement YouTube video with PowerShell on Linux demos The PowerShell Repository New PowerShell learning info   Chatroom Highlights ## What about ARM? is ## for questions? ## @halr9000 did you find a goat to come on the show as a special guest? ## jsnover needs more edge in his life ## PowerShell is dating bikers. ##Jeffrey, where were y'all hiding Ken Van Hyning all these years? ##jeffrey where did you get inspired to bulid powershell? ok, if you have q’s, then as sqlvariant has shown, please add ## to the front! ## Is that all? ## ARM support? ## SSHRM ## remember OneGet and the legal team? :stuck_out_tongue: juneb: prefix questions w/ ## ## Please tell us more about the governance model. ## how did legal allow MIT license? ## Were there lessons learned working with legal/folks unfamiliar with OSS?  I know Chef offers a webinar for legal folks, it seems like it would be helpful for large orgs like Microsoft to help sell and teach this ## Will the roles of team members change? ## How can I get paid to work on open source powershell? :-P ## I second Rich's question: what's with the release timing? ## When will core help be open-sourced? ## wait, is that why CIM? ## Can we get an explaination on the Powershell RFC Process for changes ## ## since we're now taking comments, is the team aware that some nuget packages (like Microsoft.Management.Infrastrucutre) are missing XML doc files? ## For bugs should we still use uservoice, connect, or github issues now? ### We noticed version 6 on PowerShell for Linux, when can we expect ver 6 on Winders ### ### is there any PowerShell features that don't work in PS Core? ## For Linux services would you expect to use Get-Services or a new cmdlet like Get-LinuxServices? ##This is why we need Out-DataTable to be in PowerShell core!!  :slightly_smiling_face: ## when might we see remoting to Azure/Office 365 tenants from Mac/Linux? ## will powershell run hurd ###I need mah winforms in Linux, can we get System.Windows.Forms ports to linux plz ## my thing about XML doc files ## yeah but what about rasbian ## Raspbian! ## Yes, it was around Raspbian/linux ## NEED ARM! ## need xml doc files in all the nuget pkgs ## Sooooo, how do I get PowerShell on my Chromebook? ## Do we use UserVoice or GitHub issues? ## will we be able to organize assets in our modules for different platforms?  or will module authors be required to manage that on their own? ### will script block loging work with syslog? ## Well SQL Server is on Linux so they have a business case :champagne: ## will we be able to organize assets in our modules for different platforms?  or will module authors be required to manage that on their own? ## what is next for Snover and team?  e.g. Drawbridge, rewriting kernel, on-premise DSC mgmt system ## e.g., right now I have to manually load 32 or 64 bit native dlls in my modules; will I need to manually load linux vs windows as well? ## any thoughts on deeper PS integration with Python etc, beyond strings? ## will get a restapi  for AD, and will we get Join domains cmdlets for powershell on mac ## OOH. DSC using powershell on linux, instead of the separate DSC package??? ##Have there been any advancements in DSC on Linux? ## DSC will we see a revamp of the DSC for linux to kill off python ## ##Are you aware of any other vendors writing psmodules for Linux right now? ##Snover, was PSDirect one of the drivers for PSnix?  Being able to manage Win/Nix on Nano server Hyper-V with one language ##or MAC? ## will we see modules to manage bind9 or dhcpd or other services or MYSQL? ##Forgive me if this was already answered, but can we now powershell remote to Linux? ## do we require !#/usr/local/bash ## will there ever be a means to #! powershell in a script ## what about text enconding ## what can we expect in the next 6, 12 months ## Yes, upcoming roadmap?? ## what about an update to the conhost terminal in windows ? Will it support more then 256 colour? ## ONE MORE!  What's next for Snover and team? ##Ask about Ken Van Hyning (SQL Tools team) where were they hiding him?!?! ## What question is he up to? One thing on an island? ## Oh you guys don't ask the last question any more? ###Whats next, will they take DIR away from us? ## As of PS5+, does Write-Host write to stream or no? ###Found it ## Is the redirect operator modified in linux or does it behave as per normal? ## Wouldn't this be the role of .NET Standard?? jonwalz uploaded a file: Pasted image at 2016-08-18, 8:51 PM https://powershell.slack.com/files/jonwalz/F22RNF6UQ/pasted_image_at_2016_08_18_08_51_pm.png and commented: Anyone else seeing PowerShell as the top trending topic on Facebook? https://github.com/PowerShell/PowerShell/blob/master/docs/community/governance.md https://github.com/PowerShell/PowerShell/tree/master/src/Microsoft.Management.Infrastructure.CimCmdlets Governance model: https://github.com/PowerShell/PowerShell/blob/master/docs/community/governance.md btw PowerShell going open source is trending on facebook :)  https://www.facebook.com/topic/Windows-Powershell/109314949087498?source=whfrt&position=2&trqid=6320345599904818250 @foxde https://github.com/PowerShell/PowerShell/blob/master/docs/installation/windows.md#msi halr9000 uploaded a file: Untitled https://powershell.slack.com/files/halr9000/F22S025RC/-.pl https://www.reddit.com/r/PowerShell/comments/4yb5yb/who_is_the_mystery_guest_tonight_and_what_might/d6mgixw https://stedolan.github.io/jq/ @whatevergeek https://developer.microsoft.com/en-us/windows/iot/getstarted I liked this thread in hacker news: https://news.ycombinator.com/item?id=12314653 - nice demonstration of why, even if you *can* use text, you might prefer objects Linus actually is cool with systemd:  https://linux.slashdot.org/story/15/06/30/0058243/interviews-linus-torvalds-answers-your-question https://gist.github.com/Jaykul/15a6982aad9b6197a537f1ce5a786723 regarding Aliases there is an open issue here at  https://github.com/PowerShell/PowerShell/issues/929 here's the PR @jsnover mentioned https://github.com/PowerShell/PowerShell/pull/1901 I've suggested this as a resolution https://github.com/PowerShell/PowerShell/issues/929#issuecomment-239697927 ARM!!! https://media.giphy.com/media/HJN39I1q6EBKE/giphy.gif @itadder the hard part is that other than ASP.NET http://ASP.NET Core there is no presentation stack for .NET Core.  VSCode gets a pass because it uses Electron / Chromium / Html / JS http://electron.atom.io/apps/ And you can actually write Electron using C#:  https://github.com/kexplo/electron-edge https://youtu.be/2WZwv7TxqZ0 possible alternate to ogv, don't know if works on the new stuff yet https://github.com/dfinke/PowerShellSlackathon/tree/master/ExportDataTable http://redhatloves.net/ also https://github.com/Pash-Project/Pash/issues/429 Is .ps a TLD?  RedHat should get RedHatLoves.ps http://RedHatLoves.ps First book should be written in the open on https://www.gitbook.com/ https://github.com/PowerShell/PowerShell/issues/1103 https://github.com/PowerShell/PowerShell/issues/1390 https://cloud.google.com/tools/powershell/docs/ dereksonthejob  http://prnt.sc/c7jmuo Explor .NET API Catalog:  http://apisof.net/catalog/System dereksonthejob uploaded a file: Screen Shot 2016-08-18 at 10.41.10 PM https://powershell.slack.com/files/dereksonthejob/F22SKJQLR/screen_shot_2016-08-18_at_10.41.10_pm.png and commented: here we go @jonwalz Can you include in the show notes that SQL Server got 3 new cmdlets, and people can get the new SqlServer module by installing SSMS 16.3 http://sqlps.io/dl dereksonthejob uploaded a file: @r_keith_hill https://powershell.slack.com/files/dereksonthejob/F22SRLC02/screen_shot_2016-08-18_at_10.41.10_pm.png and commented: I have it now, thx to @itadder https://github.com/PowerShell/PowerShell/issues/1621 https://github.com/PowerShell/PowerShell/issues/706 https://github.com/PowerShell/PowerShell/issues/1273 Oh, here's a video we did last month if people need to catch up to speed on what's going on with SQL PowerShell in 2016: https://youtu.be/rc6lwiTE9GI http://unix.stackexchange.com/questions/695/where-does-mac-os-x-come-from r_keith_hill uploaded a file: RE line endings: https://powershell.slack.com/files/r_keith_hill/F22SHCYDB/pasted_image_at_2016_08_18_08_54_pm.png and commented: Just a LF but concerns me the default encoding is unicode for out-file still

Did you know SSMS can make sounds? We start the show with some interesting things you might do with it. In addition, we talk about: SQL Server on Linux Hadoop fundamentals SQL Server 2016 eBook Changes to auto update statistics in SQL Server 2016 New SQL Server 2016 database level configurations Distributed Availability Groups - what is it and use cases How AWS storage works Items mentioned in the show: Guy Glantser - Hadoop Fundamentals Denny Cherry - How to make SSMS beep when a query is done? Introducing Microsoft SQL Server 2016: Mission-Critical Applications, Deeper Insights, Hyperscale Cloud, Preview 2 Erik Darling - Changes to auto update stats thresholds in SQL Server 2016 Brent Ozar - SQL Server 2016: The Death of the Trace Flag Distributed Availability Groups

For our 10th episode of the #SSMS we have a special one for all the Techno heads and its packed full of exclusive bits from the man him self! Give it up for Nurve. If you're familiar with Dubstep and Grime you will no doubt be familiar with another of Nurve's aliases Demon. The head honcho of Macabre Unit proves he is not just a 140 beat machine as he showcases his unique blend of Techno music in this absolutely pounding mix of kick drums and reese' Best served: Long dark corridor in a dingy club surrounded by confusion. Follow Nurve: FB: https://www.facebook.com/nurvemusic?fref=ts TW: https://twitter.com/NurveMusic Macabre Unit: @macabreunitdigital Track list: https://soundsmithstv.wordpress.com/2015/01/29/techno-heads-this-one-you-ssms10-nurve/

Climb into the mind of @mercy-one. After a productive 2014 releasing track with @chord-marauders & @deepheads he is back with a tasty 2015 preview mix of what you can expect in his dj sets as well as a few exclusive productions from the man him self. Respect Mr Mercy! Follow Mercy: SC: @mercy-one FB: www.facebook.com/mercydnb IG: www.instagram.com/joshruhener Tracklist: https://soundsmithstv.wordpress.com/2015/01/20/tracklist-soundsmiths-mix-series-09-mercy/

Next up giving us the mix treatment we have @shadesrecs own Herse. Following on from his latest mix for Trailer TV he delivers us an extremely diverse collection of records, he's also thrown in some sound bytes for good measure. Trust us when we say there is something in here for everyone. Follow Herse FB: www.facebook.com/djherse?fref=ts SC: @trinomic Tracklist To be completed by you! If you know one of the tracks leave a comment and we will update the list as and when. Prize to anyone that can name the whole track list. 1. 2. 3. 4. Charizma & Peanut Butter Wolf - My World Premier 5. 6. 7. Kazi - A.V.E.R.A.G.E 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26

@Dexta aka Mr @diffrent provides us with mix number seven. The drum and bass label bossman steps out of his comfort zone and goes exploring through the lower BPMs. From Prodigy to Falty DL this mix is deffinetly something a bit .... Diffrent. Follow Dexta FB: www.facebook.com/dirtydexta?fref=ts SC: @dexta TW: www.twitter.com/dirtydexta Tracklist To be completed by you! If you know one of the tracks leave a comment and we will upload the list as and when. Prize to anyone that can name the whole track list. Tracklist Deft - Every Single Thing Deft - A Little Kiss Evy Jane - Sosoft Fearful & Skekk - Part1 Prodigy - 3 Kilo’s Dexta & Mauoq - The Prequel (Chump Dub) Nuage & Gerwin - Slow Moves Machinedrum - U Still Lie Eprom - Cloud Leanmixx Arca - 2 Blunted Deft - Dionysus Park Addison Groove - Button Moon Sina - All Alone (Fybe:one Remix) Arca - Self Defense Two Fingers - Razorback Addison Groove - I Go Boom Machinedrum - She Died There FaltyDL - New Haven Eprom - Hurricane Eprom - Pentatonic Dust

Mix #05 comes from Brooklyn based DJ @thayellowrkel This ones a roller coaster... Follow @thayellowrkel Tracklist Suede - Aerosol Can (Major Lazer Remix) INSPEKTAH - Party Girls (Last Night) Is It Tomorrow - And Me KazammTheMan - Ocean in the Sky Shy Glizzy - Awwsome Mike T - Patience khadisma - endless.love Minnesota Fatz - Splash Waterfalls (Luda flip) Enchufada – Cabo-blanco-santo-diablo CARMACK - JAYZ RIP DJ RASHAD - She A Go Tommy Jacob - Better StarRo - Are you that Somebody (RIP AALIYAH) Eddy Braveaux - La Belle Fleur Missy + Aaliyah (RIP) - Best Friends :’( DJ Chose - Is it Mine (Nadus Remix) - Koenraad OW Edit sloslylove - Pretend We’re Stars Mos Def - The Panties (second half of the instrumental) Oddisee - After Thoughts Gautier - Mourning (ft. Olivia Louise) Drake - 0 to 100 (second half of it)

New Zealand Duo 'Morrison' display some lovely bass music with this deep mediative mix for #04 of our mix series. Thank you Sir's! Follow Morrison: FB: www.facebook.com/morrisonnz SC: @morrison-3 Track list @Commit - Maccha Sunrise @congi-music - Somnium @kromestar & @jay-5ive - Words @vaunuk & @animaimusic - Taking Over (@anex_dub Remix) Mala - Level Nine @jtparks & @demondubz - Rhodes To Nowhere @truthdubstep & @duttyranksmusic - Losing You @Anex_dub - Phobic @swindleuk - Forest Funk 040 - Minors @silkie - Beauty @morrison-3 - Sunny Side @nanobytedubs - Lost Time (@congi-music Remix)

Rebirth records regular and good friend Corrado Bucci has hand crafted this brilliant 'World Cup Warm Up' mix packed full of sunshine tracks. Italian resident Corrado has recently been tearing up the clubs of Miami & Ibiza and has been receiving air play from @gillespeterson on his BBC Radio 6 show amongst others. Can't afford that ticket to Brazil to experience the greatest show on turf? Then get your deck chair out crack this on the speakers and get your tropical lounge on! Follow Corrado Bucci: FB: www.facebook.com/corradobuccimusic TW: @CorradoBucci_IT SC: https://soundcloud.com/corrado-bucci Track List: Mighty Ryeders - Evil Vibrations Nu Tropic - Moonlight (Original Mix) Moodymann - I Got Werk (Original Mix) @mr-scruff - We Are Coming (Original Mix) @rainertrueby - Welcome To Our World (Original Mix) @toddterje - Alfonso Muskedunder (Original Mix) Butch & C. Vogt - The Infamous (Robytek Vs Shield Re-Edit) @porticoquartet - Line (Original Mix)

#02 of the Soundsmiths mix series features this multigenre journey from Brighton resident & @extrasolarmusic co owner Noemauk. Follow Noemauk S: @noemauk F: facebook.com/NoemaUK T: @NoemaUK Follow Soundsmiths S: @soundsmiths F: facebook.com/soundsmiths T: @soundsmithstv Tracklist 1. Stepping Stones - Jafu [Free DL www.soundcloud.com/Chord-Marauders] 2. Vancouver - Martyn [3024 Records] 3. So Clear - Commodo [Deep Medi Music] 4. What We See - Arp XP [iM:LTd] 5. Amazon Basin' - Boeboe [Free DL www.soundcloud.com/Xlr8r] 6. Irenidae - Fybe:one [Free DL www.soundcloud.com/Xlr8r] 7. Gold Dayz (Maribou State Remix) - Ultralista [Free DL www.soundcloud.com/MaribouState] 8. This Shit (Prod. Mr Phist) - SamB [Dub] 9. Waywood - dBridge [Free DL www.soundcloud.com/dBridge] 10. Before (Deadboy Remix) - Scuba [Hotflush Recordings]

#01 of the Soundsmiths mix series features a beautifully put together mix from Belgium resident Jolta. Follow Jolta @joltadubstep Follow Soundsmiths @soundsmiths Tracklist @clearlightdub - Muddy [Subaltern Records] 
@Ipman - Signal Motion [Osiris Music]
 @d-operation-drop & @geode-1 - Angkor [Subaltern Records]
 Asylum - Zero Gravity VIP [Uprise Audio]
 LAS - Malfunktions [System Music]
 @menikmusic ft. zelda-henrietta-marshall & Emma Lin - Contact (@anex_dub Rethink) [Free Download]
 Barom - Two Worlds [Midnight City Records] @jussb - Braata [dub] @crisfern - She Is [Free download]
 Kryptic Minds - 1000 Lost Cities [Black box] 
@commit - Maccha Sunrise [Deep Heads]
 @bluetech - Basement Dub [Basement Dubs EP]
 @kaijudub - Unity Dub [Lion Charge]
 @ludovicoeinaudi - Walk (@Phaeleh Remix) [In a Time Lapse (The Remixes)]
 @congi-music - Lost Path [Tidal Fragments EP Chord Marauders]