POPULARITY
2 episodes with relora
2 episodes with relora
The Latent Space crew will be at NeurIPS on Tuesday! Reach out with any parties and papers of interest. We have also been incubating a smol daily AI Newsletter and Latent Space University is making progress.Good open models like Llama 2 and Mistral 7B (which has just released an 8x7B MoE model) have enabled their own sub-industry of finetuned variants for a myriad of reasons:* Ownership & Control - you take responsibility for serving the models* Privacy - not having to send data to a third party vendor* Customization - Improving some attribute (censorship, multiturn chat and chain of thought, roleplaying) or benchmark performance (without cheating)Related to improving benchmark performance is the ability to use smaller (7B, 13B) models, by matching the performance of larger models, which have both cost and inference latency benefits.Core to all this work is finetuning, and the emergent finetuning library of choice has been Wing Lian's Axolotl.AxolotlAxolotl is an LLM fine-tuner supporting SotA techniques and optimizations for a variety of common model architectures:It is used by many of the leading open source models:* Teknium: OpenHermes, Trismigestus, CollectiveCognition* OpenOrca: Mistral-OpenOrca, Mistral-SlimOrca* Nous Research: Puffin, Capybara, NousHermes* Pygmalion: Mythalion, Pygmalion* Eric Hartford: Dolphin, Samantha* DiscoResearch: DiscoLM 120B & 70B* OpenAccess AI Collective: Manticore, Minotaur, Jackalope, HippogriffAs finetuning is very formatting dependent, it also provides prompt interfaces and formatters between a range of popular model formats from Stanford's Alpaca and Steven Tey's ShareGPT (which led to Vicuna) to the more NSFW Pygmalion community.Nous Research MeetupWe last talked about Nous at the DevDay Recap at the e/acc “banger rave”. We met Wing at the Nous Research meetup at the a16z offices in San Francisco, where they officially announced their company and future plans:Including Nous Forge:Show NotesWe've already covered the nuances of Dataset Contamination and the problems with “Open Source” in AI, so we won't rehash those topics here but do read/listen to those if you missed it.* Axolotl GitHub and Discord* The Flan paper and dataset* StackLlama model and blogpost* Multipack paper* Our episode with Tri Dao* Mamba state space models - Tri Dao and Albert GuTimestamps* [00:00:00] Introducing Wing* [00:02:34] SF Open Source AI Meetup* [00:04:09] What is Axolotl?* [00:08:01] What is finetuning?* [00:08:52] Open Source Model Zoo* [00:10:53] Benchmarks and Contamination* [00:14:29] The Case for Open Source AI* [00:17:34] Orca and OpenOrca* [00:23:36] DiscoLM and Model Stacking* [00:25:07] Datasets and Evals over Models* [00:29:15] Distilling from GPT4* [00:33:31] Finetuning - LoRA, QLoRA, ReLoRA, GPTQ* [00:41:55] Axolotl vs HF Transformers* [00:48:00] 20x efficiency with StackLlama and Multipack* [00:54:47] Tri Dao and Mamba* [00:59:08] Roadmap for Axolotl* [01:01:20] The Open Source AI CommunityTranscript[00:00:00] Introducing Wing Lian[00:00:00] [00:00:00] swyx: Welcome to Latent Space, a special edition with Wing Lien, but also with our new guest host, Alex. Hello, hello. Welcome, welcome. Again, needs no introduction. I think it's like your sixth time on Latent Space already. I think so, yeah. And welcome, Wing. We just met, but you've been very prolific online. Thanks for having me.[00:00:30] Yeah. So you are in town. You're not local. You're in town. You're from Minneapolis?[00:00:35] Wing Lian: Annapolis. Annapolis. It's funny because a lot of people think it's Indianapolis. It's I've got Minneapolis, but I used to live out at least in the San Francisco Bay Area years ago from like 2008 to 2014. So it's fairly familiar here.[00:00:50] swyx: Yep. You're the maintainer of Axolotl now, which we'll get into. You're very, very prolific in the open source AI community, and you're also the founder of the Open Access AI Collective. Yeah. Cool. Awesome. Maybe we can go over a little bit of your backgrounds into tech and then coming into AI, and then we'll cover what[00:01:06] Wing Lian: happens and why you're here.[00:01:08] Yeah. So. Back on tech, so I started years ago, I started way back when I was scraping, Apartment websites for listings and then, and then building like SEO optimized pages and then just throwing Google AdSense on it.[00:01:24] And that got me through like college basically. Is[00:01:27] swyx: that decent money? And what year[00:01:28] Wing Lian: was this? Like 2004, 2005. Yeah, that's decent money. It's like thousand bucks a month. But as a college student, that's like. Gravy. Really good money, right? So, and then there's just too much competition It's just sort of like died off. I was writing stuff in like Perl back then using like like who nobody hosted anything on Perl anymore, right? Still did a little bit more like computer tech support and then software, and web more professionally.[00:01:54] So I spent some time working on applications in the blood industry. I came out to San Francisco for, I was at SGN, so Social Gaming Network, as a startup. They started doing, with Facebook apps, and then they pivoted into doing mobile apps. And then, from there, I spent time.[00:02:14] I've quite a few more startups since then and in the last few years I've been in the music space So like I was at United Masters for a while and then past year I've been at SoundCloud, but not doing that anymore and now that I have a lot more time It's just like all right.[00:02:30] We're going full bore on axolotl and we're gonna we're gonna crush AI So yeah,[00:02:34] SF Open Source AI Meetup[00:02:34] swyx: totally you so you're here in town for the open source. Yeah, I meet up that we had yesterday Yep, yeah, that was amazing. Yeah, it was a big collection. Olama, Noose Research, Alignment Lab, Anyone else that I missed? I mean, Jeremy Howard is his own thing.[00:02:47] Yeah.[00:02:49] And Alex, you're also there. You love to bring SF to the world. Your takes?[00:02:55] Alex Volkov: It's incredible that we recorded a Thursday Eye episode after that one. And LDJ, who's usually co hosts Thursday Eye, just like briefly mentioned, Oh yeah, I talked about it.[00:03:04] Like, I saw Karpathy, and then I talked to Jeremy Howard, and the guy from Mistral came in, and it's like, He's talking about all these, titans of industry, basically, that outside of SF, You just don't meet casually hanging out in the same space. You can't, pull somebody. He ran into the Laylow from Mistral, he ran into him while, drinking water.[00:03:20] He didn't even know he was there. It's just, that type of stuff is really hard to find outside of SF. So, absolutely, absolutely great. And also, presentations from Alignment Labs, presentations from News Research, news issues, talked about. Forge, and some of[00:03:33] swyx: the other stuff they announced. We can say now they're officially a company.[00:03:36] I met Technium.[00:03:37] He[00:03:37] Alex Volkov: came over here. He didn't want to get recorded. But maybe.[00:03:41] Wing Lian: We'll wear him down at some point. Yeah, I'm excited for Forge. They've positioned it as this agentic sort of framework where it's just Drag and drop things and, fill in text with where you want to inject different variables and it opens up all of these potentials for data pipelines now, right?[00:03:56] And using your own local LLMs and not relying on GPT 4 or anything like that. Yeah, yeah,[00:04:02] swyx: good stuff. Okay, so let's maybe go into the Axolotl origin story and then we have, we have some intro or background.[00:04:09] What is Axolotl?[00:04:09] swyx: To do on like the open source model universe and also on fine tuning, but maybe just, since you're talking about your personal journey, what was your personal journey into[00:04:18] Wing Lian: axolotl?[00:04:19] Yeah, so my personal journey started like back in mid March, completely unrelated to AI and axolotl. And it really started, I fell while skiing, I torqued. Great 3 MCL sprain and being sort of like an active person that can no longer be active because the two, couldn't play soccer, because that is requires to have having knees until I, it's healed.[00:04:42] So I. I decided I needed to find something to do to take up my free time. And that became, well, let's learn how to train in, these language models. It was everywhere. So I was like, all right, I'm just going to sit down, learn. I think I used like other, I think I was using like Alpacalora.[00:05:00] Cause I think the Alpaca paper had just came out, come out then. So I was like using Alpacalora repo and sort of like learning how to use like. None of us were like GPU rich back then, and none of us, most of us still we're still all GPU poor, but I was doing what was it, like 4 bit, Alpaca Lord, there was like a 4 bit version where we were doing quant, or 8, no, 8 bit quantizations, and then I think they had released QLOR a little bit later, and I think right when, before QLOR came out, I was already starting to do fine tunes, but having this need to sort of like mix data sets together, and If you've ever looked at all the various different datasets available on HuggingFace, they all have various different prompt formats, and, it's sort of a nightmare, and then I think the other piece is if you've ever tried to fine tune, at least Back then probably the ecosystem's a little better now.[00:05:54] Everybody required that you say, alright, you put your hyperparameters as command line arguments. And so it's always like, well, I now have to go copy and paste my previous thing and to change things out. And I really wanted it. to be in a YAML file because it was more portable and reproducible.[00:06:09] So I was doing that and then the QLOR paper came out. Tim Dettmer announced that and then somebody looked it up for me yesterday and it's like between that announcement it took us seven days to get that integrated into Axolotl, right? Which is like, it's not. I wouldn't say it's really fast, but in a manner that, is in a, a reusable framework, I think it was quite the accomplishment then.[00:06:33] And so we started, picking up traction with people there. And then it's just been building models, and then just iterating what my needs are. So, yeah. Excellent. Yeah. I[00:06:44] Alex Volkov: want to ask, for folks who are listening who never heard of Axolotl, now do you describe how you got there?[00:06:49] Can you, how do you summarize this for folks who maybe haven't fine tuned anything. They know about open source LLM exists, they maybe know like LLAML, what's XLR for somebody who doesn't know. I've never heard of a data set curation[00:07:01] Wing Lian: creation before. We sort of have to take a step back and understand that, when you've got these language models, you have what I think most people refer to as like base models, also known as like foundational models, right?[00:07:15] Where some benefactor, whether it's Meta or Mistral or whoever, has gone and spent all this money. To train these models on huge corpuses of text, right? And these, these corpuses, they're generally good across lots of different things, but they're really good at just saying, talking on and on and on, but they're not good at, following instructions or having chats or anything like that.[00:07:40] So, when you think about fine tuning, it's like Saying, all right, we have this really sort of good generalized, text completion thing, and I want to turn it into something that I can talk to or have, follow instructions. So, I think fine tuning is probably best defined in like that.[00:07:58] swyx: Okay, got it.[00:07:59] And we actually[00:08:01] What is finetuning?[00:08:01] swyx: Do want to make sure that we have like an overall introduction to fine tuning for people because again like trying to make sure that we bring everyone along in this, in this journey. We already went into Loras and QLoras without explaining what[00:08:12] Wing Lian: they are. Oh yes, yes, sorry.[00:08:14] swyx: And so I will put things in my words and you can correct me as, as, as my I'll be the village idiot here.[00:08:21] So, so fine tuning is basically sort of grabbing an open source model off the shelf, and then basically doing further training on it with a custom dataset of your own. Primarily, people use it, think about it as fine tuning for JSON output, or fine tuning for a style of response. Let's say you wanted to tell jokes, or be funny, or be short, or whatever.[00:08:43] Just the open source AI community has really fine tuned in all sorts of different manner. I think we'll go over those those things now. Let's go over those things now, and then we'll talk about fine tuning methods.[00:08:52] Open Source Model Zoo[00:08:52] swyx: So there's a universe of people who fine tune stuff. Yesterday in your slides, you had, I'll just list some of these and then we'll maybe go through some of them, right?[00:08:59] So Technium is personally leading Open Hermes, which is I think the sort of premier model out of the news. news community. There's OpenOrca, which you had a hand in. News, the news research itself also has Capybara and Puffin and all the others. There's Pygmalion, which I've never messed with.[00:09:14] Eric Hartford, I am aware of his Uncensored Models and his Samantha Models. Disco Research with Disco LM. And then you personally have done Manticore, Minotaur, Jackalope, and Hippogriff. What should people know about all these names? Being part of AI Twitter is seeing all these things and going dude, I'm being DDoS'ed by all these things and I don't know how different they are.[00:09:32] What should people know? Yeah, so[00:09:34] Wing Lian: I think on a lot of these models, generally, we like to think of those as sort of general models, so If you think about it, what is GPT 4, what is Chad GPT? It's a good general model, and then, One of the services I think that OpenAI offers is like these fine tunings where you're a business and you have very specific business use cases and you might fine tune for that use case.[00:10:00] All of these models are really just general use case that you can then go and maybe Fine tune another lore over it for your use cases, but they tend to be good. With good being relative, it's open source. Open source AI is still sort of is infancy. So, good is, it's pretty reasonable.[00:10:18] It's probably still better than most, high schoolers at answering questions and being able to like figure things out and, and reasoning skills and math and those sorts of things, right?[00:10:27] swyx: And also as measured on the Hugging[00:10:29] Wing Lian: Face leaderboard. Yes, well, that's like a whole other discussion, right, there's a whole other, group of people who, and I, I mostly agree with them that, benchmarks can be, are pretty bogus these days, LM says, I think they published something recently where, even if you think the dataset's not contaminated, you can go and, find contamination And maybe we should step back and say what contamination is, right?[00:10:53] Benchmarks and Contamination[00:10:53] Wing Lian: So we have all of these data, when you go and do these benchmarks, there's a specific data set where there are these questions and usually it's multiple choice. And what can happen is, well, sometimes someone It puts the question, maybe maliciously, maybe accidentally, into the training dataset, and now the, the, your model knows how to answer the test questions really well, but it doesn't, it hasn't generalized the ability to actually do that[00:11:20] Alex Volkov: right.[00:11:21] We've seen some folks competitively announce models that are like the best at that leaderboard, but then it's, it's quite obvious that, In open source? Yeah, and in that leaderboard, for Hugging Face specific, I don't know if LMCs, if that had suffered, but we, there's been some models that seem to have been competitively trained and some leakage happened into their,[00:11:41] swyx: like, supposal.[00:11:43] I understand, once there's been a credible assertion, Hugging Face actually does take them down, right? Yeah, yeah,[00:11:48] Alex Volkov: which is really hard to know, right?[00:11:50] swyx: It's really hard to know, sometimes it's like a pure accident,[00:11:52] Alex Volkov: it's oh, oops. You're going through a mixer. I think, a responsible So acknowledgement, that this kind of happened to you is also important.[00:11:58] I saw LDJ from news research can acknowledge that. Because many of these datasets are collections of other datasets. There's a bunch of people are baking, basically. It's alchemy. Right. And so sometimes you don't know. Sometimes you pull an open source dataset and they announce, oh, you know what, actually, the MMLU benchmark which we used to Specifically identify models that did go into this data set, that then went into that data set.[00:12:22] So sometimes it's actually an accident and folks take it down. But I've seen some competitive folks who want to put their name out there because people are starting to notice which is the top[00:12:30] swyx: model. For those who want a fun take on this so the file one dataset. FindOne model from Microsoft was accused of being contaminated.[00:12:37] And I saw this joke paper that was fantastic. It was called, training on the test set is all you need. It's a super small model that just memorizes everything. It was fantastic. So yeah, contamination, I think we've actually covered it in a previous episode before. So we're good. But again, I want to give people a map into the open source AI model, the universe.[00:12:57] And Alex, you can also jump in here because you guys have spent a lot more time with them than I have. So, what should people know about Technium? What should people know about Noose? And then we can go down the list. Yeah,[00:13:05] Wing Lian: I think so. I think if we start with, Technium. When you talk to him, he's gonna say, I think, I think his response is that he wants to build GP4 on his laptop, right?[00:13:14] So, very, very good at building general models. I think with Noose, Noose Research, they're looking at more, sort of, More, more research focused things, like their Yarn models, I don't, I don't, they didn't actually train their, they have their own trainer for their Yarn models, but So they did not use Xlato for that one?[00:13:30] They didn't use that, but like Is that, you don't have support for it? I think we do support Yarn, I think, I'd have to double check that answer. Yeah, I'm just kind of curious what you can and cannot support, and Yeah, I mean, Yarn is supportable, it's basically, I think it's just replacing, I think, the rope part of that, so Yeah, not, not a big deal.[00:13:48] Yeah, it's not a big deal, it's just I haven't gotten to it, not enough people have asked, I think a lot of people have asked for other things, so it's just, squeaky wheel, right? I think at the end of the day, people are like building these data sets and I think if you sort of map things chronologically, these make more sense because it's like, how do we incrementally improve all of these models?[00:14:07] So a lot of these models are just incremental improvements over the last thing, right? Whether it is sort of through methods of how do we, how did we curate the data set? How did we improve the quality of the data set? So, you maybe LDJ talked about it right on I think for, for Capybara and Puffin, like how those, those were very specific dataset curation techniques that he works on.[00:14:29] The Case for Open Source AI[00:14:29] Alex Volkov: So there's, folks are doing this for dataset curation. Folks are doing this for skillset building as well. Definitely people understand that open source is like very important, especially after the, the, the, the, the march, the debacle, the OpenAI weekend that we all had. And people started noticing that even after developer day in OpenAI, the APIs went out.[00:14:48] And then after that, the whole leadership of the company is swiftly changed and people, there was worries about, you know. How can people continue building AI products based on these like shaky grounds that turned attention definitely to Technium at least in open RMS I started seeing this more and more on Twitter, but also other models and many companies They're gonna start with open AI just to get there quick, and then they they think about okay Maybe I don't want to share my knowledge.[00:15:13] Maybe I don't want to sign up for Microsoft. Maybe they will change their terms and conditions so What else is out there? They turned to other companies. Up until yesterday, Google was nowhere to be found. We've talked about Gemini a little bit before in a previous And you can tune in[00:15:26] swyx: to[00:15:26] Alex Volkov: Thursday Eye.[00:15:26] Yeah, you can tune in to Thursday Eye. We covered the Gemini release a little bit. And but many are turning into the open source community and seeing that Meta released and continues to release and commit to open source AI. Mistral came out and the model is way smaller than LLAMA and performs Significantly better.[00:15:43] People play with OpenRMS, which is currently techniums based, news researched, sourced, axolotl trained OpenRMS, I assume, right? And then they play with this and they see that, okay, this is like GPT 3. 5 quality. We had GPT 4. 5 birthday just a week ago. A week ago, a year ago, a week ago, we never, interacted with these models of this caliber.[00:16:04] And now there's one open source, one that's on my laptop, completely offline, that, I can continue improving for my use cases. So enterprises, companies are also noticing this. And the open source community folks are building the skill set, not only the data sets. They're building the actual kind of, here's how we're going to do this, with Axelotl, with these data sets.[00:16:21] The curation pieces. Now. Interesting. There's like recipes of curation. The actual model training is kind of a competitive thing where people go and compete on these leaderboards that we talked about, the LMC arena, and that recently added open air and recently added open chat and a bunch of other stuff that are super cool.[00:16:37] The hug and face open source leaderboard. And so there's a competitive aspect to this. There's the open source. Aspect to this, like Technium says, I want GPT 4 on my laptop. There's the, let me build a skill set that potentially turns into a company, like we saw with Noose. Noose just, started organizing, a bunch of people on Discord, and suddenly, they're announcing their company.[00:16:54] It's happening across all these modalities, and suddenly all these people who saw these green pastures and a fairly quick way to, hey, here's a cool online community I can, start doing cool stuff with. You mentioned the same in the beginning, right? Like, after your accident, what's cool, let me try this out.[00:17:08] Suddenly I start noticing that there's a significant movement of interest in enterprising companies into these areas. And, this skill set, these data sets, and this community is now very Very important, important enough to create an event which pulls in Andrei Karpathy from OpenAI to come and see what's new Jeremy Howard, like the event that we just talked about, people are flying over and this is just a meetup.[00:17:28] So, definitely, the community is buzzing right now and I think Axelot is a big piece as well.[00:17:34] Orca and OpenOrca[00:17:34] Wing Lian: Cool. Maybe we can talk about like Orca real quick, Orca, OpenOrca rather, I think there was a lot of buzz when, the first Orca paper came out. And just briefly, what is Orca? Yeah, Orca was basically having traces of like chain of thought reasoning, right?[00:17:48] So they go and they, they distill sort of GPT 4. They take, they take a sampling of data from the Flan dataset. Maybe we can like add some show notes in the Flan dataset. Yeah, but we've covered it. Okay, cool. Use GPT 4 to say, all right, explain this in a step by step reasoning, right?[00:18:06] And then you take that and you, they train the model and it showed, very good improvements across a lot of benchmarks. So OpenOrca was sort of the open reproduction of that since Microsoft Research never released that particular data set. And going back to sort of the Hugging Face leaderboard thing, those models did really well.[00:18:23] And then I think, so sort of the follow up to that was SlimOrca, right? I think Going into and building the OpenOrca dataset, we never really went in and, validated the actual answers that GPT 4 gave us, so what we did was one from OpenChat actually cross referenced the original Flan, the original Flan response, the human responses, the correct answers with the dataset, and then I went and took it and sent all of, both of them to GPT 4 and said, is this answer mostly correct, right?[00:18:54] Yeah. And then we were able to filter the dataset from, At least of the GPT 4 only answers from like 800, 000 to like 500, 000 answers or rows and then, and then retrain the model and it had the same performance as the original model to within I think, 0. 1 percent here about, and 30 percent less data.[00:19:13] So, yeah. Okay.[00:19:15] swyx: Interesting. So, I mean, there's, there's so much there that I want to highlight, but yeah. Orca is interesting. I do want people to know about it. Putting chain of thought into the data set like it's just makes a ton of sense one thing I think it would be helpful for people to scope thing these things out is how much data are we talking about when when you When people are fine tuning and then how much time or resources or money does it take to train to fine[00:19:36] Wing Lian: tune?[00:19:37] Yeah, so I think there's a little bit of overlap there with sort of like fine tuning techniques, but let's say Orca and I think even Hermes, they're both relatively large data sets like 10 billion tokens. Yeah. So large data sets being or the original Orca was, or the original open Orca was 800,000 rows.[00:19:55] I believe it was somewhere in the ballpark of like a gigabyte of data, of gigabyte, of text data. And I, I don't. I believe, Hermes was, is like a quarter million rows of data, I don't know the actual byte size on that particular one. So, going and training a, let's, let's say everybody's training 7 billion Mistral right now, right?[00:20:15] So, to tri I, I believe to fine tune 7 billion Mistral on, let's say, 8 A6000s, which have 48 gigabytes of VRAM, I believe, It takes about 40 hours, so 40, and then that's, depending on where you get your compute, 40 times 6, so it's like 500 to fine tune that model, so, and, and that's assuming you get it right the first time, right?[00:20:44] So, you know.[00:20:45] swyx: Is, is that something that X. Lotto handles, like, getting it right the first[00:20:48] Wing Lian: time? If you talk to anybody, it's like you've probably tried at least three or four runs or experiments to like find the right hyperparameters. And after a while you sort of have a feel for like which, where you need your hyperparameters to be.[00:21:04] Usually you might do like a partial training run, do some benchmark. So I guess for Al Farouk, whether you're going by his. This is Jeremy, he's, his actual name, or his twitter handle. He released the Dharma dataset, which is basically a subset of all the benchmarks. And Axolotl actually supports, you know taking that subset and then just running many benchmarks across your model every time you're doing an evaluation so you can sort of like see sort of relative it's not going to be the actual benchmark score, but you can get ideas alright, is this benchmark improving, is this benchmark decreasing, based on, you know Wait,[00:21:39] swyx: why don't you run the full benchmark?[00:21:41] What, what, what The[00:21:42] Wing Lian: full benchmarks take Take a long time. Significant, yeah, significant amount of time. Yeah. And Okay, so that's like[00:21:48] swyx: mini MMLU. Yeah. Like,[00:21:49] Wing Lian: mini BigBench or whatever. Yep, exactly.[00:21:51] Alex Volkov: It's really cool. We, when I joined Web2Masters just recently, and one of the things that I try to do is hey I'm not, I'm a software engineer by trade, I don't have an MLE background, But I joined a company that does primarily MLE, and I wanted to learn from the community, Because a lot of the open source community, they use weights and biases, And the benchmark that you said that Pharrell did, remind me of the name, sorry.[00:22:13] Dharma? Dharma, yeah, yeah. So Luigi showed me how Dharma shows inside the dashboard. In Wi and Biases dashboard and so you can actually kinda see the trending run and then you can see per each kind of iteration or, or epoch or you can see the model improving trending so you can on top of everything else.[00:22:29] The wi and biases gives like hyper parameter tracking, which like you, you started with common line and that's really hard to like remember. Also the Dharma data set, like the quick, the mini orca mini, you mini many different things. It's pretty cool to like visualize them as well. And I, I heard that he's working on a new version of, of Dharma, so Dharma 2, et cetera.[00:22:47] So hopefully, hopefully we'll see that soon, but definitely it's hard, right? You start this training around, it said like 40, 50 hours. Sometimes, sometimes it's like your SSHing into this machine. You, you start a process, you send it with God and you just go about your day, collecting data sets, and then you have to return.[00:23:04] And the whole process of instrumentation of this is still a little bit like squeaky but definitely. Tuning performance, or like grabbing performance in the middle of this, like with Dharma and some other tools, is very helpful to know that you're not wasting precious resources going somewhere you shouldn't go.[00:23:21] Yeah.[00:23:22] swyx: Yeah. Very cool. Maybe I'll, I'll, before we go into like sort of more details on fine tuning stuff, I just wanted to round out the rest of the Excel autoverse. There's, there's still Eric Hartford stuff. I don't know if you want to talk about Pygmalion, Disco, anything that you know about[00:23:35] Wing Lian: those, those things.[00:23:36] DiscoLM and Model Stacking[00:23:36] Wing Lian: Yeah, I think like one of the, definitely one of the more interesting ones was like the Disco 120b, right? Yeah, I know nothing about it. Yeah. So, so. Alpen from Pygmalion AI, right, so they, so Pygmalion is a sort of a, it's, it's, they have their own community, a lot of it is based around, roleplay models, those sorts of things, and Alpen, like, put together, merged together Llama270B, so, and Alpen, like, put together, merged together Llama270B, so, I don't remember how he stacked them together, whether he merged the layers in between. There's a whole, there's a whole toolkit for that by Charles Goddard, where you can like take a single model and like stack them together or multiple models merge.[00:24:18] That's like a whole other talk and a whole other tool set, but was able to create this 120. Billion parameter model out of a LAMA two 70 B. And then I believe the, yeah, disco is a fine tune of, of the, the, the sort of the base one 20 B is, I believe Goliath one 20 B. So, and, and what are the[00:24:37] swyx: headline results that people should know about[00:24:39] Wing Lian: disco?[00:24:39] I think for the headline results, I, I've, I haven't played with it personally because it's. It's a very large model and there's a lot of GPU, right? But, like, from what I've heard anecdotally, it performs really well. The responses are very good. Even with, like, just, even the base model is a lot better than, Llama70b.[00:24:57] So, and we, I think generally everybody's like, we would all love to fine tune Llama70b, but it's just, it's so much, it's so much memory, so much compute, right?[00:25:07] Datasets and Evals over Models[00:25:07] Wing Lian: I[00:25:07] Alex Volkov: want to touch on this point because the interesting thing That comes up out of being in this ecosphere and being friends with open source folks, tracking week to week state of the art performance on different models.[00:25:19] First of all, a lot of the stuff that the folks do a couple of weeks ago, and then something like Mistral comes out, and a lot of the stuff back then, Doesn't technically make sense anymore. Like the artifacts of that work, the actual artifacts, they don't no longer make sense. They're like lower on the on, on the hug and face leaderboard or lower on LM CS leaderboard.[00:25:36] But some of the techniques that people use, definitely the datasets. The datasets keep traveling, right? So open airmen, for example, is the dataset. The tum cleaned up for only. Open sourceable data that previously was just Hermes. And that, it was previously used to train Lama. And then once Mistral came out, it was used to train Mistral.[00:25:54] And then it became significantly better on the 7b base Mistral. So the data sets keep traveling, keep getting better a little bit here and there. And so the techniques improve as well. It looks like both things are simultaneously true. The artifacts of a month and a half ago. The, the actual models themselves, it's great the hug and face has them, because not every company can keep up with the next weeks', oh, I, I'll install this model instead, sell this model instead.[00:26:19] But the, the techniques and the, the dataset keep improving as we go further, and I think that's really cool. However, the outcome of this is that for a long time. For many, many people, including us, that we do this every week. We literally talk with people who release these models every week. It's really hard to know.[00:26:36] So, there's a few aspects of this. One, I think, like you said, the bigger model, the 70B models, you actually have to have somebody like Perplexity, for example, giving you access to the 70B really fast. Or you have to, like, Actually, find some compute, and it's expensive, especially for the bigger models. For example Falcon 180B came out, like the hugest open source model.[00:26:56] How do you evaluate this if you can't run it? Nobody liked it. It's really, so first of all, nobody liked it, but secondly, only the people who were able to find compute enough to run inference on this, they only had like, I can't run this on my laptop, and so that's why it's much easier, something like OpenRMS 7 to be, 7B, it's much easier, because you can run this on your MacBook.[00:27:14] It's much easier to evaluate. It's much easier to figure out the vibes, right? Everybody talks about the vibes as an evaluation check. If you're plugged in enough, if you follow the right people, if they say pretty much the same things all independently, then you run into a problem of whether they're repeating, and their stochastic parents are repeating the same thing, or they actually evaluated themselves.[00:27:31] Yeah, you never know. But, you never know, but like, I think on a large enough scale on Twitter, you start getting the feel. And we all know that like, OpenRMS is one of the top performing models, benchmarks, but also vibes. And I just wanted to highlight this vibes checks thing because you can have the benchmarks, you can have the evaluations, they potentially have contamination in them, potentially they not necessarily tell you the whole story because some models are good on benchmarks, but then you talk to them, they're not super helpful.[00:28:00] And I think it's a combination of the benchmarks, the leaderboards, the chatbot, because LMSys, remember, their ranking is not only based on benchmarks, it's also people playing with their arena stuff. People actually like humans, like, get two answers. I think they completely ignore benchmarks. Yeah, and then They only do ELO.[00:28:18] Oh, they do ELO completely, right? So that, for example, is just like people playing with both models and say, Hey, I prefer this one, I prefer that one. But also there's like some selection bias. The type of people who will go to LMCs to play with the models, they're a little bit specific in terms of like who they are.[00:28:33] It's very interesting. There's so many models. People are doing this in this way, that way. Some people are doing this for academic rigor only to test out new ideas. Some people are actually doing this like the Intel fine tunes of Mistral. Intel wanted to come out and show that their hardware approach is possible, Mistral, etc.[00:28:51] And it's really hard to know, like, what to pick, what to use. And especially on the bigger models, like you said, like the Llama 70B, the Falcon 180B. It's really because, like, who has the compute to validate those? So I would mention that, like, use with caution. Like, go and research and see if the biggest model that just released was actually worth the tokens and the money you spend on it.[00:29:12] To try and, if you're a business, to integrate it.[00:29:15] Distilling from GPT4[00:29:15] swyx: Since you said use of caution, I'll bring in one issue that has always been in the back of my mind whenever I look at the entire universe of open source AI models, which is that 95 percent of the data is derived from GPC 4, correct?[00:29:30] Which technically you can't use for commercial licenses,[00:29:34] Wing Lian: right?[00:29:35] swyx: What is the community's stance on this kind of stuff?[00:29:40] Wing Lian: I think from the community stance, like I feel like a lot of us are just experimenting, so for us, it's like, we're not going and building a product that we're trying to sell, right?[00:29:49] We're just building a product because we think it's interesting and we want to use it in our day to day lives, whether or not we try and integrate it. Personal use, yeah. Yeah, personal use, so like, as long as we're not selling it, yeah, it's fine. But[00:30:01] swyx: like, I as a company cannot just take OpenHermes and start serving[00:30:05] Alex Volkov: it and make money on it.[00:30:06] OpenHermes you can. Because the opening of OpenHermes, I think, is a clean up. That did after the regular Hermes, please folks, check your licenses before you listen to podcasts and say, Hey, I will tell you though, you could say the same thing about OpenAI. You could say the same thing kind of makes sense, where OpenAI or StabilityAI trains their diffusion model on a bunch of pictures on the internet, and then the court kind of doesn't strike down Sarah Silverman, I think, or somebody else, who came and said, hey, this has my work in it, because of the way how it processes, and the model eventually builds this knowledge into the model, and then it doesn't actually reproduce one to one what happened in the dataset.[00:30:45] You could claim the same thing for open source. Like, we're using And by we, I mean the, the open source community that I like happily report on uses GPT 4 to rank, for example, which is the better answer you, you, that's how you build one, one type of data set, right? Or DPO or something like this, you, you basically generate data set of like a question and four answers, for example, and then you go to GPT 4 and say, Hey, smartest model in the world right now, up to Gemini Ultra, that we should mention as well.[00:31:11] Which one of those choices is better? But the choices themselves are not necessarily written with GPT 4. Some of them may be, so there's like full syntactic datasets. But there's also, datasets are just ranked with GPT 4. But they're actually generated with a sillier model, or like the less important model.[00:31:25] The lines are very blurry as to what type of stuff is possible or not possible. And again, when you use this model that's up on Hug Face, the license says you can use this. OpenAI is not going to come after you, the user. If anything, OpenAI will try to say, hey, let's prevent this, this type of thing happening, and the brain, but I honestly don't think that they could know even, not that it makes it okay, it's just like, They also kind of do this with the Internet's archive, and also, I think that some of it is for use.[00:31:55] You use models to help you augment tasks, which is what GPT 4 lets you do.[00:32:00] swyx: Yeah, the worst thing that OpenAI can do is just kick you off OpenAI. That's because it's only enforced in the terms of service.[00:32:05] Alex Volkov: Sure, but just like to make sure, to clarify who they're going to kick out, they could kick out like News, for example, if news are abusing their service, a user of the open source, fully Apache 2 open source, for example, They won't get kicked out if they use both, just because they use both.[00:32:22] I don't believe so. I don't think OpenAI has a claim for that.[00:32:25] swyx: Well, we're not lawyers, but I just want to mention it for people to know it's an issue.[00:32:30] Wing Lian: And one of the things, like, I talked to someone recently, and I think that they also are like interested in it, but also to the point of like, right, if I use a model trained on data, using GPT for data, But I use that model to then regenerate new data.[00:32:46] Is that model, is that data okay? So like you start going down this whole rabbit hole. So yeah. All right.[00:32:53] swyx: Fantastic. Cool. Well, I think that's roughly highlights most of the open source universe. You also have your own models. Do you want to shout out any one of them? Yeah.[00:33:01] Wing Lian: I mean, I think like, I think Early on, Manicore got a lot of love.[00:33:04] I think it was mostly popular in, like, the roleplay communities. It was, it tended to be pretty truthful. It tended to be, like, have relatively good answers, depending on who you ask, right? But, I think for me, it was just, Releasing models was a way to try and, like, continue to build out the product, figure out what I needed to put into the product, how do I make it faster, and, if you've got to, like, go and debug your product, you may as well have it do something useful.[00:33:29] Awesome. So, yeah.[00:33:31] Finetuning - LoRA, QLoRA, ReLoRA, GPTQ[00:33:31] swyx: Okay, and then maybe we'll talk about just fine tuning techniques. So this is going to be a little bit more technical than just talking about model names and datasets. So we started off talking about LoRa, QLoRa. I just learned from your readme there's ReLoRa. Which I've never heard about.[00:33:45] Could you maybe talk about, like, just parameter efficient fine tuning that whole, that[00:33:50] Wing Lian: whole journey, like, what people should know. Yeah, so with parameter efficient fine tuning, I think the popular ones, again, being, let's, we'll start with lore, right? So, usually what you do is you freeze all the layers on your base, on the base model, and then you, at the same time, you sort of introduce additional Oh, this is tight.[00:34:08] No. You introduce, another set of layers over it, and then you train those, and it is done in a way that is mathematically possible, particularly with LORs that you can, then you, you, When you, when you train the model, you, you run your inputs through the base model, whose weights are frozen, but you, then you also run it through the additional weights, and then at the end you combine the weights, and then, and then, or you combine the weights to get your outputs, and then at the end, and when you're done training, you're left with this other set of weights, right, that are completely independent, and And then from that, what you can do is, some person smarter than I figured out, well, oh, they've done it in such a way that now I can merge these weights back into the original model without changing the architecture of the model, right?[00:35:03] So, so, that tends to be, like, the go to, and You're training much fewer parameters so that when you do that, yes, you still need to have all of the original weights, but you have a smaller gradient, you have a smaller optimizer state, and you're just training less weights, so you can tend to train those models on, like, much smaller GPUs.[00:35:27] swyx: Yeah. And it's roughly like, what I've seen, what I've seen out there is roughly like 1 percent the number of parameters that you're trading. Yeah, that sounds about right. Which is that much cheaper. So Axelotl supports full fine tune, LoRa, QLoRa,[00:35:40] Wing Lian: Q. Yes. So, so QLoRa is, is very similar to LoRa. The paper was, if I remember correctly, the paper was Rather, traditionally, most people who did Loras were, were, they were quant, they were putting the model weights in 8 bit, and then fine tune, parameter efficient fine tuning over the Lora weights, and then with QLora, they were quantizing all of those, they were then quantizing the weights down to 4 bit, right, and then I believe they were also training on all of the linear layers in the model.[00:36:15] And then with ReLore, that was an interesting paper, and then, I think, like, it got implemented. Some people in the community tried it, tried it out, and it showed that it didn't really have the impact that the paper indicated that it would. And from what I was told recently, that they re I guess they re released something for Relora, like, a few weeks ago, and that it's possibly better.[00:36:44] I personally haven't had the time. What was the[00:36:46] swyx: main difference,[00:36:47] Wing Lian: apart from quantization? I don't know. Okay. What was the main difference, sorry?[00:36:49] swyx: Apart from quantization, right? Like,[00:36:50] Wing Lian: Qlora's thing was, like, we'll just drop off some bits. With Relora, what they did was, you would go through, you would define some number of steps that you would train, like, your Lora with, or your Qlora.[00:37:01] Like, you could do Like, ReqLore, if you really wanted to, you would, you would train your LoRa for some number of steps, And then you would merge those weights into your base model, and then you would start over. So by starting, so, then by starting over, The optimizer has to find, like, sort of, re optimize again, and find what's the best direction to move in, and then do it all again, and then merge it in, do it all again, and theoretically, according to the paper, doing ReLore, you can do parameter efficient fine tuning, but still have sort of, like, the performance gains of doing a full fine tuning, so.[00:37:38] swyx: Yeah, and[00:37:39] Wing Lian: GPTQ? And GPTQ, so it's, I think with GPTQ, it's very similar to, more similar to QLore, where you're, it's mostly a quantization of the weights down to like 4 bit, where GPTQ is a very, is a specific methodology or implementation of quantization, so. Got it.[00:37:57] Alex Volkov: Wang, for, for folks who use Axolotl, your users, some people who maybe, Want to try it out?[00:38:03] And do they need to know the differences? Do they need to know the implementation details of QLora versus ReLora? Or is it okay for them to just know that Axolotl is the place that already integrated them? And if that's true, if that's all they need to know, how do they choose which method to use? Yeah,[00:38:22] Wing Lian: so I think like, I think most people aren't going to be using ReLora.[00:38:25] I think most people are going to be using either Lora or QLora. And I think they should have it. They should have an understanding of why they might want to use one over the other. Most people will say that with Qlora, the quality of the final model is not quite as good as like if you were to do a LoRa or a full fine tune, right?[00:38:44] Just because, you've quantized these down, so your accuracy is probably a little off, and so that by the time you've done the Qlora, you're not moving the weights how you would on a full fine tune with the full parameter weights.[00:38:56] Interesting.[00:38:57] swyx: Okay, cool. For people who are more interested, obviously, read the papers. I just wanted to give people, like, a high level overview of what these things are. And you've done people a service by making it easy for people to try it out. I'm going to, I'm going to also ask a question which I know to be wrong, but I'm curious because I get asked this all the time.[00:39:15] What is the difference between all these kinds of fine tunes[00:39:17] Wing Lian: and RLHF? Okay, between all of these sorts of fine tunes and RLHF. So all of these sorts of fine tunes are based, are, ideally, this, they are taking knowledge that the base model already knows about, and presenting it in a way to the model that you're having the model answer like, Use what it already knows to sort of answer in a particular way, whether it's, you're extracting general knowledge, a particular task, right?[00:39:44] Instruct, tune, chat, those sorts of things. And then generally with RLHF, so what is, let's go back, what is it? Reinforcement Learning with Human Feedback. So if we start with the human feedback part, What you're doing is you generally have, you have like a given prompt and then you, maybe you have one, maybe you have two, I think, like if you look at with Starling, you have like up to what, seven different, seven different possible responses, and you're sort of ranking those responses on, on some sort of metric, right, whether the metric is how much I, I might like that answer versus or I think with like starling is like how how how helpful was the answer how accurate was the answer how toxic was the answer those sorts of things on some sort of scale right and then using that to go back and like sort of Take a model and nudge it in the direction of giving that feedback, to be able to answer questions based on those preferences.[00:40:42] swyx: Yeah, so you can apply, and is it commutative? Can you apply fine tuning after and onto an RLHF model? Or should the RLHF apply, come in afterwards,[00:40:54] Wing Lian: after the fine tune? Um, I, yeah, I don't know that there's There's been enough research for one way or another, like, I don't know.[00:41:02] That's a question that's been asked on Discord. Yeah, like, I definitely would say I don't know the answer. Go and try it and report back to me and let me know so I can answer for the next guy.[00:41:10] swyx: It's shocking how much is still unknown about all these things. Well, I mean, that's what research is for, right?[00:41:16] Wing Lian: So actually I, I think I saw on the top of a leaderboard, it was a, it was a mytral base model, and they didn't actually fine tune it. They, or they, they just did RLH, they did like an RLHF fine tune on it using like, I don't, I don't recall which dataset, but it was like, and it benchmarked really well.[00:41:37] But yeah, you'd have to go and look at it. But, so it is interesting, like going back to that, it's like. Traditionally, most people will fine tune the model and then do like a DPO, PPO, some sort of reinforcement learning over that, but that particular model was, it seemed like they skipped like the supervised fine tuning or Scott.[00:41:55] Axolotl vs HF Transformers[00:41:55] swyx: Cool. One thing I did also want to comment about is the overall, like, landscape, competitive landscape, I don't know. Hugging Face Transformers, I think, has a PFT module.[00:42:05] Wing Lian: Yeah, yeah, the PEFT, the Parameter Efficient Fine Tuning, yep. Is that a competitor to you? No, no, so we actually use it. We're just a wrapper over sort of, sort of the HuggingFace stuff.[00:42:15] So, so that is their own sort of module where They have, taken the responsibility or yeah, the responsibility of like where you're doing these parameter efficient fine tuning methods and just sort of like, it is in that particular package where transformers is mostly responsible for sort of like the modeling code and, and the trainer, right.[00:42:35] And then sort of, there's an integration between the two and, there's like a variety of other fine tuning packages, I think like TRL, TRLX, that's the stability AI one. Yeah, I think TRL likes the stability, yeah, Carper, and TRL is a hugging face trainer. Even that one's just another wrapper over, over the transformers library and the path library, right?[00:43:00] But what we do is we have taken sort of those, yes, we've We also use that, but we also have more validation, right? So, there are some of us who have done enough fine tunes where like, Oh, this and this just don't go together, right? But most people don't know that, so like Example?[00:43:19] Like, people want to One and one doesn't go together. I don't have an example offhand, but if you turn this knob and this knob, right? You would think, all right, maybe this will work, but you don't know until you try. And then by the time you find out it doesn't work, it's like maybe five minutes later, it's failed.[00:43:34] It's failed in the middle of training or it's failed during the evaluation step. And you're like, ah, so we've, we've added a lot of, we've added a lot more validation in it. So that like, when you've, you've created your configuration, you run it through and now you say. The validation code says this is probably not right or probably not what you don't, not what you want.[00:43:52] So are you like a, you[00:43:53] swyx: do some linting of your YAML file?[00:43:56] Wing Lian: There, I guess you could call it linting, it's sort of like Is there a set of rules out[00:44:00] swyx: there somewhere? Yeah, there's a set of rules in there. That's amazing, you should write documentation like This rule is because, this user at this time, like, ran into this bug and that's what we invested in.[00:44:10] It's like a good collection[00:44:11] Wing Lian: of knowledge. Yeah, it is, and I guess like, if you really wanted to, like, figure it out, I guess you could, like, git blame everything, and But, yeah, it's, so, I think that's always a useful thing, it's like Because people want to experiment but they don't, people will get frustrated when you've experiment, you're experimenting and it breaks and you don't know why or you know why and you've just gone down the rabbit hole, right?[00:44:37] So, so I think that's one of the big features that's, that I think I find important because it's It prevents you from doing things you probably shouldn't have, and it, and sometimes we will let you do those things, but we'll try and warn, warn you that you've done that.[00:44:50] I[00:44:51] Alex Volkov: have a follow up question on this, actually, because yesterday we hung out to this open source event, and I spent time by you a couple times, like when people told you, oh, XLR, I use XLR, it's super cool, and then the first thing you asked is, like, immediately, like, what can we improve?[00:45:04] And yes, from multiple folks, and I think we talked about this a little bit, where there's It's a developer tool. It's like a machine learning slash developer tool. Your purpose in this is to help and keep people, as much as possible, like, Hey, here's the best set of things that you can use right now. The bear libraries are, or the bear trainer, for example, is a bear trainer.[00:45:28] And also, maybe we should talk about how fast you're implementing these things. So you mentioned the first implementation took a week or so. Now there's a core maintainer group, right? There's like, features are landing, like Qlora, for example. Neftune, I don't know if that's one example of something that people potentially said that it's going to be cool, and then eventually, like, one of those things that didn't really shake out, like, people quickly tested this out.[00:45:48] So, there's a ton of Wait, Neftune is cancelled? I don't know if it's fully canceled, but based on vibes, I heard that it's not that great. So like, but the whole point that I'm trying to make with Neftune as well is that being existing in the community of like XLR or like, I don't know, even following the, the GitHub options or following the Discord, it's a fairly good way to like, learn these, Kind of gut feelings that you just, you just said, right?[00:46:14] Like where this, maybe this knob, that knob doesn't work. Some of these are not written down. Some of these are like tribal knowledge that passes from place to place. Axel is like a great collection of many of them. And so, do you get That back also from community of folks who just use, like, how do you know who uses this?[00:46:30] I think that's still an issue, like, knowing if they trained with XLR or should they add this to things? Talk about, how do you get feedback and how else you should get feedback?[00:46:38] Wing Lian: Yeah, I mean, most of the feedback comes from the Discord, so people come in and , they don't get a training running, they run into, like, obscure errors or, errors that That's a lot of things that maybe, maybe as a product we could catch, but like, there's a lot of things that at some point we need to go and do and it's just on the list somewhere.[00:46:58] Right that's why when people come up, I'm like, what, what were your pain points? Because like, as a developer tool, if you're not happy with it, or you come in and in the first, Takes you 30 minutes and you're still not happy. You leave the tool and you may, you might move on maybe to a better tool, maybe to, one with less frustration, but it may not be as good, right?[00:47:17] So I'm trying to like, figure out, all right, how can I reduce all this frustration? Because like for me, I use it every day for the most part, right? And so I am blind to that, right? Mm-Hmm. . Mm-Hmm. . I just know, I, I go do this, this, and this. It pretty much mostly works, right? But, so I don't have sort of that, alright, that learning curve that other people are seeing and don't understand their pain points.[00:47:40] Yeah,[00:47:40] Alex Volkov: you don't have the The ability to onboard yourself as a new user completely new to the whole paradigm to like get into the doors of like, Oh, no, I don't even know how to like ask about this problem or error.[00:47:53] swyx: Cool. The last few things I wanted to cover was also just the more advanced stuff that you covered yesterday.[00:48:00] 20x efficiency with StackLlama and Multipack[00:48:00] swyx: So I'll just, caution this as like, yeah, this is more advanced. But you mentioned Stackllama and Multipack. What are they[00:48:06] Wing Lian: and what should people know? Yeah, so, so, Stack Llama was, that paper came out, so Stack Llama I think was like, two, two, two separate, two separate concepts that they announced, so the first one was They being hugging face.[00:48:20] Yeah, sorry, yes, they being hugging face, so the first one being sort of like, this idea of packing, like some packing sequences together, so like, if we think about training data, right, your training data is, let's say, to keep the math easy, let's say your training data is 500, We, we, we, we will use the terminology words.[00:48:39] Let's say your training data is 500 words long, and let's say your, your context length, you know how much data your, that your model can accept is like, or that you want feed into your model. It's, let's say, we won't use tokens again, we'll we'll use it is it's 4,000 tokens, right? So if you're training at 4K Con or four 4,000 4K contacts and you're only using 500 of it, you're sitting like with the other 1500.[00:49:05] 3, 500 words that you're not using, right? And typically that's either filled with these PAD tokens, so I think I made the analogy last night that it's like having sort of like a glass here you fill it up with a shot of liquor and then you're and that's your training data and then you just fill it up with more water and those are your PAD tokens and it's just, it doesn't do much, right?[00:49:27] It's still the same thing, but you still have to go through all of that to go through all your training data. And then, so what Stack Llama showed was you could just sort of take your training data, append the next row of training data until you filled that entire 4k context, so in this example, right, with 500 words to 4k, that's 8 rows of training data.[00:49:48] But, the problem with that is, is that with a lot of these transformer models, they're very much relying on attention, right? So, like, if you now have this sequence of words that now, in order for the, the model has seen all of these other words before, right? And then it sees another set of words, another set of words, but it's learning everything in context of all the words that it's seen before.[00:50:13] We haven't corrected the attention for that. And just real quickly, since I said that that paper was two concepts, the other one was, I believe it was like a reinforcement learning, but outside the scope of this. So going from that, I implemented that early on because I was like, Oh, wow, this is really great.[00:50:29] And. Yes, because it saves you a bunch of time, but the trade off is a little bit of accuracy, ultimately, but it still did pretty well. I think when I did Manicore, I think it used sort of that concept from Stack Llama of just sort of appending these sequences together, right? And then sort of the next evolution of that is Multipack, right?[00:50:51] So, there was a separate paper on that, it was, I believe it was referenced, it got referenced in the Orca paper, where you could, you could properly mask those out using like a, I think it was like a lower block triangular attention mask, and then sort of, so, So, there's that. I did try implementing that, manually recreating that mask, but then one from the OpenChat, so he was helping with OpenOrca as well, and he had done an implementation of Multipack, and where he used FlashAttention, so FlashAttention So that was released by TreeDAO, and it was this huge performance gain.[00:51:35] Everybody uses it now, even the Transformers library now, they've taken all of these, like, people are taking all of these models and sort of like, making it compatible with FlashAttention. But in Flash Tension, there is one particular implementation that lets you say, Well, I'm sending you all of these sequences like you would in Stack Llama, But let me send you another, another, Set of information about, this is where this set of sequences is, this is where the second set of sequences is.[00:52:06] So like, if it was like, 500 words long, and you stacked them all together, you would just send it a row of information that was like, 0, 500, 1000, 1500, etc, etc, out to 4000. And it would know, alright, I need to break this up, and then run the forward pass with it. And then it would be able to, and it was much more, much more performant.[00:52:29] And I think you end up seeing like 10x, 20x improvements over sort of, I mean, I think FlashAttention was like a 2x improvement, and then adding that with the Multipack, you start to see like, depending on, how much data you have, up to like a 20x improvement sometimes. 20x. 20x. Wow. Yeah.[00:52:48] And I only know the 20x because I, like, before last night, I was like, I re ran the alpaca, I looked up the alpaca paper because it was like, I just need a frame of reference where somebody did it, and I think they used eight A100s for three hours, and they said it cost them 100. I don't, I don't think eight A100s cost, I don't know how much it costs right now.[00:53:14] But I ended up rerunning it. Usually a dollar an hour, right? Yeah, so eight. The cheapest is like a[00:53:18] Alex Volkov: dollar, a dollar an hour for one.[00:53:20] Wing Lian: Yeah, so that's still like 24, 25. But maybe if you're going on Azure, maybe it's like, maybe it's 100 on Azure. I mean, it used to be more expensive, like, a year ago.[00:53:31] Yeah, and then, so I re ran it with sort of like, I turned on all of the optimizations just to see what it would be. And like, and usually Multipack is the biggest optimization, so Multipack with Flash Detention. And it, I think I spun it up on 8 L40s, and it ran, and I didn't let it run all the way through, I just grabbed the time, the estimated completion time, and it was like 30 minutes, so it would have cost like 4 or 5 to run the entire, like, reproduce the alpaca paper, right?[00:54:00] Which is crazy. It's crazy. 20x,[00:54:02] Alex Volkov: yeah. I want to ask about, like, you said you turned on all the optimization. Is that the yaml file with xlodl, you just go and like check off, like, I want this, I want that? Yeah, yeah,[00:54:10] Wing Lian: so there's like one particular yaml file in there, That, there's one particular YAML file in there that's like, it's under examples, llama2, fft, optimize.[00:54:20] So, I think someone had created one where they just turned, they put in all of the optimizations and turned them on. I mean, it actually, it does run, which is like, sort of surprising sometimes, because sometimes, you optimize this, optimize this, and sometimes they just don't work together, but, yeah.[00:54:36] Just turn the knobs on, and like, fine tuning should really just be that easy, right? I just want to flip the knob and move on with my life and not figure out how to implement it.[00:54:47] Tri Dao and Mamba[00:54:47] Alex Volkov: Specifically, the guy behind FlashAttention came up with something new. You want to talk about this a little bit? You want to briefly cover Mamba?[00:54:53] Yeah, let's talk about Mamba. Let's talk about Mamba. So, what is Mamba?[00:54:57] Wing Lian: Oh, gosh. I
Despite the dominance and effectiveness of scaling, resulting in large networks with hundreds of billions of parameters, the necessity to train overparametrized models remains poorly understood, and alternative approaches do not necessarily make it cheaper to train high-performance models. In this paper, we explore low-rank training techniques as an alternative approach to training large neural networks. We introduce a novel method called ReLoRA, which utilizes low-rank updates to train high-rank networks. We apply ReLoRA to pre-training transformer language models with up to 350M parameters and demonstrate comparable performance to regular neural network training. Furthermore, we observe that the efficiency of ReLoRA increases with model size, making it a promising approach for training multi-billion-parameter networks efficiently. Our findings shed light on the potential of low-rank training techniques and their implications for scaling laws. 2023: Vladislav Lialin, Namrata Shivagunde, Sherin Muckatira, Anna Rumshisky https://arxiv.org/pdf/2307.05695v2.pdf
Getting a disease is inevitable; no one is exempt from it. Some people are fortunate enough to be cured of their illnesses. However, some diseases and conditions do not have cures, such as cancer. In a pandemic-stricken world, a deeper understanding of our immune system is crucial now more than ever. The body's immune system is the body's primary defence against sickness and disease. By learning the inner workings of our body, we can help support and prepare ourselves to fight cancer and keep ourselves healthy. In this episode, Dr Elizabeth Yurth shares her knowledge on the effects of cancer and viruses on our immune system. She talks about alternative approaches based on scientific evidence that goes hand in hand with traditional cancer treatments, ranging from spermidine to vitamin D and zinc. Become pre-emptive and preventive when it comes to your health! Get ahead of diseases through early diagnosis. If you want to learn more about how you can help your immune system fight cancer and other diseases, then this episode is for you! Here are three reasons why you should listen to the full episode: Understand how your immune system works against viruses and cancer cells. Learn more alternative approaches to cancer treatment aside from chemotherapy and radiation. Discover how early testing and screening can help improve your overall health. Get Customised Guidance for Your Genetic Make-Up For our epigenetics health programme, all about optimising your fitness, lifestyle, nutrition and mind performance to your particular genes, go to https://www.lisatamati.com/page/epigenetics-and-health-coaching/. Customised Online Coaching for Runners CUSTOMISED RUN COACHING PLANS — How to Run Faster, Be Stronger, Run Longer Without Burnout & Injuries Have you struggled to fit in training in your busy life? Maybe you don't know where to start, or perhaps you have done a few races but keep having motivation or injury troubles? Do you want to beat last year's time or finish at the front of the pack? Want to run your first 5-km or run a 100-miler? Do you want a holistic programme that is personalised & customised to your ability, goals, and lifestyle? Go to www.runninghotcoaching.com for our online run training coaching. Health Optimisation and Life Coaching Are you struggling with a health issue and need people who look outside the square and are connected to some of the greatest science and health minds in the world? Then reach out to us at support@lisatamati.com, we can jump on a call to see if we are a good fit for you. If you have a big challenge ahead, are dealing with adversity or want to take your performance to the next level and want to learn how to increase your mental toughness, emotional resilience, foundational health, and more, contact us at support@lisatamati.com. Order My Books My latest book Relentless chronicles the inspiring journey about how my mother and I defied the odds after an aneurysm left my mum Isobel with massive brain damage at age 74. The medical professionals told me there was absolutely no hope of any quality of life again. Still, I used every mindset tool, years of research and incredible tenacity to prove them wrong and bring my mother back to full health within three years. Get your copy here: https://shop.lisatamati.com/collections/books/products/relentless. For my other two best-selling books Running Hot and Running to Extremes, chronicling my ultrarunning adventures and expeditions all around the world, go to https://shop.lisatamati.com/collections/books. Lisa's Anti-Ageing and Longevity Supplements NMN: Nicotinamide Mononucleotide, an NAD+ precursor Feel Healthier and Younger* Researchers have found that Nicotinamide Adenine Dinucleotide or NAD+, a master regulator of metabolism and a molecule essential for the functionality of all human cells, is being dramatically decreased over time. What is NMN? NMN Bio offers a cutting edge Vitamin B3 derivative named NMN (beta Nicotinamide Mononucleotide) that can boost the levels of NAD+ in muscle tissue and liver. Take charge of your energy levels, focus, metabolism and overall health so you can live a happy, fulfilling life. Founded by scientists, NMN Bio offers supplements of the highest purity and rigorously tested by an independent, third-party lab. Start your cellular rejuvenation journey today. Support Your Healthy Ageing We offer powerful third-party tested NAD+ boosting supplements so you can start your healthy ageing journey today. Shop now: https://nmnbio.nz/collections/all NMN (beta Nicotinamide Mononucleotide) 250mg | 30 capsules NMN (beta Nicotinamide Mononucleotide) 500mg | 30 capsules 6 Bottles | NMN (beta Nicotinamide Mononucleotide) 250mg | 30 Capsules 6 Bottles | NMN (beta Nicotinamide Mononucleotide) 500mg | 30 Capsules Quality You Can Trust — NMN Our premium range of anti-ageing nutraceuticals (supplements that combine Mother Nature with cutting edge science) combats the effects of aging while designed to boost NAD+ levels. Manufactured in an ISO9001 certified facility Boost Your NAD+ Levels — Healthy Ageing: Redefined Cellular Health Energy & Focus Bone Density Skin Elasticity DNA Repair Cardiovascular Health Brain Health Metabolic Health My ‘Fierce' Sports Jewellery Collection For my gorgeous and inspiring sports jewellery collection, 'Fierce', go to https://shop.lisatamati.com/collections/lisa-tamati-bespoke-jewellery-collection. Episode Highlights 01:04 Lisa's Journey with Cancer and her Mom Lisa's mother has been diagnosed with high-grade diffuse B-cell lymphoma. These recent events have motivated Lisa to delve deeper into cancer research. Understanding how cancer works and the approaches besides traditional procedures such as chemotherapy and radiation are necessary and relevant. In the following episodes, she will be doing a series of interviews revolving around the theme of cancer. 09:12 How Cancer Rewrites the Definition of Healthy When diagnosed with cancer, building muscles and promoting metabolic activity becomes less of a priority. The goal of cancer treatment is to slow everything down. We want to get rid of damaged cells and generate new ones. Dr Yurth adds that while it is vital to maintain glucose levels and everything, the general goal is to block muscle growth. Cancer begins at a mitochondrial level, and cancer cells need tons of energy to support their growth rate. We need to block all the typical pathways that make us healthy to stop its growth. 12:58 Spermidine and Autophagy Pathways The effects of spermidine are different for different types of cancer, but it appears to be very beneficial to patients suffering from colon cancer. Blocking the polyamine pathways in prostate cancer has been good in getting rid of cancer cells. A high dose of spermidine will result in a dramatic autophagy effect. However, Dr Yurth advises that she wouldn't recommend this until research reveals more information. Another good approach is doxycycline. Mitochondria started as bacteria and have their own DNA. Since doxycycline is an antibacterial agent, it works well in getting rid of these mitochondria. Dr Yurth advises us to stick to approaches that have better answers. 17:19 Autophagy as Recycling Cancer cells can use recycled proteins as another fuel source, so it's vital to block these pathways. Dipyridamole is a drug that works well in blocking nucleotide salvage pathways. However, chemotherapeutic drugs work on only one pathway, and cancer cells tend to find other pathways to get their energy, which makes working against these cells challenging. 19:00 Fight Cancer By Starving It Dr Yurth remarks that McClellan's work that includes safe, inexpensive, and readily available drugs in combating cancer is an innovative approach. She recommends doing them hand in hand with conventional approaches to cancer. Most oncologists will discourage patients from taking unconventional practices with cancer treatments. 21:25 Taking Others' Input, Making Own Decisions We need new scientific research about making decisions about cancer treatment. For example, there may be detrimental effects to patients undergoing unconventional approaches. Dr Yurth adds that she does extensive research in everything she does. 23:00 Drug Interactions Understanding drug interactions is vital in understanding cancer treatments. However, oncologists only look at it at the metabolic cellular level and follow their study protocols. 23:34 Understanding the Metabolism of Cancer Cancer is not just a genetic disease. While genetics causes a disposition towards cancer, the disease itself starts in the mitochondria. The mitochondria are independent and can become cancerous. There is no homogenous pattern when it comes to cancer cells mutating. That is why we need to treat the mitochondria and target metabolism. 26:20 Progress in Understanding Cancer There has been headway in understanding cancer, but you'll realise that research in the field has gotten further if we put everything together. The immune system responds appropriately to keep the cancer cells in check. However, as we age, the immune system becomes dysfunctional and can no longer get rid of cancer cells. Cancer is an age-related disease because the immune system starts to deteriorate. Viruses like cytomegalovirus, Epstein Barr virus (EBV), and COVID end up sticking around and dysregulating the immune system. Over time, these viruses can eventually link to a form of cancer. 28:40 Viruses and their Relationship to the Immune System Over time, the immune system becomes depleted in keeping check with viruses in the body. The ratio of neutrophils and lymphocytes is vital for keeping your immune system working well. Having disproportionate lymphocytes means that your immune system can't protect your body from viruses. Dr Yurth shares that her son's white blood cell count dropped after experiencing EBV and COVID at the same time. Many people don't appreciate how viruses play a role in the long haul in our immune system. Dr Yurth suggests looking more aggressively at latent viruses in the body. Getting a simple CBC can determine how your immune system is currently faring. 33:53 What Peptides Are Peptides are, in a sense, small proteins. A protein has more than 50 amino acids, and peptides have less than 50 amino acids. Our body makes thousands of peptides, each made with a specific function. Synthetic peptides are also present in the body. The thymus gland, responsible for producing peptides, becomes smaller as we age. For patients with autoimmune diseases, giving back peptide thymosin alpha 1 in doses helps their immune system fight viruses. Some countries use a drug called Daxon for this purpose or as an adjunct to vaccines. Immune system support is critical for cancer patients going through chemotherapy. However, this remains an expensive and inaccessible approach. 39:12 Emerging Home Diagnostic Tests in the Market It is vital to get ahead and find early signs of diseases through testing. Dr Yurth shares an anecdote of her patient with dementia. There is a correlation between the brain scan and the immune system test results, which pointed towards a viral influence. Your lymphocyte count can be the marker in determining the status of your immune system. 40:52 Being Pre-emptive and Preventive Your GP typically has you undergo a CBC test, but the results of this test tend to go overlooked. The risk of dying becomes dramatically higher when your blood count is above 1.5 to 1. Albumin dropping is a sign that the body is in a stressed state. This marker predicts if your body will respond well to cancer. Cell size and red blood cell distribution can also help healthcare providers determine the health status of your cells. 43:13 Background on Dr Yurth's BLI Academy Course The course will be available in January and includes a blood test. Dr Yurth and her team will walk you through your test results and the possible diseases you need to consider. It's essential to look for markers that tell you if something's wrong with your body. There are long term ramifications to latent viruses such as EPV, which could lead to lymphoma. 46:25 COVID and How You Can Support the Immune System Vitamin D is essential to support your immune system fight viruses, and by extension, fight cancer and the coronavirus. Dr Yurth recommends 1000 IU/day of Vitamin D and taking it with Vitamin K2 intracellularly. People lacking a binding protein may experience issues taking Vitamin D naturally through sunlight exposure. Fortunately, there are substitutes, including certain wavelength lamps and supplements. Sunlight exposure and vitamin D influences the frequency of colds and flu in winter. The vitamin D levels from the sun decrease during this season. 50:07 The Importance of Vitamin D Vitamin D upregulates an antimicrobial peptide, LL 37, which fights infections. Instead of buying expensive LL 37, you can take more vitamin D and upregulate your own LL 37. It is a great antiviral agent as well. Some people fear getting too much vitamin D. However, studies have shown that you need to be taking massive amounts before it becomes toxic. Dr Yurth recollects the case of a patient taking around 500,000 IU/day. When she stopped taking massive amounts, her levels went back to normal. 52:17 The Problem with Taking Vitamin D Naturally Wearing sunblock can protect us from skin cancer and wrinkles but can also affect vitamin D absorption. African Americans and people of Mediterranean origin have skin less likely to be damaged by sun rays but tend to have lower vitamin D levels. In contrast, people who have Norwegian and similar European origins have higher vitamin D levels because their bodies aren't under the sun all the time. Through testing, you can determine if you need more vitamin D in your body. 53:54 The Importance of Zinc Viruses use your zinc to replicate. Zinc deficiency is present in people who contracted COVID, causing loss of taste and smell. A smell test is employed to determine whether a person has zinc deficiency. Higher levels of zinc can help protect our bodies against viral infections. Dr Yurth recommends a 1:1 zinc and copper ratio. You can add 2 milligrams of copper along with your zinc intake or use copper peptide face creams to achieve this recommendation. Some patients can take GHK Copper peptide intravenously or topically. 56:52 Quercetin and Resveratrol Quercetin helps get zinc into the cell. Resveratrol and Quercetin also help in treating viruses. These drugs also block glucose transport which makes them beneficial in treating cancer. 57:52 Honokiol or Magnolia Bark Extract and Zeolite Also known as Relora, it is an immune-modulating agent with antiviral properties. The drug has an adrenal function; it relieves stress and calms down cortisol present in the body at night. Zeolite is volcanic ash that forms a negatively charged structure upon hitting the water. Dentists commonly use it for mercury detox. Its cage-like structure can trap harmful toxins and viruses that get into our bodies. It also has anti-cancer properties. Ingesting zeolite while undergoing chemotherapy is not recommended because it can get rid of the toxins present in the treatment. 1:02:04 Mental Work in the Cancer Journey As a caretaker, stress shows prominent signs in physical and mental health. Dr Yurth shares that when her mother died in September, her father's prostate cancer was exacerbated by stress, which eventually led to his death. Similarly, Lisa's family experienced the loss of her father, followed by her mother's cancer diagnosis. Don't neglect the mental side of cancer. Resources Gain exclusive access and bonuses to Pushing the Limits Podcast by becoming a patron! How to Starve Cancer by Jane McLelland Visit Lorraine Picard's website for copper peptide creams here. Interested in BLI Academy Courses? Check out their offered services here. Learn more from Dr Elizabeth Yurth from her previous podcast episodes: Episode 187: Back to Basics: Slow Down Ageing and Promote Longevity with Dr Elizabeth Yurth Rethinking the Function of Mitochondria for Our Health with Dr Elizabeth Yurth You can also check out Dr Elizabth Yurth's podcast with the Boulder Longevity Institute here! Connect with Dr Elizabeth Yurth: Instagram l LinkedIn 7 Powerful Quotes “What happens in cancer though, is we have this accumulation of damage. And so now we have to go really aggressively at getting rid of everything that's allowing cell growth.” “Neutrophils, think of them as your chronic inflammatory and lymphocytes are your innate immune system. When your innate immune system starts to become taxed and not functioning well, then you'll start to see your lymphocyte count drop,” “I'm never an advocate of saying, Don't do all traditional approaches, because I do think there's some value to them, but we want to support the body along with it.” “And yet, we know that for every point above 1.5, to 1 that your risk of dying goes up considerably. You know, it's dramatic. And there are other little signs too, that things are going awry.” “(A higher risk) happens as we get older because our immune system is now taxed for so long. So if we could say okay, yes, you had this virus, let's try and eradicate it. Let's keep your immune system as healthy as it can, you know, until 150.” “The data really supports that if your vitamin D is robust, your likelihood of getting sick is really small.” “The studies have shown that it has to be taking massive amounts of (vitamin D) to ever become toxic. So at 10,000 IU/day, nobody is going to become toxic. You really honestly don't have to worry.” About Dr Yurth Dr Elizabeth Yurth is the co-founder and medical director of the Boulder Longevity Institute. Dr Elizabeth Yurth has more than 25 years of experience as a practising orthopedist specialising in sports, spine, and regenerative medicine. She has a Stanford affiliated Fellowship in Sports and Spine Medicine and a dual-Fellowship in Anti-Aging and Regenerative Medicine (FAARM) and Anti-Aging, Regenerative and Functional Medicine (FAARFM) through the American Academy of Anti-Aging Medicine (A4M). She also has a Fellowship in Human Potential and Epigenetic Medicine and is one of the first providers to receive the A4M National Peptide Certification. She also serves as a faculty member in the same organisation and the International peptide Society (IPS). You can connect with Dr Yurth through her Instagram and LinkedIn. Enjoyed This Podcast? If you did, be sure to subscribe and share it with your friends! Post a review and share it! If you enjoyed tuning in, then leave us a review. You can also share this with your family and friends so they can know how to optimise their immune system and fight disease more efficiently. Have any questions? You can contact me through email (support@lisatamati.com) or find me on Facebook, Twitter, Instagram and YouTube. For more episode updates, visit my website. You may also tune in on Apple Podcasts. To pushing the limits, Lisa
Galera hoje vou falar sobre o relora! Um excelente fitoterápico para controle do stress e ansiedade!Dando continuidade aos vídeos aqui no canal sobre ativos e opção para stress e ansiedade, hoje eu vou falar da opção que eu mais gosto para essa finalidade, que é o Relora.
Stefan Molyneux interviews Dr Paul Cottrell about the looming health, financial and economic effects of the #coronavirus pandemic.What is the latest news on death rates? Are they going down? Was the danger of the pandemic overblown?How long will it last, and what should you do to prepare?www.the-studio-reykjavik.com/Paul's Channel: https://www.youtube.com/channel/UCZyk2NYx6wGnpoJ7ApTxWKgDr Paul Cottrell is a researcher in chaos theory and has interests in modeling financial markets. Some have considered him a polymath of sorts. Born in Detroit, Michigan he has extensive professional experience in engineering and design. After retiring from automotive engineering he pursued interests in corporate finance involving the publishing, retail, and non-profit industries. He currently is a proprietary trader specializing in currency and energy markets.In addition to modeling financial markets, his research interests are in developmental economics, behavioral finance, complexity science, energy industry, risk management, and the uses of artificial intelligence in trading financial markets. He has been awarded a Ph.D. specializing in finance from Walden University. He has also earned a M.B.A. and B.S. from Wayne State University. Engineering certificates from Massachusetts Institute of Technology (M.I.T.) have also been awarded to him. Dr. Cottrell is currently at Harvard University as an ALM candidate specializing in Biology and has completed the Pre-Medical program at Fordham University.Dr Cottrell uses:For sickness:Forskolincalendula Reloralicorice Root ExtractBirch Bark ExtractNano-silver (gel or liquid) for immune systemAnti-inflammation (cytokine storm)Rosemary high doses Turmeric high dosesHis daily protocol:Multi vitaminsZinc Iodine dropsNMNPQQ & CoQ10 (DNA force)Brain ForceBodease (turmeric)C60Vit CK2 MK-7D3Probiotic Filtered water (zero water)▶️ Donate Now: www.freedomain.com/donate▶️ Sign Up For Our Newsletter: www.fdrurl.com/newsletterYour support is essential to Freedomain, which is 100% funded by viewers like you. Please support the show by making a one time donation or signing up for a monthly recurring donation at: www.freedomain.com/donate▶️ 1. Donate: www.freedomain.com/donate▶️ 2. Newsletter Sign-Up: www.fdrurl.com/newsletter▶️ 3. On YouTube: Subscribe, Click Notification Bell▶️ 4. Subscribe to the Freedomain Podcast: www.fdrpodcasts.com▶️ 5. Follow Freedomain on Alternative Platforms🔴 Bitchute: bitchute.com/freedomainradio🔴 Minds: minds.com/stefanmolyneux🔴 Steemit: steemit.com/@stefan.molyneux🔴 Gab: gab.ai/stefanmolyneux🔴 Twitter: www.twitter.com/stefanmolyneux🔴 Facebook: facebook.com/stefan.molyneux🔴 Instagram: instagram.com/stefanmolyneuxAmazon Affiliate Links▶️ US: www.fdrurl.com/Amazon▶️ Canada: www.fdrurl.com/AmazonCanada▶️ UK: www.fdrurl.com/AmazonUK
Coronavirus Update - ENGINEERED? Dr. Paul Cottrell and Stefan Molyneux Watch the interview at- https://youtu.be/fTDWg-nftHQ Stefan Molyneux Stefan Molyneux interviews Dr Paul Cottrell about the looming health, financial and economic effects of the #coronavirus pandemic. How long will it last, and what should you do to prepare? Audio: https://soundcloud.com/stefan-molyneu... www.the-studio-reykjavik.com/ Paul's Channel: https://www.youtube.com/channel/UCZyk... Dr Paul Cottrell is a researcher in chaos theory and has interests in modeling financial markets. Some have considered him a polymath of sorts. Born in Detroit, Michigan he has extensive professional experience in engineering and design. After retiring from automotive engineering he pursued interests in corporate finance involving the publishing, retail, and non-profit industries. He currently is a proprietary trader specializing in currency and energy markets. In addition to modeling financial markets, his research interests are in developmental economics, behavioral finance, complexity science, energy industry, risk management, and the uses of artificial intelligence in trading financial markets. He has been awarded a Ph.D. specializing in finance from Walden University. He has also earned a M.B.A. and B.S. from Wayne State University. Engineering certificates from Massachusetts Institute of Technology (M.I.T.) have also been awarded to him. Dr. Cottrell is currently at Harvard University as an ALM candidate specializing in Biology and has completed the Pre-Medical program at Fordham University. Dr Cottrell uses: For sickness: Forskolin calendula Relora licorice Root Extract Birch Bark Extract Nano-silver (gel or liquid) for immune system Anti-inflammation (cytokine storm) Rosemary high doses Turmeric high doses His daily protocol: Multi vitamins Zinc Iodine drops NMN PQQ & CoQ10 (DNA force) Brain Force Bodease (turmeric) C60 Vit C K2 MK-7 D3 Probiotic Filtered water (zero water) ▶️ Donate Now: www.freedomain.com/donate ▶️ Sign Up For Our Newsletter: www.fdrurl.com/newsletter Your support is essential to Freedomain, which is 100% funded by viewers like you. Please support the show by making a one time donation or signing up for a monthly recurring donation at: www.freedomain.com/donate ▶️ 1. Donate: www.freedomain.com/donate ▶️ 2. Newsletter Sign-Up: www.fdrurl.com/newsletter ▶️ 3. On YouTube: Subscribe, Click Notification Bell ▶️ 4. Subscribe to the Freedomain Podcast: www.fdrpodcasts.com ▶️ 5. Follow Freedomain on Alternative Platforms
My Foundational Five are the best supplements to help form a foundation of good nutrition, no matter what your goals may be. Before I recommend the best muscle-building supplements, weight loss pills, or detox programs, I almost always recommend starting with the following five supplements. Then, as you make progress toward your goals, you might consider adding to these five, but not taking something in place of them. Of course, as potent as these supplements are, you still need to eat well, too. As I often say... No supplement is so good it can offset the effect of a poor diet. No diet is so good that it can't benefit from supplementation. Your nutrition should complement a good strength training program, and quality sleep, as well. If you're serious about your health, consider making the following five supplement nonnegotiables each day. I've included the specific products I use and recommend as part of my Foundational Five as well. The Wellevate buttons take you to my practitioner dispensary, where you can access more than 300 brands of supplements. 1. High-Quality Multivitamin Imagine building your dream home without using any screws, nails, glue, or caulk. You could use all the best wood, granite, and glass available. But if you don't have the materials to tie it all together, your home won't stand very long. A high-quality multivitamin provides the micronutrients necessary to avoid gaps that could cause your metabolism, and then your body, to break down. When I recommend a high-quality multivitamin, I mean high-quality. Here's what to look for in a high-quality multivitamin: Natural folate, not folic acid (labeled as Metafolin®, L-5-methyltetrahydrofolate, or methylated folate)Methylcobalamin instead of cyancobalaminMineral glycinate chelates (i.e. magnesium bisglycinate, magnesium glycinate)Both K1 and K2Exceeds Daily Value (DV) for most micronutrientsEfficacious doses of any extra ingredients to enhance the formula This is a starting point. It's not an all-encompassing list of what it takes to make the best multivitamin. .ugb-074c6d2 .ugb-button{background-color:#3578e5;border-radius:5px !important}.ugb-074c6d2 .ugb-button .ugb-button--inner,.ugb-074c6d2 .ugb-button svg:not(.ugb-custom-icon){color:#ffffff !important}.ugb-074c6d2 .ugb-button:before{border-radius:5px !important}.ugb-074c6d2 .ugb-button-container{text-align:center !important}@media screen and (min-width:768px){.ugb-074c6d2 .ugb-img{width:1000px;height:auto !important}}Thorne Multi-Vitamin EliteHighly bioavailable nutrients for athletes and adults committed to looking, feeling, and performing at their best.* Green Tea Phytosome to enhance energy production, Curcumin Phytosome to support normal inflammation levels, Relora® to enhance recovery and calm the mind.*ORDER FROM WELLEVATE .ugb-71c5a8d .ugb-button{background-color:#3578e5;border-radius:5px !important}.ugb-71c5a8d .ugb-button .ugb-button--inner,.ugb-71c5a8d .ugb-button svg:not(.ugb-custom-icon){color:#ffffff !important}.ugb-71c5a8d .ugb-button:before{border-radius:5px !important}.ugb-71c5a8d .ugb-button-container{text-align:center !important}@media screen and (min-width:768px){.ugb-71c5a8d .ugb-img{width:1024px;height:auto !important}}Young Living Master FormulaYoung Living Master Formula is a high-quality multi-vitamin, multi-mineral formula distributed by Young Living Essential Oils. In addition to high-quality micronutrients, it contains a variety of botanicals and essential oils.ORDER FROM YOUNG LIVING 2. Fish Oil According to one survey, almost 63% of people knew they did not get enough omega-3 fats in their diet. However, only 26% of people supplemented with omega-3s. Omega-3 fatty acids play an essential role in your brain health, as well as the function of almost every system in the body. Like micronutrients, omega-3s are essential for health. High-quality fish oil is the best way to increase your omega-3 fatty acid levels. ...
The calories you burn in your workout today doesn't matter. Wait, what?That's right. It's not the energy expenditure of your workout that is going to stick with you. It's the long-lasting adaptations that your body creates due to the stress of your workout.The magic of your workout happens AFTER the workout.Adaptations like Rewiring of your cerebellum to prevent neurodegenerative diseaseImproved muscle density, size, and functionExcessive post-exercise oxygen consumptionStrength, power, stability, range of motionHormone modulationInjury preventionAre all paramount compared to caloric burnAnd it turns out that there are some great "secrets" that are proven to scientifically support your workout effort by boosting your recovery efforts! Mentioned in the show Weekly Updatesdefiningdadbod.com/ddb1Summer Shreddefiningdadbod.com/shredHow To Work Out While Sleep Depriveddefiningdadbod.com/blog/9Relora (for DHEA)https://www.amazon.com/gp/product/B0013YW7VC/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B0013YW7VC&linkCode=as2&tag=definingdadbo-20&linkId=309eb4cfcf945f373403399904e0cd4cReferences[1] Talbott SM, Talbott JA, Pugh M. Effect of Magnolia officinalis and Phellodendron amurense (Relora®) on cortisol and psychological mood state in moderately stressed subjects. Journal of the International Society of Sports Nutrition. 2013;10:37. doi:10.1186/1550-2783-10-37.[1] Increased free cortisol secretion after awakening in chronically stressed individuals due to work overload. Stress Medicine 1998; 14:91-97.[1] Shimomura Y, Murakami T, Naoya Nakai N, Nagasaki M, Harris RA (2004). “Exercise Promotes BCAA Catabolism: Effects of BCAA Supplementation on Skeletal Muscle during Exercise”. J. Nutr. 134 (6): 1583S–1587S. Retrieved 22 March 2011.[1] A O'Connor, S Simbo, et. Al. Powdered tart cherry supplementation moderates post-exercise immunosuppression, total cholesterol, and antioxidant status with no effect on performance recovery following an acute bout of intense lower body resistance exercise. A O'Connor, S Simbo, et. Al. Journal of the International Society of Sports Nutrition 2014 11(Suppl 1):P32. DOI: 10.1186/1550-2783-11-S1-P32.[1] Fassett, Robert G.; Coombes, Jeff S. (2009). “Astaxanthin, oxidative stress, inflammation and cardiovascular disease”. Future Cardiology 4 (3): 333–342.doi:10.2217/fca.09.19. PMID 19656058.[1] Hsu, C. H.; Cheng, A. L. (2007). “Clinical studies with curcumin”. Advances in Experimental Medicine and Biology 595: 471–480. doi:10.1007/978-0-387-46401-5_21. ISBN 978-0-387-46400-8. PMID 17569225.[1] Ftaiti F, Jemni M, Kacem A, Zaouali MA, Tanbka Z, et al. Effect of hyperthermia and physical activity on circulating growth hormone. Appl Physiol Nutr Metab. 2008;33(5):880-887[1] http://www.scientificamerican.com/podcast/episode/meditation-meta-analysis/[1] B L Alderman, R L Olson, C J Brush, and T J Shors. MAP training: combining meditation and aerobic exercise reduces depression and rumination while enhancing synchronized brain activity. Department of Exercise Science, Rutgers, The State University of New Jersey. Received 5 October 2015; Revised 1 December 2015; Accepted 5 December 2015Become a supporter of this podcast: https://www.spreaker.com/podcast/faithful-fitness-by-better-daily--5150768/support.
The calories you burn in your workout today doesn't matter. Wait, what?That's right. It's not the energy expenditure of your workout that is going to stick with you. It's the long-lasting adaptations that your body creates due to the stress of your workout.The magic of your workout happens AFTER the workout.Adaptations like Rewiring of your cerebellum to prevent neurodegenerative diseaseImproved muscle density, size, and functionExcessive post-exercise oxygen consumptionStrength, power, stability, range of motionHormone modulationInjury preventionAre all paramount compared to caloric burnAnd it turns out that there are some great "secrets" that are proven to scientifically support your workout effort by boosting your recovery efforts! Mentioned in the show Weekly Updatesdefiningdadbod.com/ddb1Summer Shreddefiningdadbod.com/shredHow To Work Out While Sleep Depriveddefiningdadbod.com/blog/9Relora (for DHEA)https://www.amazon.com/gp/product/B0013YW7VC/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B0013YW7VC&linkCode=as2&tag=definingdadbo-20&linkId=309eb4cfcf945f373403399904e0cd4cReferences[1] Talbott SM, Talbott JA, Pugh M. Effect of Magnolia officinalis and Phellodendron amurense (Relora®) on cortisol and psychological mood state in moderately stressed subjects. Journal of the International Society of Sports Nutrition. 2013;10:37. doi:10.1186/1550-2783-10-37.[1] Increased free cortisol secretion after awakening in chronically stressed individuals due to work overload. Stress Medicine 1998; 14:91-97.[1] Shimomura Y, Murakami T, Naoya Nakai N, Nagasaki M, Harris RA (2004). “Exercise Promotes BCAA Catabolism: Effects of BCAA Supplementation on Skeletal Muscle during Exercise”. J. Nutr. 134 (6): 1583S–1587S. Retrieved 22 March 2011.[1] A O’Connor, S Simbo, et. Al. Powdered tart cherry supplementation moderates post-exercise immunosuppression, total cholesterol, and antioxidant status with no effect on performance recovery following an acute bout of intense lower body resistance exercise. A O’Connor, S Simbo, et. Al. Journal of the International Society of Sports Nutrition 2014 11(Suppl 1):P32. DOI: 10.1186/1550-2783-11-S1-P32.[1] Fassett, Robert G.; Coombes, Jeff S. (2009). “Astaxanthin, oxidative stress, inflammation and cardiovascular disease”. Future Cardiology 4 (3): 333–342.doi:10.2217/fca.09.19. PMID 19656058.[1] Hsu, C. H.; Cheng, A. L. (2007). “Clinical studies with curcumin”. Advances in Experimental Medicine and Biology 595: 471–480. doi:10.1007/978-0-387-46401-5_21. ISBN 978-0-387-46400-8. PMID 17569225.[1] Ftaiti F, Jemni M, Kacem A, Zaouali MA, Tanbka Z, et al. Effect of hyperthermia and physical activity on circulating growth hormone. Appl Physiol Nutr Metab. 2008;33(5):880-887[1] http://www.scientificamerican.com/podcast/episode/meditation-meta-analysis/[1] B L Alderman, R L Olson, C J Brush, and T J Shors. MAP training: combining meditation and aerobic exercise reduces depression and rumination while enhancing synchronized brain activity. Department of Exercise Science, Rutgers, The State University of New Jersey. Received 5 October 2015; Revised 1 December 2015; Accepted 5 December 2015
The calories you burn in your workout today doesn't matter. Wait, what?That's right. It's not the energy expenditure of your workout that is going to stick with you. It's the long-lasting adaptations that your body creates due to the stress of your workout.The magic of your workout happens AFTER the workout.Adaptations like Rewiring of your cerebellum to prevent neurodegenerative diseaseImproved muscle density, size, and functionExcessive post-exercise oxygen consumptionStrength, power, stability, range of motionHormone modulationInjury preventionAre all paramount compared to caloric burnAnd it turns out that there are some great "secrets" that are proven to scientifically support your workout effort by boosting your recovery efforts! Mentioned in the show Weekly Updatesdefiningdadbod.com/ddb1Summer Shreddefiningdadbod.com/shredHow To Work Out While Sleep Depriveddefiningdadbod.com/blog/9Relora (for DHEA)https://www.amazon.com/gp/product/B0013YW7VC/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B0013YW7VC&linkCode=as2&tag=definingdadbo-20&linkId=309eb4cfcf945f373403399904e0cd4cReferences[1] Talbott SM, Talbott JA, Pugh M. Effect of Magnolia officinalis and Phellodendron amurense (Relora®) on cortisol and psychological mood state in moderately stressed subjects. Journal of the International Society of Sports Nutrition. 2013;10:37. doi:10.1186/1550-2783-10-37.[1] Increased free cortisol secretion after awakening in chronically stressed individuals due to work overload. Stress Medicine 1998; 14:91-97.[1] Shimomura Y, Murakami T, Naoya Nakai N, Nagasaki M, Harris RA (2004). “Exercise Promotes BCAA Catabolism: Effects of BCAA Supplementation on Skeletal Muscle during Exercise”. J. Nutr. 134 (6): 1583S–1587S. Retrieved 22 March 2011.[1] A O’Connor, S Simbo, et. Al. Powdered tart cherry supplementation moderates post-exercise immunosuppression, total cholesterol, and antioxidant status with no effect on performance recovery following an acute bout of intense lower body resistance exercise. A O’Connor, S Simbo, et. Al. Journal of the International Society of Sports Nutrition 2014 11(Suppl 1):P32. DOI: 10.1186/1550-2783-11-S1-P32.[1] Fassett, Robert G.; Coombes, Jeff S. (2009). “Astaxanthin, oxidative stress, inflammation and cardiovascular disease”. Future Cardiology 4 (3): 333–342.doi:10.2217/fca.09.19. PMID 19656058.[1] Hsu, C. H.; Cheng, A. L. (2007). “Clinical studies with curcumin”. Advances in Experimental Medicine and Biology 595: 471–480. doi:10.1007/978-0-387-46401-5_21. ISBN 978-0-387-46400-8. PMID 17569225.[1] Ftaiti F, Jemni M, Kacem A, Zaouali MA, Tanbka Z, et al. Effect of hyperthermia and physical activity on circulating growth hormone. Appl Physiol Nutr Metab. 2008;33(5):880-887[1] http://www.scientificamerican.com/podcast/episode/meditation-meta-analysis/[1] B L Alderman, R L Olson, C J Brush, and T J Shors. MAP training: combining meditation and aerobic exercise reduces depression and rumination while enhancing synchronized brain activity. Department of Exercise Science, Rutgers, The State University of New Jersey. Received 5 October 2015; Revised 1 December 2015; Accepted 5 December 2015
Podcast Episode 108 - Relora Soooo...this is the last part, in the 4 part series talking about the Youtube video I saw of he Dr. Oz Show. If you are just linking with me, then let me tell you that not too long ago, I came across a video on Youtube of Dr. Oz talking about 4 belly blasters/weight loss supplements. Relora is one of them. In fact, its the last of the 4 items on the Dr. Oz Show list. So, if you enjoyed the other 3, or if you are just joining me now, and you are a stress eater...take a listen here for todays podcast! Please remember to Subscribe, Rate & Review! To leave a comment, or feedback, leave a message at 404-828-0051
Episode 17 - Continuing the Wizkids open coverage, we have two more of the RiF blind boosters. As an unexpected treat, while examining Portland winner Mark's Weyoun/Varel Tac Cube build, the man himself is able to join us in the chat and we get to pick his brain a little bit! If you cannot see the audio controls, your browser does not support the audio element
Hi. In the next three minutes I want to share with you some important stuff I've learned about stress. Stress is something I suffer from myself quite a bit and it has been a long hard road to find out what works for me. I will also give you a natural product that I have found to be very helpful when I am feeling overwhelmed. We all suffer from stress particularly now in this modern world with the aid of modern technology where we are expected to do the job of 2 or 3 people as well as the demands that we place on our selves. Here's the big secret though. It's not stress itself that causes the issue. It's how we respond to it that matters. When you learn how to make stress work with you instead of against you, you can free yourself from a lot of unwanted pain. It almost becomes a zen practise where you learn to eliminate unnecessary stressors in your own life. There are often more than you think that can easily be let go off. This simple practise can help free up a lot of energy. Here are a few other things that can help. Remember to take short breaks often. Get up, stretch you legs and remember to drink a lot of pure water. Remember to get some good exercise. walking, running, dancing even some gently weight training. and it's important to get outdoors and out into nature. Do it for at least 30 minutes three times a week. I personally like rock climbing as it get me out of my head and forces me just to be present. Do whatever exercise you like best. Next, the things we soft often do to help calm us temporarily when we get stress often just make things worse, such Cigarettes, drugs, and packaged convenience foods and drinks that are full of chemicals. These all stress the body and your mind and just lead to more overwhelm. Eliminate some of these things from your life and within a few weeks you will be more present and able to handle all the difficult tasks that come up. Now, the product I have found helpful when things start to get to me is called Calm Thoughts by Source Natural. It's formulated to reduce anxiety and stress while increasing calm focus whenever you need it. It's completely natural, based on what's most needed when we're stressed. In addition to important B complex vitamins, it is rich in GABA which I find particularly helpful to calm and clear the mind, L-Tyrosine to counter depression, Relora from the Magnolia to reduce anxiety by regulating cortisol and lots of other natural plant factors and important amino acids. You'll find a link to it here: http://www.iherb.com/Source-Naturals-Calm-Thoughts-90-Tablets/1092?rcode=yes026 Of course, read the label carefully and check it out with your health practitioner to make sure that it's right for you.
Hi. In the next three minutes I want to share with you some important stuff I’ve learned about stress. Stress is something I suffer from myself quite a bit and it has been a long hard road to find out what works for me. I will also give you a natural product that I have found to be very helpful when I am feeling overwhelmed. We all suffer from stress particularly now in this modern world with the aid of modern technology where we are expected to do the job of 2 or 3 people as well as the demands that we place on our selves. Here’s the big secret though. It’s not stress itself that causes the issue. It’s how we respond to it that matters. When you learn how to make stress work with you instead of against you, you can free yourself from a lot of unwanted pain. It almost becomes a zen practise where you learn to eliminate unnecessary stressors in your own life. There are often more than you think that can easily be let go off. This simple practise can help free up a lot of energy. Here are a few other things that can help. Remember to take short breaks often. Get up, stretch you legs and remember to drink a lot of pure water. Remember to get some good exercise. walking, running, dancing even some gently weight training. and it’s important to get outdoors and out into nature. Do it for at least 30 minutes three times a week. I personally like rock climbing as it get me out of my head and forces me just to be present. Do whatever exercise you like best. Next, the things we soft often do to help calm us temporarily when we get stress often just make things worse, such Cigarettes, drugs, and packaged convenience foods and drinks that are full of chemicals. These all stress the body and your mind and just lead to more overwhelm. Eliminate some of these things from your life and within a few weeks you will be more present and able to handle all the difficult tasks that come up. Now, the product I have found helpful when things start to get to me is called Calm Thoughts by Source Natural. It’s formulated to reduce anxiety and stress while increasing calm focus whenever you need it. It’s completely natural, based on what’s most needed when we’re stressed. In addition to important B complex vitamins, it is rich in GABA which I find particularly helpful to calm and clear the mind, L-Tyrosine to counter depression, Relora from the Magnolia to reduce anxiety by regulating cortisol and lots of other natural plant factors and important amino acids. You’ll find a link to it here: http://www.iherb.com/Source-Naturals-Calm-Thoughts-90-Tablets/1092?rcode=yes026 Of course, read the label carefully and check it out with your health practitioner to make sure that it’s right for you.
Hi. In the next three minutes I want to share with you some important stuff I’ve learned about stress. Stress is something I suffer from myself quite a bit and it has been a long hard road to find out what works for me. I will also give you a natural product that I have found to be very helpful when I am feeling overwhelmed. We all suffer from stress particularly now in this modern world with the aid of modern technology where we are expected to do the job of 2 or 3 people as well as the demands that we place on our selves. Here’s the big secret though. It’s not stress itself that causes the issue. It’s how we respond to it that matters. When you learn how to make stress work with you instead of against you, you can free yourself from a lot of unwanted pain. It almost becomes a zen practise where you learn to eliminate unnecessary stressors in your own life. There are often more than you think that can easily be let go off. This simple practise can help free up a lot of energy. Here are a few other things that can help. Remember to take short breaks often. Get up, stretch you legs and remember to drink a lot of pure water. Remember to get some good exercise. walking, running, dancing even some gently weight training. and it’s important to get outdoors and out into nature. Do it for at least 30 minutes three times a week. I personally like rock climbing as it get me out of my head and forces me just to be present. Do whatever exercise you like best. Next, the things we soft often do to help calm us temporarily when we get stress often just make things worse, such Cigarettes, drugs, and packaged convenience foods and drinks that are full of chemicals. These all stress the body and your mind and just lead to more overwhelm. Eliminate some of these things from your life and within a few weeks you will be more present and able to handle all the difficult tasks that come up. Now, the product I have found helpful when things start to get to me is called Calm Thoughts by Source Natural. It’s formulated to reduce anxiety and stress while increasing calm focus whenever you need it. It’s completely natural, based on what’s most needed when we’re stressed. In addition to important B complex vitamins, it is rich in GABA which I find particularly helpful to calm and clear the mind, L-Tyrosine to counter depression, Relora from the Magnolia to reduce anxiety by regulating cortisol and lots of other natural plant factors and important amino acids. You’ll find a link to it here: http://www.iherb.com/Source-Naturals-Calm-Thoughts-90-Tablets/1092?rcode=yes026 Of course, read the label carefully and check it out with your health practitioner to make sure that it’s right for you.
2 years ago, I decided I was fed up with supplements. Just freakin' tired of all the tablets, capsules, pills, powders and oils. And bottles. Lots and lots of plastic bottles scattered everywhere. I distinctly remember the day I finally decided something needed to change. It was when I swung open the door of my refrigerator to dive into my daily regimen of vitamins, antioxidants, nutrients, micronutrients, fish oils... ...and I realized it was getting overwhelming. Sure, it was better living through science, but it was anything but simple. I'm sure you've experienced the same thing. You open what seems like nearly a dozen different bottles to launch into your morning supplement regimen.... ...or you pack your suitcase to travel and struggle to prioritize which bottles to shove into your limited suitcase space... ...or you spend an inordinate amount of your precious time tracking, ordering and restocking your vitamin D or your fish oil, or your greens, or your multi-vitamin or your minerals or your antioxidants or your amino acids... You get the idea. Supplement clutter, supplement confusion and utter lack of supplement simplicity. ------------------------------------ 9 Reasons You May Not Want To Throw Out All Your Supplements Now don't get me wrong. I don't think supplements are an inconvenient and unnecessary curse that should simply all be tossed out. There is certainly evidence that , but even if you're eating a “healthy” diet, studies have shown that it is nearly impossible in our modern era to get all our nutrients, vitamins and minerals only from food. For example, a : "Large portions of the population had total usual intakes below the estimated average requirement for vitamins A (35%), C (31%), D (74%), and E (67%) as well as calcium (39%) and magnesium (46%). Only 0%, 8%, and 33% of the population had total usual intakes of potassium, choline, and vitamin K above the adequate intake when food and multivitamin use was considered. The percentage of the population with total intakes greater than the tolerable upper intake level (UL) was very low for all nutrients..." "...Conclusions: In large proportions of the population, micronutrient sufficiency is currently not being achieved through food solutions for several essential vitamins and minerals. Use of age- and gender-specific multivitamin supplements may serve as a practical means to increase the micronutrient status in subpopulations of Americans while not increasing intakes above the upper intake level." So how on earth could this be? Is the planet broken or something? There's actually nine reasons for the failure of modern diets to supply our bodies with all the nutrients we need. First, modern commercial farming practices strip our precious soil of nutrients. When plants are repeatedly grown on the same land for year-after-year (a relatively new phenomenon in agriculture that beats up our land but gives us lots and lots of calories), the overused soil loses it's nutrients faster than they can be replaced. Over time, the plants have fewer nutrients to grow, so the farmers are forced to fertilize. Fertilizer does indeed contain enough nutrition for a plant to survive until harvesting, but not enough nutrients to support your body's nutrient needs. As discussed in the excellent book "", this results in plants that up to 75% fewer micronutrients. This problem is compounded by the fact that most plants are not harvested fresh, and the nutrients degrade as they often sit on airplanes, trucks, shelves, and counters for weeks before they arrive on your plate. Second, most , and not maximum nutrients, and because of this much of your fresh produce that us humans would have historically relied upon for nutrients are instead . And of course, this is all passed on to the animals (including you!) that are eating the plants. For example, found copper levels have dropped by 90% in dairy, 55% in meat, and 76% in vegetables! Third, pesticides are making this problem even worse. Because polyphenols are produced as a defense against bugs and pathogens, . To make matters even more frustrating, even if you buy organic plants, So unless you're going out of your way to grow your own fresh fruits and vegetables in a pristine backyard garden which contains soil that you've treated with added minerals, you are at a huge risk for malnourishment and nutrient deficiency that adversely affects your performance, fat loss, recovery, digestion, brain, sleep, and hormone balance. Fourth, it's not just nutrient-poor plants that are affecting the nutrient status of the actual animals we rely upon for food, but it's also what many of those animals are being fed - especially grains. Compared to grass-fed meat, grain-fed meat is , but unfortunately, eating high-quality, organic grass-fed (and grass-finished) meat, pastured pork and naturally raised poultry is simply not a reality for many people living in urban environments or frequently traveling. Fifth, the nutrients in your food aren't the only things that are disappearing - your liquids are being affected too. Unless you're drinking from your own well, or relying upon fancy mineralized water such as Pellegrino, Perrier and Gerolsteiner, your bottled water, filter water and tap water is also - and if does indeed contain adequate minerals, these are typically accompanied by enormous levels of flouride and chlorine. So this means that unless you want to spend lots and lots of money on fancy water, you are stuck in a situation where you must filter your water to get rid of the dangerous stuff, then figure out other methods, such as supplementation, to get the minerals you'd normally be getting from the water. Unfortunately, milk is not much better than water. The majority of nutrients in milk are found in the fat, so while , most of us simply don't have ready access to it, especially at the typical grocery store. In addition, pasteurization can destroy many of the nutrients in both skim and full fat milk. , which increases your nutrient needs even more, creating a vicious cycle. So whether it's due to nutrient-poor plants, nutrient-poor animals, nutrient-poor milk, or nutrient-poor water, most diets are now so micronutrient deficient that they require on average . In other words, in our modern era, you would need to be stuffing your face all day long to actually get what your body needs, and this does not into account the fact that if you are trying to lose weight by restricting calories or if you are a physically active person like a triathlete, marathoner, Crossfitter, weightlifter, or other exercise enthusiast, then your risk of nutrient deficiency is going to be even higher! In other words, if you are A) trying to lose weight by not stuffing your body full of excessive calories or B) an athlete or physically active person, then you're pretty much guaranteed to be at a constantly malnourished nutrient deficit (unless you figure out a way to somehow squeeze in all those added nutrients). The older you get, the fewer nutrients you absorb, so as you age, the worse things become. And that's reasons six (weight loss), seven (exercise) and eight (age) that you may need a little help over and above what you'd get from food. So let's say you dig a well, grow your own produce, milk your own goats, eat strictly grass-fed meat and wild-caught fish, you're relatively young and you avoid excessive physical activity or calorie restriction for weight loss. Then you're safe, right? No added nutrients necessary, right? The unfortunate ninth fact is that unless you live on a pristine mountaintop in the Himalayas, you're exposed every day to airborne pollutants, xenoestrogens from plastics and BPA, environmental toxins, mold, industrial solvents, electromagnetic fields, heavy metals, fluorescent lighting, constant stress, sleep disruption and a host of other modern-day assailants upon your biology. Your body simply wasn't mean to deal with these physiological stressors - all of which vastly increase your need for nutrients to help you fight the daily battle. So for the nine reasons above - depleted soil, nutrient-poor produce, pesticides, conventional meat, subpar water and milk, weight loss, physical activity, age, and our modern post-industrial living environment, you definitely shouldn't throw out all your supplements. --------------------------------------- Is It Possible For Supplements To Be Simpler? So this was the conundrum I found myself in two years ago as I stood in front of my refrigerator door and sighed in frustration at the dizzying array of supplements and bottles. I knew that because of my extremely active lifestyle and the other reasons I just listed that I couldn't and shouldn't just throw out all my supplements. I knew that I needed to keep my precious nutrients topped off for better performance, faster recovery, optimized fat burning, clean digestion, superior mental focus, better sleep and hormone balance... ...but I also knew that I was sick and tired of having to lug around a bunch of bottles, painstakingly use a weekly pill box to keep track of my supplements like an old man tracks his medication, and constantly be headed over to Amazon or some other website to re-order something I was running low in. And then there was that gnawing worry at the back of my mind that any of these supplements I was ingesting could be laced with something like chemicals, toxins, metals or illegal-performance enhancing drugs that could potentially damage my body, put pie in my face and get me banned from competing in a sanctioned sport like triathlon. I wanted better living through science. But I also desperately needed more simplicity. And I knew that somewhere, somehow, there must be a way to have both: the ultimate combination of science and simplicity. --------------------------------------- Seven Criteria a Multivitamin Must Have At that point, I embarked upon my quest to discover the greatest multivitamin on the face of the planet - something that could cover all my bases from a nutrient standpoint without requiring a dizzying, inconvenient and exhausting array of pills, bottles and packets. After all, that's kind of the definition of a multivitamin, right? It supposed to cover multiple needs. As a nutritionist, personal trainer, exercise physiologist and strength and conditioning coach with strict standards about what I find acceptable for myself or my clients to ingest, I also had a very distinct set of criteria that I knew a multivitamin must have - seven criteria to be precise... --------------------------------------- My Multivitamin Criteria #1: Capsules, Not Tablets Before completely shoving tablets under the bus, I'll acknowledge the two benefits tablets possess. First, because tablets are pressed together so hard, more nutrients can fill the same amount of space than you can put into a capsule. Second, it’s cheaper to make tablets than capsules. There are several drawbacks, however, to using tablets. First, tablets are much harder to digest. Because tablets are packed together so tightly and need far more binders and diluents to keep them together, it takes much longer for them to break down in the digestive system. A capsule, on the other hand, breaks down within just minutes of hitting your digestive tract. This is important because of the little known fact that most of the nutrients in a multivitamin actually need to be absorbed in your stomach. When a tablet takes too long to break down in the stomach, and passes on to your intestinal tract, its content will be far less broken down and absorbed. And that means expensive pee. Tablets also need lots of extra nasty ingredients, such as binders, lubricants, coatings, disintegrants, and other excipients. These ingredients must be added to make the tablet stay together, to make the tablet shiny, and to make the tablet break apart. Tablet press machines compact the powdered nutrients together (that have already been mixed with binders and lubricants) with a tremendous amount of force. Then, to make the tablets shiny and easier to swallow, the tablets are often sprayed with coatings such as shellac (just like the shellac found on furniture but instead labeled “pharmaceutical glaze”) or coated with “vegetable protein,” (most often a protein derived from GMO corn). Yuk. I'll take a capsule, please. --------------------------------------- My Multivitamin Criteria #2: One Morning Dose & One Evening Dose Your body needs nutrients throughout the course of the day, not just one huge shotgun dose when you get out of bed in the morning or before you go to bed at night. So I needed to find a multivitamin designed in such a way that it delivered important nutrients for daily metabolism and activity in the morning dose, and important nutrients for recovery, rest, and sleep in the evening dose, but without a dizzying array of capsules. I'll admit that it does take slightly more discipline to take a multivitamin twice a day – once in the morning and once in the evening – but it’s a much better way to deliver the optimal nutrients at the optimal times throughout the day. After all, you'd rarely eat only breakfast or only dinner, right? I figure if I can handle at least two meals, I can handle two supplement doses - as long it's not a dozen different bottles of stuff. --------------------------------------- My Multivitamin Criteria #3: Low Number of Capsules One extremely inconvenient factor in most multivitamin formulas out there is the number of capsules necessary for daily use if you truly are going to be able to get all the nutrients that you need from a single formula. I think that taking three capsules in the morning and three capsules in the evening is practical and doable for me. Anything less than the six capsules per day can significantly reduce the efficacy of a multivitamin formula, but once you get over six capsules, it becomes a pain to manage, difficult to swallow, and a hassle to travel with. --------------------------------------- My Multivitamin Criteria #4: Vegan & Vegetarian Friendly Capsules Most capsule-based multivitamins use capsules made from bovine gelatin - that's cow. And although I do not personally eat a plant-based diet, I have many clients who are either vegan or vegetarian. So I wanted the actual capsule material to be made from the best alternative to cattle-based bovine gelatin. The ideal answer would be a capsule made of cellulose, which is a natural, vegetable-based material, and is pure plant fiber (usually from pine trees). In other words, I know it's important to many of my clients and friends that no animals be harmed in the formulation of a good multivitamin. --------------------------------------- My Multivitamin Criteria #5: No Magnesium Stearate or Other Nasty Fillers Magnesium stearate is probably the most common flowing agent that you will find in the “Other Ingredients” section on the Supplement Facts box on any multivitamin label. Magnesium stearate is a lubricant that is used to manufacture nutritional supplements to help powders flow through manufacturing equipment faster. It allows more tablets to be manufactured per hour and is therefore a cost-savings choice for a manufacturer. However, studies have shown that magnesium stearate can reduce the absorption rate of some nutrients. It literally wraps around nutrients and keeps them from being absorbed in the digestive tract. So although magnesium stearate is not necessarily harmful, it can vastly reduce the effectiveness of the nutritional supplements that you are taking. Instead, I'd rather see a multivitamin use something like calcium laurate, which helps powders flow through the manufacturing equipment, but is a completely naturally-occurring material that does not inhibit absorption. Calcium laurate is not as effective of a flowing agent as magnesium stearate is, so products can’t be manufactured as fast, but it can certainly get the job done without the need to use magnesium stearate. When it is necessary to fill the capsule with something in addition to a nutrient or botanical, I'd rather see a multivitamin manufacturer use magnesium citrate, a the pure element reacted with pure citric acid; cellulose, a pure plant fiber, magnesium citrate-laurate, a pure element reacted with citric acid and lauric acid (a small fat molecule that does not inhibit absorption), silica, a pure element found in sand, and the amino acid leucine. Any of these fillers are completely natural and allow you to take a multivitamin without worrying about unnatural or harmful ingredients you might be ingesting. --------------------------------------- My Multivitamin Criteria #6: No GMO's For personal health and longevity reasons, and for consideration of the earth's environment, I avoid like the plague any product that contains Genetically Modified Organisms (known as GMOs). Unfortunately, most multivitamins contain GMO-laden ingredients - as well as corn, gluten, and some of the other nasty fillers mentioned earlier. This requires that any multivitamin I would ever use would need to closely analyzed by the manufacturer and guaranteed to be absolutely certain that none of ingredients are derived from GMO sources nor that they utilize GMOs in their production. --------------------------------------- My Multivitamin Criteria #7: Certified to Have No Banned Ingredients As I mentioned, it's very important to me that any supplement I take is not only CGMP certified (manufactured in a certified good manufacturing practices facility to ensure purity), but is also free of any ingredients that would get me banned from competing in triathlons, or taint my body with unnatural steroids or hormones. For years, I've been under the impression that something called an "NSF For Sport" certification was the strictest, third-party verification standard by which to judge whether a supplement contains these potentially illegal or banned ingredients. Turns out, I was wrong. There's something even more strict than NSF. And that is Australia’s Therapeutic Goods Administration (TGA), the pharmaceutical regulatory agency of Australia. Considered the toughest regulatory agency in the world, the TGA inspection and certification is conducted at a pharmaceutical level standard. Yes, that means the supplements certified by the TGA are verified to be as pure as an actual pharmaceutical drug. For me to hold a multivitamin to as strict a standard as a TGA certification would be tough. Only a handful of U.S. companies are able to achieve the status of holding a TGA certification and there is really only one dietary supplement company in the U.S. that ever has completed all three strict levels of a TGA certification, since the certification requires not only a review of Good Manufacturing Practices (GMPs) every two years, but also on-site inspection of every aspect of a multivitamin's quality control, laboratory procedures, pharmaceutical good manufacturing practices, and standard operating procedures. But that's not all. I'd also prefer for a multivitamin to be certified by Informed Choice. Informed Choice requires five samples from three different production runs of product to be tested, which would further guarantee, because of extremely rigid raw material and finished product testing and production procedures, absolutely no ingredient banned by any amateur or professional organization. In other words, I want rock-solid, bulletproof confidence that I can hand any multivitamin that I endorse or personally use to the best athlete on the face of the planet with zero reservations about whether or not the multivitamin is pure or might get them banned. --------------------------------------- Notoriously Neglected Ingredients That A Multivitamin Must Have I also know that there are notoriously neglected ingredients in most multivitamin complexes that make them barely scratch the surface of your true nutrient needs, especially if you're a hard charging athlete, exercise enthusiast, an aging person, a busy professional living a CEO-esque lifestyle, or anyone else with advanced nutrient needs. Take Vitamin D, for example. The 200-400IU of Vitamin D found in most popular multivitamins is basically enough to keep a child from getting rickets. This is not even close enough to support optimal hormonal balance and physical or mental performance. Problem is, the multivitamins that actually do indeed have the 2000+IU per day of Vitamin D that I like to see do not include optimal amounts of Vitamin A or Vitamin K to allow for ideal absorption and utilization of Vitamin D, or reduced risk of Vitamin D toxicity. Or take Vitamin K - a crucial fat soluble vitamin usually found in complete zero amounts in a multivitamin. The body actually needs nearly 400mcg per day of Vitamin K to support bone health, calcium and magnesium balance, and vitamin D absorption, among a host of other metabolic and immune-boosting functions. Yet strangely enough, it's not included at all in most multivitamins. So what else is neglected in most multivitamins that I searched for on my quest to find the greatest multivitamin on the face of the planet? 1. Vitamin B6, B12 and B2 In The Correct Form The bioactive form of Vitamin B6 is Pyridoxal 5’-Phosphate. But the most common form found in most multivitamins is Pyridoxine HCl, which is not the active form of Vitamin B6. So the Pyridoxal 5’-Phosphate is a must-have in my opinion. The most common form of Vitamin B12 used in nutritional supplements and processed foods is cyanocobalamin. Cyanocobalamin, however, is absorbed at a much lower rate than methylcobalamin, which is the form of vitamin B12 that a multivitamin should actually have. Riboflavin 5’-Phosphate is the bioactive form of vitamin B2, but unfortunately Riboflavin HCl is the form commonly used in multivitamins, and that form needs to be converted in an extra step in the liver to the active form. Why do most multivitamins cut corners on these forms of Vitamin B? They're either uninformed, or going for the cheap alternative. 2. Folate In The Correct Form Folic acid is the most common form of folate used in nutritional supplements and fortified foods. However, a significant percentage of the population has trouble converting folic acid to folate, which is the natural form found in foods and in the body. A good multivitamin should use a form of folate called l-5-methyltetrahydrofolate to ensure folate is absorbed. It does cost considerably more than folic acid, but again, I only want a multivitamin with ingredients that work - not that just look pretty on a label and then give me expensive pee. 3. Minerals That Are Actually Absorbable Minerals by themselves are poorly absorbed, but when bound to an amino acid (also called a "mineral chelate") they are absorbed much better. Technically the most absorbable form of a mineral is the bisglycinate chelate form, and to avoid the minerals in a multivitamin simply creating expensive pee, I would specifically like to see calcium, magnesium, copper, zinc, manganese, chromium, and molybdenum minerals in a chelated form. 4. Curcumin That Is Actually Absorbable Curcumin is one of my favorite supplements, and has received a lot of attention recently as a potent natural anti-inflammatory nutrient that can control brain and muscle inflammation, positively affect muscle health, joint structure and function, and antioxidant activity in the gut, liver, and heart. But the trouble with most curcumin ingredients is that they are extremely poorly absorbed. However, there is one form of curcumin called the phytosome form which was developed by the researchers in Milan, Italy, and has been shown to be absorbed 29 times better than a standard curcumin extract. This is the only form of curcumin that I'd like to see in a multivitamin. 5. Green Tea Extract That Is Actually Absorbable Green tea is another powerful antioxidant that has been shown to have a mild effect on raising metabolic rate and improving fat oxidation. But like standard curcumin extracts, standard green tea extracts are not well absorbed either. But the same researchers who discovered the curcumin phyotosome have also developed green tea phytosome, which provides a much better absorbed form of green tea extract, and would also ideally be in a multivitamin. 6. Adaptogens Stress-fighting adaptogens are another notoriously missing component of most multivitamins, but in our hectic era are a must-have for sleep, de-stressing and hormone balance. One of the most potent adaptogenic compounds that I discovered in my multivitamin research to both lower cortisol levels and raise testosterone levels is called "Relora", and is a blend of magnolia officinalis and phellodendron amurense. If added to a multivitamin, it could also help curb cravings and stress-related eating, and help calm the mind and support rest and relaxation prior to bedtime. As a high intensity guy, this addition would be a very important to me as a necessary multivitamin ingredient. Remember - this all comes down to simplicity. This means that if you or I can't get our hands on a capsule-based, pure and safe, minimalist and simple multivitamin that provides everything I just described above... ...then it means we are instead forced to buy 6, 8, 10 or 12 different supplements to supply our daily nutrient needs and fill in all the "holes" the multivitamin isn't giving. --------------------------------------- Where A Tiny Town In North Idaho Fits In So that's what I've been doing over the past two years. I have been taking a deep, deep dive into educating myself on the wide world of nutrients, supplements, ingredients, fillers, formulations, encapsulation technology and every aspect that goes into creating a multivitamin. And I have literally been pulling my hair out and hitting dead-end after dead-end as I have been trying to hunt down something that meets my strict quality criteria for a multivitamin. Of course, I have also found it incredibly frustrating when my clients, or readers, or podcast listeners or me have to simply keep on taking a crazy number of supplements or buying an unbelievable number of bottles and spending inordinate amounts of money when it's so straightforward to see what a multivitamin that meets all our daily nutrients needs should really have (and not have!) in it. I've even gone so far as to consider simply hunting down the raw ingredients and formulating my own multivitamin. But frankly, I'm an author, an athlete and a teacher - and don't fancy myself as a supplement manufacturer. I just don't have a desire to get into that business. I simply want a multivitamin that I can, with a clear conscience, recommend to you and also take myself. Is that asking too much? And then it happened. During one of the dozens and dozens of phone calls I made and countless hours attempting to hunt down the perfect multivitamin formulation, a raw ingredient manufacturer informed me of the existence of a new multivitamin formulation created by a company called "Thorne Research", in Dover, Idaho. That's right - just 60 miles from the front door of my house, in the tiny town of Dover, Idaho, are the Thorne Research facilities (pictured below). After months and months of searching, it turns out the solution could possibly be right in my own backyard. For over 30 years, Thorne Research has set the standard for exceptional formulations, quality and purity in the dietary supplement industry - and is considered the most respected nutritional supplements brand in the healthcare practitioner space. That's right - this is is the stuff physicians use with their patients. I'd actually heard of Thorne before, but always knew it was just for the fancy doctors and clinicians so I couldn't really access it or use it. So until recently, none of the coveted Thorne Research products were available for me or any other personal trainer to actually recommend to our clients, readers or listeners. Instead, these high-quality, private formulations were only available for physicians or dietitians to use with their patients, and only for certified health care providers to use with their clients. But then, in January 2014, Thorne Research created a new branch called "". ThorneFX was specifically designed to address the growing demand of fitness experts like me who are seeking a solution for our clients, but don't have the privileged access to the fancy formulations that normally only a doctor or clinician would be able to prescribe or order. --------------------------------------- The Greatest Multivitamin On The Face Of The Planet So I thought it was pretty interesting (and exciting) that I now had access to some of the highest quality formulations on the face of the planet, produced in a facility just an hour's drive from my house. But it turned out there was even more good news. I also discovered that when Thorne Research created the ThorneFX branch, they had not just created a multivitamin formulation but they had specifically left no stone unturned to launch the most powerful, absorbable multivitamin on the face of the planet. This was a multivitamin that possibly met all of my strict criteria, but that previously had never been available to the general population unless their doctor or dietitian knew about it. It's called the , and I immediately got my hands on it. Upon closer inspection and using it for 30 days, I instantly knew that this is not your typical multivitamin. Formulated with superior forms of vitamins and minerals, is actually two unique formulations - an AM formula is designed to support daytime energy, and a PM formula designed to support better rest and recovery in the evening. Some of the key differentiators in include: -Vitamins K1 and K2 (most use only K1 or no K2 at all). -Natural folate (5-MTHF) instead of folic acid. -Methylcobalamin instead of cyancobalamin. -Chelated minerals to ensure optimal absorption. -Higher dose of magnesium and calcium than typically found in a multivitamin. -Curcumin Phytosome (curcumin complexed with phosphatidylcholine for superior bioavailability) -Green Tea Phytosome (green tea complexed with phosphatidylcholine for superior bioavailability) -Relora, a blend of adaptogenic plant extracts to aid in relaxation, curb late-night cravings, and balance cortisol, testosterone and DHEA levels. The (pictured below) literally satisfies every criteria I require in a multivitamin: all the notoriously neglected ingredients, a TGA and Informed Choice certification, no GMO's, no nasty fillers and ingredients, a capsule instead of a tablet...everything. I couldn't believe when I first found out about the new formulation. Three easy capsules in the morning, three in the evening. Boom. Simple science. My quest was over. Mission accomplished. See for yourself - below are the label and full ingredients. Morning dose - three capsules contain: Vitamin A (3,000 IU as Beta Carotene and 2,000 IU as Palmitate) 5,000 IU Vitamin C (as Ascorbic Acid) 250 mg Vitamin D (as Vitamin D3) 2,000 IU Vitamin E (as d-Alpha Tocopheryl Acid Succinate) 20 IU Vitamin K (100 mcg as Vitamin K1 and 100 mcg as Vitamin K2) 200 mcg Thiamin (as Thiamin HCl) 50 mg Riboflavin (as Riboflavin 5’-Phosphate Sodium) 12 mg Niacin (as Niacinamide) 60 mg Vitamin B6 (as Pyridoxal 5’-Phosphate) 20 mg Folate (as L-5-Methyltetrahydrofolate from L-5-Methyltetrahydrofolic Acid, Glucosamine Salt)* 600 mcg Vitamin B12 (as Methylcobalamin) 600 mcg Biotin 500 mcg Pantothenic Acid (as Calcium Pantothenate) 45 mg Calcium (as Calcium Bisglycinate Chelate) 38 mg Iodine (as Potassium Iodide) 75 mcg Magnesium (as Magnesium Bisglycinate Chelate) 63 mg Zinc (as Zinc Bisglycinate Chelate) 10 mg Selenium (as L-Selenomethionine) 100 mcg Copper (as Copper (II) Bisglycinate Chelate) 750 mcg Manganese (as Manganese (II) Bisglycinate Chelate) 1.5 mg Chromium (as Chromium Nicotinate Glycinate Chelate) 200 mcg Molybdenum (as Molybdenum (II) Glycinate Chelate) 50 mcg Curcumin Phytosome** (Curcuma longa extract (root) / Phosphatidylcholine complex) 200 mg Green Tea Phytosome** (Camellia sinensis extract (leaf) decaffeinated / Phosphatidylcholine complex) 75 mg Choline Citrate 50 mg d-Gamma Tocopherol (from Mixed Tocopherols) 24 mg Boron (as Bororganic™ Glycine) 1 mg Lutein (from Aztec Marigold extract (flower) (Tagetes erecta)) 300 mcg Evening dose, three capsules contain: Vitamin A (600 IU as Beta Carotene and 400 IU as Palmitate) 1,000 IU Vitamin C (as Ascorbic Acid) 150 mg Vitamin D (as Vitamin D3) 500 IU Vitamin E (as d-Alpha Tocopheryl Acid Succinate) 20 IU Vitamin K (100 mcg as Vitamin K1 and 100 mcg as Vitamin K2) 200 mcg Folate (as L-5-Methyltetrahydrofolate from L-5-Methyltetrahydrofolic Acid, Glucosamine Salt)* 400 mcg Biotin 500 mcg Calcium (as Calcium Bisglycinate Chelate) 50 mg Iodine (as Potassium Iodide) 75 mcg Magnesium (as Magnesium Bisglycinate Chelate) 107 mg Zinc (as Zinc Bisglycinate Chelate) 20 mg Selenium (as L-Selenomethionine) 100 mcg Copper (as Copper (II) Bisglycinate Chelate) 750 mcg Manganese (as Manganese (II) Bisglycinate Chelate) 1.5 mg Chromium (as Chromium Nicotinate Glycinate Chelate) 200 mcg Molybdenum (as Molybdenum (II) Glycinate Chelate) 50 mcg Proprietary Blend** 200 mg Magnolia officinalis (bark) extract Phellodendron amurense (bark) extract Choline Citrate 100 mg d-Gamma Tocopherol (from Mixed Tocopherols) 24 mg Boron (as Bororganic™ Glycine) 2 mg Lutein (from Aztec Marigold extract (flower) (Tagetes erecta)) 600 mcg The science is obviously there - every nutrient your body needs. But the simplicity is so amazingly refreshing - especially when I pack to travel and literally have one morning bottle and one evening bottle to toss into my suitcase - and of course, a heck of a lot of extra space in my refrigerator. Better yet, if you need some icing on the cake - such as anything extra added in like some , , , a , a , a , - , with the same huge, unparalleled quality that I've never been able to find before. It's like a one-stop shop for everything I have ever looked for or wanted in a nutrition supplements line. --------------------------------------- Summary So that's it: pure and simple. I have never before been able to recommend a complete multivitamin that satisfies all my criteria. And I have always had a refrigerator full of a bunch of bottles. But now the problem is solved. And yes, this means that the ThorneFX is now my recommended, go-to source for the ultimate combination of science and simplicity. Furthermore, the is now something I back, endorse, vouch for and am now using in my own personal daily routine with great success. And the best thing yet? An entire month of the is just 34 bucks. You heard me right: 34 dollars. So rather than forking over hundred of dollars for bottle after bottle of fancy supplements, powders, tablets, pills, and oils, you get one multivitamin that hits all your needs for a mere fraction of the price. Go ahead - spend that extra money you were blowing on 10 different supplements on a fancy steak dinner or chock it away for your kid's college or your next car. Of course, if you're familiar with me, my podcast, or articles I've written in the past, then you may be wondering... ...what about the other supplements I've previously recommended? What about ? What about ? What about my ? The good news is that all of these are still available and still consist of specific ingredients that I may use with certain clients or recommend in certain situations when the need arises. After all, everything I have recommended in the past is all excellent, extremely high-quality stuff, and if you're happy with what you're currently using...then that's fine! But for science, simplicity, peace of mind, and some extra room in my refrigerator and my suitcase, I am moving in the direction of shifting nearly my entire personal supplement protocol to - and specifically what I have discovered to be the greatest multivitamin on the face of the planet: the . , feel free to ask me about how to use or implement into your current protocol, and leave any questions about any substitutions or replacements for other supplements you're currently taking. I promise to personally respond and walk you through everything. Enjoy the new simplicity in your life!
Cortisol is a stress hormone, which is produced under stressful conditions, one of them being heavy training. A CrossFit viewer asked how they could nutritionally support their hormones under heavy training loads. The first hormone that comes into mind is modulating the hormone Cortisol. Cortisol is a stress hormone that is produced in the adrenal glands and is designed to be released in short bursts. Cortisol is an anti-inflammatory hormone and it also helps us mobilize energy. This is used in times of fight or flight when these short energy bursts are necessary to run or fight. In modern times, we disrupt our Cortisol production by poor sleep, overworking, heavy training and poor nutrition. Cortisol production has been directly correlated with training load. It is important to pay attention to your body and note that if you are overstressing your body and producing large amounts of cortisol, your body will start to break down. It will break down your muscle because it is a catabolic hormone. Your body makes cortisol preferentially over other hormones. When there is a large amount of cortisol being made, you will see other hormones decline like testosterone and DHEA. These hormones are responsible for building your body up. This is why it is important to take a rest after heavy training. Make sure you get 20-25 grams of protein within ten minutes after heavy training. This will blunt the rise in cortisol. Triton Nutrition makes Vital Vegan Protein and ProFit Whey and these can be used in combination with Vital Greens or Vital Reds. Please check out our site where delicious recipes are listed using these proteins. Another great product is Triton Nutrition's Immuno Growth Factor with PRP, which is a pure, cold-pressed colostrum product made from grass-fed, organically grown cows. This cold-pressed technique maintains Immunoglobulins (IgGs) and proline-rich polypeptides (PRPs). PRPs help balance inflammation and IgGs are needed because the immune system can be weakened during heavy workouts. This is a very efficient form of protein and can be combined with the other powdered forms of protein. There are two products by Triton Nutrition that also help with Cortisol regulation: Relora-plex and Adrenal Shield. Adrenal Shield contains extract from three herbs: Chordyceps sinensis, Ginseng root and Rhodiola rosea root, which helps balance the cortisol and the other hormones responsible for balancing energy, like DHEA. This will help with afternoon fatigue. Relora-plex is a specific blend of herbs designed to reduce cortisol. Relora-plex is specifically helpful when someone would like to start on hormone replacement therapy but his or her cortisol is elevated. You don't want to start therapy before getting the elevated cortisol under control first because the other hormones supplied during therapy won't function properly. Relora-plex allows the person to feel less anxiety and more smooth energy throughout the day. At nighttime they fall asleep faster and they stay asleep. It is important to sleep well because that is when 80% of growth hormone is produced. by Robert Seik, PharmD
© 2020-2025 Ivy Podcast Discovery LLC
