Podcasts about QK

In this special episode, Let's Be Having You is live from Soccer Aid with a very special guest - former Manchester City and England goalkeeper Joe Hart.  Joe reflects on his early days at Shrewsbury Town, sharing what it was like to grow up as a promising young goalkeeper, which included trips to the local Liquid nightclub! He also opens up about his move to Manchester City - and, of course, relives the unforgettable moment of winning the Premier League title in 2012. We'll see you soon for the next series, but in the meantime, if you want a regular fix of Let's Be Having You head over to the LBHY fan club at letsbehavingyou.com If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x *Soccer Aid is live on @itv 6pm on Sunday. To donate head to: socceraid.org.uk Learn more about your ad choices. Visit podcastchoices.com/adchoices

In this special episode, Let's Be Having You is live from Soccer Aid with a very special guest - former Arsenal midfielder Jack Wilshere. Jack opens up about his early football journey, from idolising Paolo Di Canio and David Beckham as a youngster, to being scouted by a referee and signing with Arsenal at just nine years old - for a mere £1,500! He shares memories of his iconic performance in the victory over Barcelona, what it was like living with Wojciech Szczesny, and how he nearly botched his Goal of the Season strike against Norwich City. And yes - Jack also dishes on grabbing the mic to mock arch-rivals Tottenham during Arsenal's FA Cup victory parade. We'll see you soon for the next series, but in the meantime, if you want a regular fix of Let's Be Having You head over to the LBHY fan club at letsbehavingyou.com If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x *Soccer Aid is live on @itv 6pm on Sunday. To donate head to: socceraid.org.uk Learn more about your ad choices. Visit podcastchoices.com/adchoices

It's time for the Let's Be Having You End of Series Quiz! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! There are a few familiar favourites and some new additions as Chris and Alex are put to the test – but who will come out on top? Will Alex continue the streak and go 2-0? We'll see you soon for the next series, but in the meantime, if you want a regular fix of Let's Be Having You head over to the LBHY fan club at letsbehavingyou.com If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

It's time for the inaugural Let's Be Having You End of Series Quiz! There are a few familiar favourites and some new additions as Chris and Alex are put to the test – but who will come out on top? Will Alex continue the streak and go 2-0? We'll see you soon for the next series, but in the meantime, if you want a regular fix of Let's Be Having You head over to the LBHY fan club at letsbehavingyou.com If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by simply one of the biggest football cult heroes of the 00's (… and whisper it very quietly, he's not even a footballer)…  Harchester United legend Karl Fletcher AKA Terry Kiely!  Part 1 of the podcast came out last week, so if you missed it... make sure to go back and listen! Terry takes us deep inside The Dragons' Lair, spilling all the behind-the-scenes secrets - including the surprising name originally lined up to play Fletch. Trust us, it's sensational stuff. From working alongside icons like Linsey Dawn McKenzie, Katie Price and Big Ron Atkinson to reliving his most memorable moments in the iconic purple and white of Harchester, Terry doesn't hold back. And what about that unforgettable half-time team talk from Mike Bassett (played by Ricky Tomlinson)? Did Terry manage to keep a straight face? He gives us the full story on being part of another legendary 00's cult classic. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by simply one of the biggest football cult heroes of the 00's (… and whisper it very quietly, he's not even a footballer)… Harchester United legend Karl Fletcher AKA Terry Kiely! Terry takes us deep inside The Dragons' Lair, spilling all the behind-the-scenes secrets - including the surprising name originally lined up to play Fletch. Trust us, it's sensational stuff. From working alongside icons like Linsey Dawn McKenzie, Katie Price and Big Ron Atkinson to reliving his most memorable moments in the iconic purple and white of Harchester, Terry doesn't hold back. And what about that unforgettable half-time team talk from Mike Bassett (played by Ricky Tomlinson)? Did Terry manage to keep a straight face? He gives us the full story on being part of another legendary 00s cult classic. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by Oh What A Time's Tom Craine to review one of the all time great 00's football documentaries - Trouble At The Top: Luton Kicks Off.  Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! We look back at John Gurney's infamous 55-day spell as Luton chairman, including one of the craziest managerial appointment processes of the decade: MANAGER IDOL!  And that's only the beginning of the chaos… If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by Oh What A Time's Tom Craine to review one of the all time great 00's football documentaries - Trouble At The Top: Luton Kicks Off.  We look back at John Gurney's infamous 55-day spell as Luton chairman, including one of the craziest managerial appointment processes of the decade: MANAGER IDOL!  And that's only the beginning of the chaos… If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

So here we go! It's episode one of the Big Ron Manager watchalong with comedian Ivo Graham.  Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! Yes, you heard right, Ivo is moving from Steve Bruce's literary masterclass to Big Ron's attempt at a media relaunch at League Two Peterborough United. We get our first sighting of young managerial buck Steve Bleasdale and it doesn't disappoint… we're talking bust-ups, foul mouthed rants and of course a visit to the local factory.  Plus, we dig into your correspondence, including news from Georgi Kinklandze's vineyard and an update on Dream Team's Sean Hocknell.  If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY. So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

So here we go! It's episode one of the Big Ron Manager watchalong with comedian Ivo Graham.  Yes, you heard right, Ivo is moving from Steve Bruce's literary masterclass to Big Ron's attempt at a media relaunch at League Two Peterborough United. We get our first sighting of young managerial buck Steve Bleasdale and it doesn't disappoint… we're talking bust-ups, foul mouthed rants and of course a visit to the local factory.  Plus, we dig into your correspondence, including news from Georgi Kinklandze's vineyard and an update on Dream Team's Sean Hocknell.  If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY. So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we're joined by former Leeds, Chelsea and Middlesbrough striker Jimmy Floyd Hasselbaink. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! He explains the truth behind his name ‘Jimmy', starting as a goalkeeper (yes, really), and we even get to witness a Premier League Golden Boot in the wild!  Jimmy also chats about giving Arsenal fans nightmares and takes us back to that wild European rollercoaster with Middlesbrough. And was Premier League ref Uriah Rennie actually name-dropped in Ja Rule's Smokin' and Ridin'? We dig into your correspondence. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Tickets for Legends Ibiza are available to buy now, head to https://legendsibiza.uk for all the information you'll need ahead of the sporting festival of the summer! Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we're joined by former Leeds, Chelsea and Middlesbrough striker Jimmy Floyd Hasselbaink. He explains the truth behind his name ‘Jimmy', starting as a goalkeeper (yes, really), and we even get to witness a Premier League Golden Boot in the wild!  Jimmy also chats about giving Arsenal fans nightmares and takes us back to that wild European rollercoaster with Middlesbrough. And was Premier League ref Uriah Rennie actually name-dropped in Ja Rule's Smokin' and Ridin'? We dig into your correspondence! If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Tickets for Legends Ibiza are available to buy now, head to https://legendsibiza.uk for all the information you'll need ahead of the sporting festival of the summer! Learn more about your ad choices. Visit podcastchoices.com/adchoices

The Adventures of Buckaroo Banzai Across the 8th Dimension is beloved by those bold enough to embrace a fantastical new reality, and abhorred by those aghast at this bizarre spectacle of unknown providence. It's a film as divisive as it is confusing as it is gonzo, but we've assembled a massive crew of colourful weirdos to help make sense of it all; just like Buckaroo would have wanted! I Have Some Notes welcomes David Rae and Laena Anderson from Quantum Kickflip. QK is an actual-play podcast about teen hoverboarders traveling to strange new dimensions to explore, get into trouble, and look cool doing it. Dave and Laena have two seasons worth of multiverse adventure storytelling experience, so they might be the only ones clever or crazy enough to handle the wild trip that is Buckaroo Banzai. Listen to Quantum Kickflip, including the upcoming Season 3 dropping June 4, at www.quantumkickflip.com. Follow them on Bluesky at quantumkickflip.bsky.social. The I Have Some Notes Discord Server is a great place to keep the conversation going. Let us know what you thought of our pitches and share your own thoughts on the next film we cover!

For this episode, we are joined by a dream guest - it's former Coventry City, Southampton and Celtic manager Gordon Strachan!  Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! Gordon shares all on his time at Coventry City: from working with Big Ron Atkinson to having Noel Whelan as a lodger, and even trying to bring a teenage Andrea Pirlo to Highfield Road! He talks us through his spell at Southampton and gives us his thoughts on Rupert Lowe's bold decision to bring Sir Clive Woodward into football. And of course, we get into his period at Celtic – featuring all the chaos and comedy of trying (and failing) to make sense of Thomas Gravesen! Plus, our inbox is BULGING! We dig into your excellent correspondence… but seriously, was the kick-off of Liverpool's UEFA Cup semi-final against Barcelona really delayed because of Eastenders? If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by a dream guest - it's former Coventry City, Southampton and Celtic manager Gordon Strachan!  Gordon shares all on his time at Coventry City: from working with Big Ron Atkinson to having Noel Whelan as a lodger, and even trying to bring a teenage Andrea Pirlo to Highfield Road! He talks us through his spell at Southampton and gives us his thoughts on Rupert Lowe's bold decision to bring Sir Clive Woodward into football. And of course, we get into his period at Celtic – featuring all the chaos and comedy of trying (and failing) to make sense of Thomas Gravesen! Plus, our inbox is BULGING! We dig into your excellent correspondence… but seriously, was the kick-off of Liverpool's UEFA Cup semi-final against Barcelona really delayed because of Eastenders? If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

This week, we're joined by a true legend of the commentary world – Ian Darke! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! He shares why it's best not to strike up a conversation with Des Lynam during the first 40 seconds of a flight, and what it's really like handling fiery managers like Sir Alex Ferguson and Neil Warnock. Ian also reflects on his time as a boxing commentator, including the tense moments where he had to navigate riots and even gunfire. Plus, there's time to chat about his lifelong support for Portsmouth FC and the unforgettable highs of the club's remarkable run through the 00's. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

This week, we're joined by a true legend of the commentary world – Ian Darke! He shares why it's best not to strike up a conversation with Des Lynam during the first 40 seconds of a flight, and what it's really like handling fiery managers like Sir Alex Ferguson and Neil Warnock. Ian also reflects on his time as a boxing commentator, including the tense moments where he had to navigate riots and even gunfire. Plus, there's time to chat about his lifelong support for Portsmouth FC and the unforgettable highs of the club's remarkable run through the 00's. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by a no nonsense, tough tackling defender from the 00's - it's former England and Leeds United right-back Danny Mills!  Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! We finally get to the bottom of his rivalry with Arsenal fans - so does Danny actually hate them?  There's more revelations from Peter Risdale's office... Plus, Danny reveals what the England players were up to before that legendary 2002 World Cup showdown with Argentina, and why Stuart Pearce thought it was a great idea to chuck David James up front for Manchester City! Alex discusses his near death experience and we dig into some Dream Team correspondence.  If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by a no nonsense, tough tackling defender from the 00's - it's former England and Leeds United right-back Danny Mills!  We finally get to the bottom of his rivalry with Arsenal fans - so does Danny actually hate them?  There's more revelations from Peter Risdale's office... Plus, Danny reveals what the England players were up to before that legendary 2002 World Cup showdown with Argentina, and why Stuart Pearce thought it was a great idea to chuck David James up front for Manchester City! Alex discusses his near death experience and we dig into some Dream Team correspondence.  If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

This week, we're joined by a true cult hero of the 00's and the ultimate showman - Swansea City legend, Lee Trundle! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! Lee shares his journey as a late bloomer in professional football and why, at 48, he's still out there tearing it up on the pitch. He takes us through his unforgettable goals, jaw-dropping skills, and the moment Soccer AM catapulted him into national fame. We also get the inside scoop on the Premier League move that almost happened, the surprising story behind his face appearing on a thong, and the time he landed an ASBO! Plus, we dive into your messages, including Guus Hiddink's bizarre eating habit and Ray Wilkins' unique way of lifting Chelsea's spirits after their dramatic 2009 Champions League exit to Barcelona. Got something to contribute? Drop us an email at hello@letsbehavingyou.com. And if you're a member of the QK Fan Club—good news! As we transition to the LBHY Fan Club, you'll still have access to all the old QK subscriber episodes, plus exclusive monthly specials from Let's Be Having You (which just might feature Big Ron Manager…).   So, good night Richard Wright, Chris, Alex, and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

This week, we're joined by a true cult hero of the 00's and the ultimate showman - Swansea City legend, Lee Trundle! Lee shares his journey as a late bloomer in professional football and why, at 48, he's still out there tearing it up on the pitch. He takes us through his unforgettable goals, jaw-dropping skills, and the moment Soccer AM catapulted him into national fame. We also get the inside scoop on the Premier League move that almost happened, the surprising story behind his face appearing on a thong, and the time he landed an ASBO! Plus, we dive into your messages, including Guus Hiddink's bizarre eating habit and Ray Wilkins' unique way of lifting Chelsea's spirits after their dramatic 2009 Champions League exit to Barcelona. Got something to contribute? Drop us an email at hello@letsbehavingyou.com. And if you're a member of the QK Fan Club—good news! As we transition to the LBHY Fan Club, you'll still have access to all the old QK subscriber episodes, plus exclusive monthly specials from Let's Be Having You (which just might feature Big Ron Manager…).   So, good night Richard Wright, Chris, Alex, and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by a tireless, box-to-box midfielder – former Wimbledon, West Ham United and Aston Villa star Nigel Reo-Coker. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! Did the June Bug drink really propel West Ham to the 2006 FA Cup final? Nigel reveals all on infamous trips to the big 00's hangout…TGI Friday's!  We also revisit his early career at Wimbledon, getting his firsthand take on the club's controversial move to Milton Keynes. Plus, Nigel explains how he inadvertently told then-England manager Sven Goran Eriksson to “f*** off” ahead of the 2006 World Cup.  And, of course, we dive into your correspondence, including a singing Premier League manager from the 00's who could have easily headlined a music festival! Got something to contribute? Drop us an email at hello@letsbehavingyou.com. And if you're a member of the QK Fan Club—good news! As we transition to the LBHY Fan Club, you'll still have access to all the old QK subscriber episodes, plus exclusive monthly specials from Let's Be Having You (which just might feature Big Ron Manager…).   So, good night Richard Wright, Chris, Alex, and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by a tireless, box-to-box midfielder – former Wimbledon, West Ham United and Aston Villa star Nigel Reo-Coker. Did the June Bug drink really propel West Ham to the 2006 FA Cup final? Nigel reveals all on infamous trips to the big 00's hangout…TGI Friday's!  We also revisit his early career at Wimbledon, getting his firsthand take on the club's controversial move to Milton Keynes. Plus, Nigel explains how he inadvertently told then-England manager Sven Goran Eriksson to “f*** off” ahead of the 2006 World Cup.  And, of course, we dive into your correspondence, including a singing Premier League manager from the 00's who could have easily headlined a music festival! Got something to contribute? Drop us an email at hello@letsbehavingyou.com. And if you're a member of the QK Fan Club—good news! As we transition to the LBHY Fan Club, you'll still have access to all the old QK subscriber episodes, plus exclusive monthly specials from Let's Be Having You (which just might feature Big Ron Manager…).   So, good night Richard Wright, Chris, Alex, and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

We're back with series two of Let's Be Having You — and this time, we're joined by a true cult hero of sports broadcasting! A man once dubbed a “f****** legend” by Noel Gallagher… it's none other than former Match of the Day, Football Focus, and Final Score host, Ray Stubbs! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! With a career spanning six decades, Ray takes us through some wild moments—from bizarre sponsorship deals with his boyhood club Tranmere Rovers to convincing Diego Maradona and Gazza to take part in ‘Mystery Sportsman' on A Question of Sport. Plus, he shares the chaos of covering Liverpool's UEFA Cup opponents after one too many lunchtime shots. We also dig into your correspondence, including a look at former Premier League referee Uriah Rennie's unexpected role as the ‘Ice Judge' on ITV's 00s game show Freeze Out! Got something to contribute? Drop us an email at hello@letsbehavingyou.com. And if you're a member of the QK Fan Club—good news! As we transition to the LBHY Fan Club, you'll still have access to all the old QK subscriber episodes, plus exclusive monthly specials from Let's Be Having You (which just might feature Big Ron Manager…). So, good night Richard Wright, Chris, Alex, and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

We're back with series two of Let's Be Having You — and this time, we're joined by a true cult hero of sports broadcasting! A man once dubbed a “f****** legend” by Noel Gallagher… it's none other than former Match of the Day, Football Focus, and Final Score host, Ray Stubbs! With a career spanning six decades, Ray takes us through some wild moments—from bizarre sponsorship deals with his boyhood club Tranmere Rovers to convincing Diego Maradona and Gazza to take part in ‘Mystery Sportsman' on A Question of Sport. Plus, he shares the chaos of covering Liverpool's UEFA Cup opponents after one too many lunchtime shots. We also dig into your correspondence, including a look at former Premier League referee Uriah Rennie's unexpected role as the ‘Ice Judge' on ITV's 00s game show Freeze Out! Got something to contribute? Drop us an email at hello@letsbehavingyou.com. And if you're a member of the QK Fan Club—good news! As we transition to the LBHY Fan Club, you'll still have access to all the old QK subscriber episodes, plus exclusive monthly specials from Let's Be Having You (which just might feature Big Ron Manager…). So, good night Richard Wright, Chris, Alex, and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

It's finally time for the inaugural Let's Be Having You End of Series Quiz! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! There are a few familiar favourites from QuicklyKevin and some new additions as Chris and Alex are put to the test – but who will come out on top? BIG NEWS – We've commissioned ourselves for a second series! We'll see you soon for the next series, but in the meantime, if you want a regular fix of Let's Be Having You head over to the LBHY fan club at letsbehavingyou.com If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

It's finally time for the inaugural Let's Be Having You End of Series Quiz! There are a few familiar favourites from QuicklyKevin and some new additions as Chris and Alex are put to the test – but who will come out on top? BIG NEWS – We've commissioned ourselves for a second series! We'll see you soon for the next series, but in the meantime, if you want a regular fix of Let's Be Having You head over to the LBHY fan club at letsbehavingyou.com If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by comedian and one part of the Oh What a Time podcast, Tom Craine, to watch back the first episode of ITV's The Premiership. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! We review ITV's brave new world of 7pm starts, scoreboards and fact graphics flanked by Andy Townsend's Tactics Truck. We also get Sunderland manager Peter Reid in what looks like a hostage video, Fabrizio Ravanelli cooking pasta with Derby's Spaghetti Bob and Leeds United's Alan Smith – the first winner of Ally McCoist's Golden Goals - scoring a goal which is probably better than Dennis Bergkamp's v Newcastle. Plus, we dive into your correspondence which includes the TRUTH behind why Wolves were sponsored by Doritos and the first Let's Be Having You Verified confirms an interesting story from Alex's past. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by comedian and one part of the Oh What a Time podcast, Tom Craine, to watch back the first episode of ITV's The Premiership. We review ITV's brave new world of 7pm starts, scoreboards and fact graphics flanked by Andy Townsend's Tactics Truck. We also get Sunderland manager Peter Reid in what looks like a hostage video, Fabrizio Ravanelli cooking pasta with Derby's Spaghetti Bob and Leeds United's Alan Smith – the first winner of Ally McCoist's Golden Goals - scoring a goal which is probably better than Dennis Bergkamp's v Newcastle. Plus, we dive into your correspondence which includes the TRUTH behind why Wolves were sponsored by Doritos and the first Let's Be Having You Verified confirms an interesting story from Alex's past. If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

What a way to start the Let's Be Having You Film Club as we're joined by comedian Ben Partridge to review this not so classic from 2005, Green Street! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! None of us will be standing our ground and fighting in defence of this film… We tackle the big questions: Elijah Wood's career choice, Charlie Hunnam's cockney accent and a batshit script! Plus, we dive into your correspondence which includes 00s footballer boyband, The Players! If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

What a way to start the Let's Be Having You Film Club as we're joined by comedian Ben Partridge to review this not so classic from 2005, Green Street! None of us will be standing our ground and fighting in defence of this film… We tackle the big questions: Elijah Wood's career choice, Charlie Hunnam's cockney accent and a batshit script! Plus, we dive into your correspondence which includes 00s footballer boyband, The Players! If you've got anything at all to contribute, you can email us at: hello@letsbehavingyou.com If you're a member of the QK Fan club - good news! As the QK fan club transitions to LBHY fan club you'll still get access to all the old QK subscriber episodes and you'll be getting monthly special episodes from LBHY as well (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

In this episode we'll be firing up the DeLorean and travelling back to relive England qualifying for Euro 2000 via those infamous playoffs with Scotland, we'll see Phil Neville put in THAT tackle against Romania, England in the rain at the last game at the old Wembley, FIVE-ONE, Beckham's garbage yet match-winning penalty against Argentina and much, much more. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! Now would you go to see Don Hutchison perform his stand-up show ‘Behind The Frontman' at the Edinburgh Fringe? If yes, please tell us why you think that'd be a good idea: hello@letsbehavingyou.com Plus, we dive into your correspondence as we get DEFINITIVE PROOF on Seth Johnson's contract negotiations at Leeds United. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night, Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

In this episode we'll be firing up the DeLorean with Matt Forde and travelling back to relive England qualifying for Euro 2000 via those infamous playoffs with Scotland, we'll see Phil Neville put in THAT tackle against Romania, England in the rain at the last game at the old Wembley, FIVE-ONE, Beckham's garbage yet match-winning penalty against Argentina and much, much more. Now would you go to see Don Hutchison perform his stand-up show ‘Behind The Frontman' at the Edinburgh Fringe? If yes, please tell us why you think that'd be a good idea: hello@letsbehavingyou.com Plus, we dive into your correspondence as we get DEFINITIVE PROOF on Seth Johnson's contract negotiations at Leeds United. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night, Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by one of the 90s and 00s most entertaining and longest serving chairman…BRING YA DINNER… it's Barry Hearn! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! The former Leyton Orient owner talks Club For A Fiver (…or £2.43), needing ‘therapy' after one too many play-off final defeats and how he managed to take a 35-man squad to Las Vegas for just £5,000. But more than anything, it's life lessons from one of the world's greatest promoters! If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by one of the 90s and 00s most entertaining and longest serving chairman…BRING YA DINNER… it's Barry Hearn! The former Leyton Orient owner talks Club For A Fiver (…or £2.43), needing ‘therapy' after one too many play-off final defeats and how he managed to take a 35-man squad to Las Vegas for just £5,000. But more than anything, it's life lessons from one of the world's greatest promoters! If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Our guest this week is one of the 00s best box-to-box midfielders – former Leeds United, Newcastle United, West Ham United and Birmingham City star Lee Bowyer. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! We hear his crazy story of surviving a plane crash with the rest of the Leeds squad, his hatred of cold weather and Jean-Alain Boumsong egging him and Kieron Dyer on for ANOTHER fight! Plus, we delve into your correspondence, including more 00s injury news – this time it's David Bentley's fitness ahead of Norwich's home match against Manchester United in 2005. If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Our guest this week is one of the 00s best box-to-box midfielders – former Leeds United, Newcastle United, West Ham United and Birmingham City star Lee Bowyer. We hear his crazy story of surviving a plane crash with the rest of the Leeds squad, his hatred of cold weather and Jean-Alain Boumsong egging him and Kieron Dyer on for ANOTHER fight! Plus, we delve into your correspondence, including more 00s injury news – this time it's David Bentley's fitness ahead of Norwich's home match against Manchester United in 2005. If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by Daily Telegraph senior sports writer and QPR fan, Thom Gibbs, to discuss quite possibly the best football documentary of the decade – Four Year Plan! Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! If you want mad owners making batshit decisions constantly, this is the football watchalong for you… Managerial sackings, scandal, and of course… reducing the costs of flowers in corporate hospitality, there's nothing this doc doesn't have. Plus, we delve into your correspondence, which includes one fan going to extraordinary lengths to find out the status of Shaun Goater's fitness ahead of the 2000/01 season and another fan lobbing their season ticket but this time it has a positive impact! If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

For this episode, we are joined by Daily Telegraph senior sports writer and QPR fan, Thom Gibbs, to discuss quite possibly the best football documentary of the decade – Four Year Plan! If you want mad owners making batshit decisions constantly, this is the football watchalong for you… Managerial sackings, scandal, and of course… reducing the costs of flowers in corporate hospitality, there's nothing this doc doesn't have. Plus, we delve into your correspondence, which includes one fan going to extraordinary lengths to find out the status of Shaun Goater's fitness ahead of the 2000/01 season and another fan lobbing their season ticket but this time it has a positive impact! If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Our guest this week is one of the 00's most colourful characters – former Arsenal, Norwich City, Blackburn Rovers and Tottenham Hotspur midfielder David Bentley. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! From fronting up to Patrick Vieira to being at Carrow Road for Delia's infamous half-time speech to throwing up in Craig Bellamy's hat during Blackburn's Christmas do… this podcast has everything. Plus, we delve into possibly one of the greatest pieces of correspondence – Paul Jewell, Wigan Athletic and a fan lobbing his season ticket… do we need to say anymore? If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com And IN CASE YOU MISSED IT: last week we revealed that our guest for the first LBHY live show at the Clapham Grand is none other than Terry Kiely aka Karl Fletcher from Dream Team! Saturday 12th October, it's going to be an incredible night (and there may well be another guest to be announced..) Get your tickets right now (before they all go): https://dice.fm/partner/website/event/eogx66-lets-be-having-you-the-00s-football-podcast-live-from-cheerful-earful-podcast-festival-12th-oct-the-clapham-grand-london-tickets Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Our guest this week is one of the 00's most colourful characters – former Arsenal, Norwich City, Blackburn Rovers and Tottenham Hotspur midfielder David Bentley. From fronting up to Patrick Vieira to being at Carrow Road for Delia's infamous half-time speech to throwing up in Craig Bellamy's hat during Blackburn's Christmas do… this podcast has everything. Plus, we delve into possibly one of the greatest pieces of correspondence – Paul Jewell, Wigan Athletic and a fan lobbing his season ticket… do we need to say anymore? If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com And IN CASE YOU MISSED IT: last week we revealed that our guest for the first LBHY live show at the Clapham Grand is none other than Terry Kiely aka Karl Fletcher from Dream Team! Saturday 12th October, it's going to be an incredible night (and there may well be another guest to be announced..) Get your tickets right now (before they all go): https://dice.fm/partner/website/event/eogx66-lets-be-having-you-the-00s-football-podcast-live-from-cheerful-earful-podcast-festival-12th-oct-the-clapham-grand-london-tickets Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

This week we welcome Quickly Kevin royalty, comedian Tom Parry, to Let's Be Having You to discuss his beloved Wolverhampton Wanderers in the 00s. Wolves had Steve Bull in the 90s BUT this is the decade they finally make it back to the top flight! Tom chats all about the victorious play-off final in 2003 and, of course, possibly the most memorable front of shirt sponsor of the 00s - Doritos! From Dave Jones to Glenn Hoddle to Mick McCarthy - it was a decade of differing styles of play and success. Plus, we delve into your correspondence - did Bolton Wanderers really try to disrupt Dave Challinor's long throw by moving the advertising hoardings close to the pitch in their League Cup semi-final against Tranmere Rovers in 2000? If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com And IN CASE YOU MISSED IT: last week we revealed that our guest for the first LBHY live show at the Clapham Grand is none other than Terry Kiely aka Karl Fletcher from Dream Team! Saturday 12th October, it's going to be an incredible night (and there may well be another guest to be announced..) Get your tickets right now (before they all go): https://dice.fm/partner/website/event/eogx66-lets-be-having-you-the-00s-football-podcast-live-from-cheerful-earful-podcast-festival-12th-oct-the-clapham-grand-london-tickets Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Our guest this week is quite simply one of the greatest players of the era this entire podcast is about. 3x Premier League titles, the Champions League, the Europa league, 4x FA Cups, 2x League cups, 106 caps for his country and Chelsea's all-time record goalscorer.. One of the greatest midfielders ever, what an honour to welcome to Let's Be Having You, Frank Lampard. Part 1 of the podcast came out on Monday, so if you missed it... make sure to go back and listen! From THAT Q&A at Rollerbowl and Harry Redknapp's incredibly prophetic words to watching Jose Mourinho emerge from a laundry basket, this is a noughties footballer interview for the ages. Plus we find out even more about what the cast of Dream Team are up to now plus there's even more (slightly more accurate) trivia to be had. If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com And IN CASE YOU MISSED IT: last week we revealed that our guest for the first LBHY live show at the Clapham Grand is none other than Terry Kiely aka Karl Fletcher from Dream Team! Saturday 12th October, it's going to be an incredible night (and there may well be another guest to be announced..) Get your tickets right now (before they all go): https://dice.fm/partner/website/event/eogx66-lets-be-having-you-the-00s-football-podcast-live-from-cheerful-earful-podcast-festival-12th-oct-the-clapham-grand-london-tickets Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Our guest this week is quite simply one of the greatest players of the era this entire podcast is about. 3x Premier League titles, the Champions League, the Europa league, 4x FA Cups, 2x League cups, 106 caps for his country and Chelsea's all-time record goalscorer.. One of the greatest midfielders ever, what an honour to welcome to Let's Be Having You, Frank Lampard. From THAT Q&A at Rollerbowl and Harry Redknapp's incredibly prophetic words to watching Jose Mourinho emerge from a laundry basket, this is a noughties footballer interview for the ages. Plus we find out even more about what the cast of Dream Team are up to now plus there's even more (slightly more accurate) trivia to be had. If you've got anything to contribute to the show, you know what to do: hello@letsbehavingyou.com And IN CASE YOU MISSED IT: last week we revealed that our guest for the first LBHY live show at the Clapham Grand is none other than Terry Kiely aka Karl Fletcher from Dream Team! Saturday 12th October, it's going to be an incredible night (and there may well be another guest to be announced..) Get your tickets right now (before they all go): https://dice.fm/partner/website/event/eogx66-lets-be-having-you-the-00s-football-podcast-live-from-cheerful-earful-podcast-festival-12th-oct-the-clapham-grand-london-tickets Thank you for subscribing to the show, your support means the world and we couldn't make the show without you. As a reminder, as the QK fan club transitions to the LBHY fan club you'll still get access to all the old QK subscriber episodes, PLUS you'll be getting monthly special episodes from LBHY (which may very well include, Big Ron Manager..) So good night Richard Wright, Chris, Alex and Nick x Learn more about your ad choices. Visit podcastchoices.com/adchoices

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Measuring Structure Development in Algorithmic Transformers, published by Jasmina Nasufi on August 22, 2024 on The AI Alignment Forum. tl;dr: We compute the evolution of the local learning coefficient (LLC), a proxy for model complexity, for an algorithmic transformer. The LLC decreases as the model learns more structured solutions, such as head specialization. This post is structured in three main parts, (1) a summary, giving an overview of the main results, (2) the Fine Print, that delves into various cross-checks and details and (3) Discussion and Conclusions. Structure Formation in Algorithmic Transformers In this work we study the development of simple algorithmic transformers, which are transformers that learn to perform algorithmic tasks. A major advantage of this setup is that we can control several (hyper)parameters, such as the complexity of the training data and network architecture. This allows us to do targeted experiments studying the impacts of these parameters on the learning dynamics. The main tool we use to study the development is the Local Learning Coefficient (LLC) and we choose cases where we have a reverse-engineered solution. Why use the LLC for this purpose? It is a theoretically well motivated measure of model complexity defined by Lau et.al. For an overview of Singular Learning Theory (which serves as the theoretical foundation for the LLC) see Liam Carol's Distilling SLT sequence. For a brief overview of the LLC see e.g. this post. We use the same setup as CallumMcDougall's October Monthly Algorithmic Mech-Interp Challenge. The model is an attention only transformer, trained on sorting numbers with layer norm and weight decay on a cross-entropy loss function using the Adam optimizer. The residual stream size is 96 and the head dimension is 48. It is trained on sequences of the form and to predict the next token starting at the separation token. The numbers in the list are sampled uniformly from 0 to 50, which together with the separation token produce a vocabulary of 52 tokens. Numbers do not repeat in the list. The images making up the gifs can be found here. 1-Head Model Let's first look at the case of a 1-head transformer: The model reaches 100% accuracy around training step 100, confirming that a single attention head is sufficient for sorting, as noted in previous work. Once maximum accuracy is reached, the full QK and OV circuits[2] behave as described by Callum for the 2-head model: In the QK circuit, source tokens attend more to the smallest token in the list larger than themselves. This results in a higher value band above the diagonal and a lower value band below the diagonal. The OV circuit copies tokens, as seen by the clear positive diagonal pattern. In addition, we observe a transition around training step 1000, where the LLC decreases while the accuracy stays unchanged. This is supported by a drop in the sum of the ranks[3] of the matrices in the heat maps. It also coincides with the formation of the off-diagonal stripes in the OV-circuit. We speculate that these are simpler than the noisier off-diagonal OV pattern observed at peak LLC, and correspond to the translational symmetry of the problem. We define a Translational Symmetry measure[1] (see purple line in the plot) to capture the degree to which the circuits obey this symmetry. It increases throughout most of the training, even after the other measures stabilize. 2-Head Model Let's now turn our attention to the 2-head transformer in Callum's original setup. We see a lot of qualitative similarities to the evolution of the full QK and OV circuits for the 1-head model. As the LLC begins to drop (around training step 1000), we note the following: QK circuit: Slight changes[5] to the attention pattern, which crystallize into triangular regions late in the training, long aft...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Measuring Structure Development in Algorithmic Transformers, published by Micurie on August 22, 2024 on LessWrong. tl;dr: We compute the evolution of the local learning coefficient (LLC), a proxy for model complexity, for an algorithmic transformer. The LLC decreases as the model learns more structured solutions, such as head specialization. This post is structured in three main parts, (1) a summary, giving an overview of the main results, (2) the Fine Print, that delves into various cross-checks and details and (3) Discussion and Conclusions. Structure Formation in Algorithmic Transformers In this work we study the development of simple algorithmic transformers, which are transformers that learn to perform algorithmic tasks. A major advantage of this setup is that we can control several (hyper)parameters, such as the complexity of the training data and network architecture. This allows us to do targeted experiments studying the impacts of these parameters on the learning dynamics. The main tool we use to study the development is the Local Learning Coefficient (LLC) and we choose cases where we have a reverse-engineered solution. Why use the LLC for this purpose? It is a theoretically well motivated measure of model complexity defined by Lau et.al. For an overview of Singular Learning Theory (which serves as the theoretical foundation for the LLC) see Liam Carol's Distilling SLT sequence. For a brief overview of the LLC see e.g. this post. We use the same setup as CallumMcDougall's October Monthly Algorithmic Mech-Interp Challenge. The model is an attention only transformer, trained on sorting numbers with layer norm and weight decay on a cross-entropy loss function using the Adam optimizer. The residual stream size is 96 and the head dimension is 48. It is trained on sequences of the form and to predict the next token starting at the separation token. The numbers in the list are sampled uniformly from 0 to 50, which together with the separation token produce a vocabulary of 52 tokens. Numbers do not repeat in the list. 1-Head Model Let's first look at the case of a 1-head transformer: The model reaches 100% accuracy around training step 100, confirming that a single attention head is sufficient for sorting, as noted in previous work. Once maximum accuracy is reached, the full QK and OV circuits[2] behave as described by Callum for the 2-head model: In the QK circuit, source tokens attend more to the smallest token in the list larger than themselves. This results in a higher value band above the diagonal and a lower value band below the diagonal. The OV circuit copies tokens, as seen by the clear positive diagonal pattern. In addition, we observe a transition around training step 1000, where the LLC decreases while the accuracy stays unchanged. This is supported by a drop in the sum of the ranks[3] of the matrices in the heat maps. It also coincides with the formation of the off-diagonal stripes in the OV-circuit. We speculate that these are simpler than the noisier off-diagonal OV pattern observed at peak LLC, and correspond to the translational symmetry of the problem. We define a Translational Symmetry measure[1] (see purple line in the plot) to capture the degree to which the circuits obey this symmetry. It increases throughout most of the training, even after the other measures stabilize. 2-Head Model Let's now turn our attention to the 2-head transformer in Callum's original setup. We see a lot of qualitative similarities to the evolution of the full QK and OV circuits for the 1-head model. As the LLC begins to drop (around training step 1000), we note the following: QK circuit: Slight changes[5] to the attention pattern, which crystallize into triangular regions late in the training, long after the LLC has stabilized. OV circuit: The heads specialize, splittin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2, published by Neel Nanda on July 7, 2024 on The AI Alignment Forum. This post represents my personal hot takes, not the opinions of my team or employer. This is a massively updated version of a similar list I made two years ago There's a lot of mechanistic interpretability papers, and more come out all the time. This can be pretty intimidating if you're new to the field! To try helping out, here's a reading list of my favourite mech interp papers: papers which I think are important to be aware of, often worth skimming, and something worth reading deeply (time permitting). I've annotated these with my key takeaways, what I like about each paper, which bits to deeply engage with vs skim, etc. I wrote a similar post 2 years ago, but a lot has changed since then, thus v2! Note that this is not trying to be a comprehensive literature review - this is my answer to "if you have limited time and want to get up to speed on the field as fast as you can, what should you do". I'm deliberately not following academic norms like necessarily citing the first paper introducing something, or all papers doing some work, and am massively biased towards recent work that is more relevant to the cutting edge. I also shamelessly recommend a bunch of my own work here, sorry! How to read this post: I've bolded the most important papers to read, which I recommend prioritising. All of the papers are annotated with my interpretation and key takeaways, and tbh I think reading that may be comparable good to skimming the paper. And there's far too many papers to read all of them deeply unless you want to make that a significant priority. I recommend reading all my summaries, noting the papers and areas that excite you, and then trying to dive deeply into those. Foundational Work A Mathematical Framework for Transformer Circuits (Nelson Elhage et al, Anthropic) - absolute classic, foundational ideas for how to think about transformers (see my blog post for what to skip). See my youtube tutorial (I hear this is best watched after reading the paper, and adds additional clarity) Deeply engage with: All the ideas in the overview section, especially: Understanding the residual stream and why it's fundamental. The notion of interpreting paths between interpretable bits (eg input tokens and output logits) where the path is a composition of matrices and how this is different from interpreting every intermediate activations And understanding attention heads: what a QK and OV matrix is, how attention heads are independent and additive and how attention and OV are semi-independent. Skip Trigrams & Skip Trigram bugs, esp understanding why these are a really easy thing to do with attention, and how the bugs are inherent to attention heads separating where to attend to (QK) and what to do once you attend somewhere (OV) Induction heads, esp why this is K-Composition (and how that's different from Q & V composition), how the circuit works mechanistically, and why this is too hard to do in a 1L model Skim or skip: Eigenvalues or tensor products. They have the worst effort per unit insight of the paper and aren't very important. Superposition Superposition is a core principle/problem in model internals. For any given activation (eg the output of MLP13), we believe that there's a massive dictionary of concepts/features the model knows of. Each feature has a corresponding vector, and model activations are a sparse linear combination of these meaningful feature vectors. Further, there are more features in the dictionary than activation dimensions, and they are thus compressed in and interfere with each other, essentially causing cascading errors. This phenomena of compression is called superposition. Toy models of superpositio...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decomposing the QK circuit with Bilinear Sparse Dictionary Learning, published by keith wynroe on July 2, 2024 on LessWrong. This work was produced as part of Lee Sharkey's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort Intro and Motivation Sparse dictionary learning (SDL) has attracted a lot of attention recently as a method for interpreting transformer activations. They demonstrate that model activations can often be explained using a sparsely-activating, overcomplete set of human-interpretable directions. However, despite its success for explaining many components, applying SDL to interpretability is relatively nascent and have yet to be applied to some model activations. In particular, intermediate activations of attention blocks have yet to be studied, and provide challenges for standard SDL methods. The first challenge is bilinearity: SDL is usually applied to individual vector spaces at individual layers, so we can simply identify features as a direction in activation space. But the QK circuits of transformer attention layers are different: They involve a bilinear form followed by a softmax. Although simply applying sparse encoders to the keys and queries[1] could certainly help us understand the "concepts" being used by a given attention layer, this approach would fail to explain how the query-features and key-features interact bilinearly. We need to understand which keys matter to which queries. The second challenge is attention-irrelevant variance: A lot of the variance in the attention scores is irrelevant to the attention pattern because it is variance in low scores which are softmaxed to zero; this means that most of the variability in the keys and queries is irrelevant for explaining downstream behaviour[2]. The standard method of reconstructing keys and queries would therefore waste capacity on what is effectively functionally irrelevant noise. To tackle these two problems (bilinearity and attention-irrelevant variance), we propose a training setup which only reconstructs the dimensions of the keys and queries that most affect the attention pattern. Training Setup Our training process has two steps: Step 1: Reconstructing the attention pattern with key- and query- encoder-decoder networks Step 2: Finding a condensed set of query-key feature pairs by masking Step 1: Reconstructing the attention pattern with key- and query-transcoders Architecture Our first training step involves training two sparse dictionaries in parallel (one for the keys and one for the queries). The dictionaries both take in the layer-normalized residual stream at a given layer (normalised_resid_pre_i) and each output a [n_head * d_head] vector, representing the flattened keys and queries[3]. Figure 1: High-level diagram of our training set-up Loss functions However, rather than penalising the reconstruction loss of the keys and queries explicitly, we can use these keys and queries to reconstruct the original model's attention pattern. To train the reconstructed attention pattern, we used several different losses: KL divergence between the attention pattern (using reconstructed keys and reconstructed queries) and the ground-truth attention pattern produced by the original model. We also added two auxiliary reconstruction losses both for early-training-run stability, and to ensure our transcoders do not learn to reconstruct the keys and queries with an arbitrary rotation applied (since this would still produce the same attention scores and patterns): KL divergence between the attention pattern (using reconstructed keys and the original model's queries) and the ground-truth attention pattern produced by the original model. KL divergence between the attention pattern (using the original models' keys and the reconstructed queries) and the ground-truth atten...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decomposing the QK circuit with Bilinear Sparse Dictionary Learning, published by keith wynroe on July 2, 2024 on The AI Alignment Forum. This work was produced as part of Lee Sharkey's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort Intro and Motivation Sparse dictionary learning (SDL) has attracted a lot of attention recently as a method for interpreting transformer activations. They demonstrate that model activations can often be explained using a sparsely-activating, overcomplete set of human-interpretable directions. However, despite its success for explaining many components, applying SDL to interpretability is relatively nascent and have yet to be applied to some model activations. In particular, intermediate activations of attention blocks have yet to be studied, and provide challenges for standard SDL methods. The first challenge is bilinearity: SDL is usually applied to individual vector spaces at individual layers, so we can simply identify features as a direction in activation space. But the QK circuits of transformer attention layers are different: They involve a bilinear form followed by a softmax. Although simply applying sparse encoders to the keys and queries[1] could certainly help us understand the "concepts" being used by a given attention layer, this approach would fail to explain how the query-features and key-features interact bilinearly. We need to understand which keys matter to which queries. The second challenge is attention-irrelevant variance: A lot of the variance in the attention scores is irrelevant to the attention pattern because it is variance in low scores which are softmaxed to zero; this means that most of the variability in the keys and queries is irrelevant for explaining downstream behaviour[2]. The standard method of reconstructing keys and queries would therefore waste capacity on what is effectively functionally irrelevant noise. To tackle these two problems (bilinearity and attention-irrelevant variance), we propose a training setup which only reconstructs the dimensions of the keys and queries that most affect the attention pattern. Training Setup Our training process has two steps: Step 1: Reconstructing the attention pattern with key- and query- encoder-decoder networks Step 2: Finding a condensed set of query-key feature pairs by masking Step 1: Reconstructing the attention pattern with key- and query-transcoders Architecture Our first training step involves training two sparse dictionaries in parallel (one for the keys and one for the queries). The dictionaries both take in the layer-normalized residual stream at a given layer (normalised_resid_pre_i) and each output a [n_head * d_head] vector, representing the flattened keys and queries[3]. Figure 1: High-level diagram of our training set-up Loss functions However, rather than penalising the reconstruction loss of the keys and queries explicitly, we can use these keys and queries to reconstruct the original model's attention pattern. To train the reconstructed attention pattern, we used several different losses: KL divergence between the attention pattern (using reconstructed keys and reconstructed queries) and the ground-truth attention pattern produced by the original model. We also added two auxiliary reconstruction losses both for early-training-run stability, and to ensure our transcoders do not learn to reconstruct the keys and queries with an arbitrary rotation applied (since this would still produce the same attention scores and patterns): KL divergence between the attention pattern (using reconstructed keys and the original model's queries) and the ground-truth attention pattern produced by the original model. KL divergence between the attention pattern (using the original models' keys and the reconstructed queries) and the groun...

Jennifer is a one of a kind. She owns Quinebaug Kennels. QK is a premier facility in Connecticut. From training to trialing, boarding and grooming, you name it, and QK does it. I was able to meet up with Jennifer this past October in my home state of Michigan. I caught up with her on the last leg of her yearly hunt that started in ND and ended in the U.P. She travels across the country, meeting up with friends and their dogs, and does it in a classic upland fashion.