Podcast appearances and mentions of mark donnigan

For this special podcast, Mark Donnigan and I recap key takeaways from the 2025 NAB Show in Las Vegas, discussing what we heard on the show floor and in sessions. We detail the NAB's attendance numbers and the impact I saw on the number of attendees due to the uncertainty in the financial market. We also highlight the many different metrics that exhibitors, sponsors and attendees use to determine the success of shows. We discuss what messaging we think worked best from vendors and key points made by speakers and presenters around CDN, AI, CTV, niche OTT services and where we see the need for improvement in transparency across the industry. Podcast produced by Security Halt Media

In the conversation, Mark Donnigan and I discuss the challenges and opportunities of marketing campaigns in a rapidly changing business landscape. We emphasize the importance of understanding the intersection of marketing and sales, and how startups can optimize their marketing budget for remarkable results. Later, we discuss category design, active listening, and the evolving nature of sales and marketing strategies. We highlight the significance of simplicity, authenticity, and tailored approaches for breaking into new markets. Finally, Mark emphasizes the importance of a conductor-like figure to coordinate various marketing disciplines.Look at these links to discover more about Mark and his work:Website:       https://growthstage.marketing/Facebook:    https://www.facebook.com/MarkDonniganEntrepreneurYouTube:      https://www.youtube.com/c/MarkDonniganLinkedIn:       https://www.linkedin.com/in/markdonnigan/ABOUT THE GUESTMark has been a virtual CMO and Start Up Marketing and Business Consultant for over 14 years. During his career he has sported such titles as VP of Marketing, VP in Sales, General Manager and Business Development Director, to name a few.Mark seeks out all those that consider themselves as ambitious technical or product-focused founders in the video streaming or technology startup ecosystem and are ready to disrupt.ABOUT THE HOSTMy name is Dave Barr and I am the Founder and Owner of RLB Purchasing Consultancy Limited.I have been working in Procurement for over 25 years and have had the joy of working in a number of global manufacturing and service industries throughout this time.I am passionate about self development, business improvement, saving money, buying quality goods and services, developing positive and effective working relationships with suppliers and colleagues, and driving improvement through out the supply chain.Now I wish to share this knowledge and that of highly skilled and competent people with you, the listener, in order that you may hopefully benefit from this information.CONTACT DETAILS@The Real Life BuyerEmail: david@thereallifebuyer.co.ukWebsite: https://linktr.ee/thereallifebuyerFor Purchasing Consultancy services:https://rlbpurchasingconsultancy.co.uk/Email: contact@rlbpurchasingconsultancy.co.ukFind and Follow me @reallifebuyer on Facebook, Instagram, X, Threads and TikTok.Click here for some Guest Courses - https://www.thereallifebuyer.co.uk/guest-courses/Click here for some Guest Publications - https://www.thereallifebuyer.co.uk/guest-publications

This week, Mark Donnigan and I recap news from IBC about using cloud-based compute services for video workflows, encoding optimization, AV1/codec adoption, live encoding use cases, and AI's potential impact on encoding platforms. We also give updates on the Venu/Fubo lawsuit, price increases for NBA League Pass, the upgraded Roku Ultra device, Google rebranding their FAST channels, and some viewership numbers from Peacock. Finally, we discuss some interesting statements from Netflix regarding their ad tier, the NFL games on Christmas and how they view one-off sporting events versus full seasons.Companies mentioned include Akamai, Amagi, AOM, ATEME, AWS, Bitmovin, Brightcove, CDN77, Disney+ Hotstar, Dolby, Edgio, Harmonic, iSIZE, Neilsen, Netflix, NETINT, NVIDIA, THEO Technologies, Visionular, WaveOne, YouTube.Thanks to this week's podcast sponsors: Integrated Digital Solutions and Netskrt Systems.Podcast produced by Security Halt Media

In this episode of Marketing B2B Technology, Mike chats with Mark Donnigan, a virtual CMO who works with tech companies. Mark shares insights into his career journey, discusses his approach to building long-term client relationships, and emphasises the importance of understanding the market and customers. Mark also shares his advice on marketing tactics, highlights the value of focusing on go-to-market strategy, and talks about the importance of getting into the field to understand customers.   About Mark Donnigan Mark Donnigan designs and executes marketing programs and go-to-market strategies to establish and grow markets for disruptive startup companies. As a transformative B2B marketing and business leader, Mark understands what's required to succeed in today's winner-takes-all market. Well-versed in SaaS, software licensing, enterprise technology, and platform business models, Mark helps companies build efficient marketing teams that routinely outperform larger marketing departments.   Time Stamps [00:44.0] - Mark shares his career journey and explains his role as Virtual CMO. [12:57.0] - The benefits of hiring a Virtual CMO versus full-time CMO. [14:45.0] - Mark talks about his approach to building marketing plans. [18:42.0] - Overrated marketing channels and tactics [23:26.0] - Challenges with fixed KPIs in marketing [24:37.0] - Mark offers some marketing advice [25:52.0] - Mark's contact details   Quotes “Get into the field. Know the market. Know the customers. Know how they think. Know what they care about. Know how they make decisions.” Mark Donnigan, Virtual CMO.   Follow Mark: Mark Donnigan on LinkedIn: https://www.linkedin.com/in/markdonnigan/   Growth Stage Marketing website: https://growthstage.marketing/   Follow Mike: Mike Maynard on LinkedIn: https://www.linkedin.com/in/mikemaynard/ Napier website: https://www.napierb2b.com/ Napier LinkedIn: https://www.linkedin.com/company/napier-partnership-limited/   If you enjoyed this episode, be sure to subscribe to our podcast for more discussions about the latest in Marketing B2B Tech and connect with us on social media to stay updated on upcoming episodes. We'd also appreciate it if you could leave us a review on your favourite podcast platform. Want more? Check out Napier's other podcast - The Marketing Automation Moment: https://podcasts.apple.com/ua/podcast/the-marketing-automation-moment-podcast/id1659211547  

With all the recent sports streaming news, co-host Mark Donnigan and I did a special “Sports Streaming Podcast Episode” this week. We discuss the upcoming Venu Sports offering (Disney/WBD/FOX sports JV) and the limitations we expect the service to have when it launches. We weigh in on some of the tech behind the scenes and consider whether a 'beta' label could soften the blow for fans expecting a fully polished service from day one.We also cover Netflix's deal with the NFL, including its importance as the first global licensing deal with no blackout restrictions and how the agreement is structured. Finally, we highlight the NFL's 2024 fragmented distribution strategy across broadcast and streaming, which is great for the NFL's revenue but terrible for fans. (Correction: I say on the podcast the games will be on seven broadcast networks and platforms, but it's ten or eleven, depending on if you separate ESPN from ABC and add in YouTube TV.)Also briefly discussed are Roku's deal with MLB for Sunday morning baseball games, Sky Sports+ launching this August, and how, months later, Max's Bleacher Report (B/R) Sports Add package is still free for users.Podcast produced by Security Halt Media

Whether you are a SaaS founder looking to make your first marketing hire or a CMO building out your team, the rocky tech economy has created a surplus of marketing talent. VPs and Directors from big tech are free agents…fantastic hires, right? According to Mark Donnigan, virtual CMO and consultant, the answer isn't always yes. In this episode, we tackle choosing the right type of hire (generalist or conductor?) and then dive into how to think more strategically about the top talent available today. Connect with Mark: https://www.linkedin.com/in/markdonnigan/ Connect with Lindsey: https://www.linkedin.com/in/lindseygroepper/ Learn more about BLASTmedia: https://www.blastmedia.com/

 Subscribe: https://www.youtube.com/channel/UCNJI2Tv2yxOLBI02nyJoprA  Learn more about us: https://www.pragmaticinstitute.com/  Start your Product Manager or Product Marketer certification here "In Marketing, our job is to keep our ear very close to the ground, understand the process, listen to the buyers, listen to the customer, and always be thinking about what drives the customer." -Mark Donnigan, Startup Marketing Consultant In this episode of Product Chats, we are joined by marketing superhero Mark Donnigan. With a focus on growth-stage marketing, Mark has carved a niche in the tech industry as a consultant who helps B2B companies, particularly startups and those in the growth phase, to craft and execute successful go-to-market strategies. His approach is deeply rooted in the belief that marketing is an integral component of business strategy, inseparable from product development and sales. Key Takeaways: Consistently Listen and Engage with Customers: The significance of staying attuned to the customers' needs and the market dynamics cannot be overstated. Engaging with customers regularly and genuinely understanding their business can unveil invaluable insights for product development and marketing. Guide Your Buyers to Purchase: There is a five-step framework that guides B2B purchases. This process includes identifying problems, exploring solutions, building requirements, selecting vendors, and undergoing a validation process. Mark advocates for creating valuable, educational content tailored to each step of the customer's buying journey. This approach is vital for engaging potential buyers early and establishing your brand as a trusted advisor. Leverage an Iterative Marketing Approach: Embrace a highly iterative marketing framework that focuses on executing in small chunks, measuring results, and refining strategies based on market feedback. This process ensures that marketing efforts are aligned with customer needs and market trends. Understanding the Buying Process is Vital: Recognize the critical milestones in the customer's buying journey and ensure that your marketing materials and strategies support and facilitate this process, helping to guide the customer from problem identification to solution validation. ABOUT OUR GUEST: Mark Donnigan is a Virtual CMO and GTM consultant that designs and executes go-to-market and marketing programs that produce real business results for technology startup companies. Dive deeper into Mark's insights and strategies by checking out his Minibook, "First Principles in Marketing for Startup Founders and CEOs." This guide is a treasure trove of knowledge, offering a fresh perspective on marketing by breaking down problems to their core and building innovative solutions that meet and redefine customer expectations. ABOUT PRAGMATIC INSTITUTE: Since 1993, we've provided training and professional development for product, design, and data professionals to accelerate innovation, increase customer satisfaction, and bolster revenue. Through our comprehensive methods and propriety Pragmatic Framework, we train professionals with the skills needed to accelerate their product management and product marketing careers, increase team efficiency, and boost revenue. Our online, in-person, and on-demand courses are taught by industry experts with decades of experience at top companies like Microsoft, Cisco, Dell, Hallmark, Quaker Oats, and Dun & Bradstreet. We offer individual course training in product, design and data, and certification tracks for Product Management Certifications, Product Marketing Certifications, and the Pragmatic Product Master Certification. All attendees gain access to the Pragmatic Alumni Community, a network of over 30,000 alumni worldwide. If you're looking for product management or product marketing training or are pursuing a product marketing certification or product management certification, browse our certification paths at https://www.pragmaticinstitute.com/product/. Courses are available online, in-person, or on-demand to fit your schedule. Want to learn more about becoming a Pragmatic Certified Product Manager? Learn more here. #ProductManagement #ProductManager #ProductManagementCertification #ProductManagementStrategy #PragmaticInstitute

On this episode of Healthcare Market Matrix, host John Farkas is joined by Mark Donnigan, virtual CMO and go-to-market consultant at Growth Stage Marketing, a company that builds sustainable marketing strategies while executing go-to-market plans that yield real business results by being rooted in first principles thinking. Mark has extensive experience working alongside sales, product, operations, and finance to ensure his client's objectives are always exceeded. As an accomplished storyteller and expert content marketer, Mark believes the modern marketing function is built on the foundation of content publishing with category design, ABM, SEO, conversion rate optimization, email outreach, and paid advertising tactics. Throughout the episode, Mark shares his vast knowledge around the importance of leading with value during go-to-market efforts and analyzes a recent product launch that failed to effectively and easily convey its value proposition to its buyers, ultimately leading to its downfall.   Show Notes (1:47) Introducing Mark Donnigan and Growth Stage Marketing (10:12) Understanding Where to Start from the Nature of Go-to-Market (18:21) Leading with Value in a Constrained Healthcare Climate (31:19) Analyzing Humane AI's Recent Launch Failure (45:15) Asking Hard Questions during Go-to-Market Efforts (56:34) Closing Thoughts  

Guest: Mark Donnigan, Virtual CMO and Go-to-Market Engineer for Tech Startups at Growth Stage MarketingOn LinkedIn | https://www.linkedin.com/in/markdonnigan/Host: Brendon RodOn ITSPmagazine | https://www.itspmagazine.com/itspmagazine-podcast-radio-hosts/brendon-rod____________________________This Episode's SponsorsAre you interested in sponsoring an ITSPmagazine Channel?

Guest: Mark Donnigan, Virtual CMO and Go-to-Market Engineer for Tech Startups at Growth Stage MarketingOn LinkedIn | https://www.linkedin.com/in/markdonnigan/Host: Brendon RodOn ITSPmagazine | https://www.itspmagazine.com/itspmagazine-podcast-radio-hosts/brendon-rod____________________________This Episode's SponsorsAre you interested in sponsoring an ITSPmagazine Channel?

This week on the Digital Velocity Podcast, Mark Donnigan of Growth Stage Marketing joins Erik and Tim to discuss why category design marketing is so beneficial when launching, building, and scaling a business. https://www.digitalvelocitypodcast.com/episodes/57-why-category-design-is-so-beneficial-mark-donnigan  

Join Scott "Shalom" Klein on his weekly radio show, Get Down To Business with guests: Drew Jones Merideth Mehlberg Michelle LaBrosse Mark Donnigan

In today's riveting episode of the SaaS Fuel podcast, Jeff engages in a conversation with Mark Donnigan, a B2B tech maestro, Virtual CMO extraordinaire, and the brains behind Growth Stage Marketing. Mark exposes the secret sauce of messaging that not only grabs attention but also sparks action. Forget the boring product feature lists! Mark takes us on a journey where storytelling reigns supreme. And he's not just about selling a solution; he's about connecting with the soul of a problem. Measuring marketing effectiveness? It's not as straightforward as it seems but Mark delves into the intricacies, laying bare the challenges that many of us grapple with. Plus, ever heard of category design? He also unpacks this mind-bending concept, giving us a glimpse into the future of marketing strategy.Get ready for some serious insights into the art of navigating those convoluted sales cycles and join us for a chat that's equal parts insightful and downright entertaining. You don't want to miss this!

Mark Donnigan joins Otis and Camden to discuss building companies, go-to-market strategies, and the advantages of lifelong learning.  How can category design help your business?  What can you do to avoid your business' product/service being treated as a commodity?  How can you differentiate industry disruptors from bad ideas?  Mark, Cam, and Otis dive into all this and more on this fantastic episode!Thank you to our sponsors Evergood Adventure Wines and Tribe and Purpose!!- Order your lemon wine at  https://www.evergoodadventurewines.com/buy-online/              DISCOUNT CODE “CAM-OTIS” for 20% off your order!!- Learn how The Green Beret Leadership Program can help your business:  https://findyourpurpose.coach/GBLP/ More About Mark:Mark Donnigan designs and executes marketing programs and go-to-market strategies to establish and grow markets for disruptive innovation startup companies. As a transformative B2B marketing and business leader, Mark understands what's required to succeed in today's winner-takes-all market. Leveraging marketing and growth tactics that work such as category design, Mark produces real business results for early and growth-stage technology companies. Well-versed in SaaS, software licensing, enterprise technology, and platform business models, Mark helps companies build nimble, highly efficient marketing teams that routinely outperform larger marketing departments. Mark is passionate about extracting the most value from every marketing dollar invested. He provides startup founders at all stages of growth with a way to build their revenue engine with high-impact marketing playbooks to reach their revenue goals and scale sustainably.Mark DonniganLinkedIn: https://www.linkedin.com/in/markdonnigan/Facebook: https://www.facebook.com/MarkDonniganEntrepreneurBusiness LinkedIn: https://www.linkedin.com/company/growthstagemarketing/

We've all seen the stats about how CMOs have the shortest average tenure in the C-Suite.But having a constant revolving door of marketing leadership isn't just bad for individual marketing leaders, it can have major negative impacts on teams, brands, and the marketing industry as a whole.We sat down with Virtual CMO, Mark Donnigan to discuss his tried and tested framework for ensuring CMOs have the time and space to really show their value.

Join Mark Donnigan as he unravels the secrets behind building iconic brands in the tech and marketing world. Delve into the strategies that propelled Apple to smartphone dominance and Red Bull to energy drink royalty. Mark sheds light on the misconceptions of online gurus and shares insights that resonate with real-life experiences. Discover the power of category creation and positioning, empowering startups to stand out and dominate their markets. In this episode, Mark challenges the fear of niche targeting and advocates for amplifying your voice through strategic marketing, a lesson applicable to businesses of all sizes. Tune in for a masterclass in brand choreography and learn how to orchestrate success in the competitive landscape.   More about Mark:  Today's guest, Mark Donnigan, builds technology companies as a virtual CMO and business consultant. Starting his career as a seller and advancing to sales leadership while bridging to marketing and business strategy functions, Mark is a revenue-first marketer who has produced $500 million in revenue and enterprise value for the companies he's worked with.    Want to find more from Mark? Check out… https://growthstage.marketing ------------------------------------------------------------------ Want more Business Choreography?  Check out… Business Choreography Website  Join The Business Choreography Group  Text Us "CHOREO NOW" To Get Started: 385-442-7188   Need the tech to build your Business Choreography? Check out.... ChoreoSuite   Connect With Michael Johnson: LinkedIn | Instagram | Facebook | YouTube | Twitter | TikTok  

Visit thedigitalslicepodcast.com for complete show notes of every podcast episode. In this episode of The Digital Slice Podcast, Brad Friedman and Mark Donnigan chat about designing and executing marketing programs and go-to-market strategies for disruptive innovation startup companies. Mark Donnigan builds technology companies as a virtual CMO and business consultant. Starting his career as a seller and advancing to sales leadership while bridging to marketing and business strategy functions, Mark is a revenue-first marketer who has produced $500 million in revenue and enterprise value for the companies he's worked with. Be prepared to have your marketing beliefs challenged and your business thinking put on tilt.

In this episode of the Interesting B2B Marketers podcast, Steve Goldhaber is joined by Mark Donnigan, a dynamic professional whose career spans across computer science, music, and revenue strategy. Mark's unique blend of technical expertise and marketing savvy offers listeners a rich, multifaceted perspective on the marketing landscape.The conversation covers a range of compelling topics, including:The complexities and extended sales cycles in B2B technology sales, such as video encoding integration.The evolution of marketing strategies that emphasize a deep understanding of the audience's mindset.The shift from traditional brand promotion to fostering industry-wide dialogues and building communities.The importance of understanding customer problems over solely focusing on product specifications in B2B sales and marketing within technical environments.The need for marketers to have a deep understanding of their industry ecosystem, the customer's needs, and the ability to frame a product's value in a problem-solving context.The role of credibility and setting aside ego in marketing is exemplified by personal anecdotes from Mark's early experiences in marketing.The risks of marketers becoming isolated in their 'marketing bunker' and the importance of direct conversation in marketing.The challenges and cultural aspects of Salesforce integration, and the importance of understanding user perspectives.A brief reflection on their experiences and excitement working in the tech sector.To keep up with the insightful conversations and learn more, connect with Mark Donnigan and Steve Goldhaber on LinkedIn.

Want to know the true secret to growing a business? Psst, it's the customers. This week's episode will make your company start caring about its customers. Today's guest is Mark Donnigan, Virtual CMO at Growth Stage Marketing. Listen in as Mark joins host, Casey Cheshire, and shares his hack on building a better relationship with your customers through excellent business strategy.     Busted Myths:  Companies can have a great strategy and never lose customers. Takeaways: Learn your competitor's tools, then learn their process, and then implement it. It is all about driving more usage and purchase of their solution. It is not actually about selling our products. Just because you hire someone who is good on paper does not mean that they are going to be great. Actions speak louder than words. Get on sales calls and go to sales calls. This will help you learn about your customers and about other businesses.  No matter what you have going on with your business. The customers most likely will not care. They only care about what they will get out of it. They only care if it has anything to do with them.  Marketing podcasts are a great way to get real-time marketing information. Marketing books are typically made within the year while podcasts are made almost every week. Diversity of thought is required because without it, we would fail. We would not execute meetings well it will lead to no business impact. It's more important to get any information to your customers. Make it readily available, and make it easily accessible for your customer.     Quote of the Show: “The customer does not care about us. They only care about them.” (11:28) Book Recommendations: Play Bigger by Al Ramadan, Dave Peterson, Christopher Lochhead, and Kevin Maney     Links: LinkedIn: https://www.linkedin.com/in/markdonnigan/ Website: https://growthstage.marketing/ Podcast: https://www.owltail.com/people/ORNFP-mark-donnigan/appearances  Company Youtube: https://www.youtube.com/channel/UCWivu309kk1u1P9jrcv3jLA Ways to Tune In: Apple Podcasts: https://podcasts.apple.com/us/podcast/the-hard-corps-marketing-show/id1338838763 Amazon Music/Audible: https://music.amazon.com/podcasts/37228621-2f9c-4905-a223-1844effb49dd Spotify:  https://open.spotify.com/show/1vVLpNI1LssMTiL6Kdsamn Stitcher: https://www.stitcher.com/podcast/the-hard-corps-marketing-show Google Podcasts: https://play.google.com/music/m/Im7mytmu2wa2mekhoeixlja5hpe?t=The_Hard_Corps_Marketing_Show YouTube: https://youtu.be/5MrbOivXAz8

Marketing and sales alignment is pivotal in the modern era of sales, and building credibility and rapport in your communication for all departments is essential. In today's episode of The Sales Evangelist, Donald is joined by the CMO of Growth Stage Marketing, Mark Donnigan, to discuss his take on building rapport while unifying departmental collaboration. Good CMOs think about marketing campaigns. Great CMOs bring value to the enterprise. Between the evolution of technology and the pandemic, sales has completely transformed. Buyers no longer attend trade shows to talk with peers; they organize entirely through online forums. Buyers are 50% or more through the buying process before reaching out to their first vendor. A seller's job is no longer to get the first meeting; it's to state the problem that the solution solves while differentiating from competitors. CMO turnover is almost exclusively because they repeatedly try the old playbook rather than embrace these new changes. Credibility plays into rapport building: There's one thing sellers can do at all times - add value to their network through social channels. Nobody wants to be sold to, so content shouldn't be explicitly sales-focused. Instead, create helpful content the audience will read.  People are drawn to others who make intelligent observations about a problem or solution.  Add value to customers as someone who can bring thoughtful insights to the industry - that's the open door in the modern sales landscape. Only 2% of LinkedIn users post content, so seeing thoughtful and relevant content from a person on the platform makes an impression on anyone.  Create your own opportunities by providing internal and external value. Too many executives see themselves as professional managers, but that position is a commodity. However, when you can add insight to a large enterprise organization, you become a lot harder to replace. The new Rolodex isn't names and addresses; it's the community that forms around the network. People who like, respect, and are engaged with your content are more valuable than a simple name and number registry. Create a network that respects your insights and content. There are plenty of other competitors who are just as competent and insightful. However, nobody knows it because they don't post. Mark's final takeaway? Focus on the seller - find every possible way to add value to those in the ecosystem around you. Visit his company website at growthstage.marketing and connect with Mark on LinkedIn for more interesting and insightful content.  This episode is brought to you in part by LinkedIn Sales Navigator. The Great Resignation has become the Great Reshuffle, meaning it can be difficult for sales professionals like you to find leads and close deals. Luckily, Sales Navigator from LinkedIn is here for you! Sales Navigator from LinkedIn is the only tool that uses real-time alerts and up-to-date insights to help you know when prospects are ready to buy. And, with over 30 advanced filters, sales professionals can quickly find genuine leads with the intent to purchase. Gain the advantage of accurate, quality lead generation data from LinkedIn Sales Navigator. You can get a 60-day free trial of Sales Navigator at www.LinkedIn.com/TSE.  This episode is brought to you in part by Skipio. Are you sick of crickets? As a salesperson, the pain of reaching out with phone calls or emails and not receiving a response is real. But all text messaging is not created equal. 85% of people prefer text over email and phone calls because they want to engage in a conversation, not listen to bots. Be more like people and start having conversations that end in the conversions you want. Try Skipio at www.Skipio.com. This episode is brought to you in part by Closers.io. Closers.io helps sales reps land their dream remote sales gig, where they can set their own hours, work from anywhere, and make six or even multi-6 figures per year. That sure sounds good to us! Committed to helping sales reps make a shift, Closers.io will place you in an available sales role that will increase your commissions and help you live the life you want. Apply for free now at go.closers.io/TSE. This episode is brought to you in part by Scratchpad. Are you tired of a digital workspace cluttered with notes, folders, files, and half-filled spreadsheets? (Not that we're speaking from personal experience.) Luckily, we've found the solution. Scratchpad is the first Revenue Team Workspace specifically designed to adapt to each salesperson's workflow, so you don't have to change your habits. Scratchpad creates a streamlined workflow that allows everyone to be a little more productive each day without the hassle of updating a database with whatever info you can find. Get Scratchpad free at Scratchpad.com.  As one of our podcast listeners, we value your opinion and always want to improve the quality of our show. Complete our two-minute survey here: thesalesevangelist.com/survey. We'd love for you to join us for our next episodes by tuning in on Apple Podcast, Google Podcast, Stitcher, or Spotify. Audio provided by Free SFX, Soundstripe, and Bensound. Other songs used in the episodes are as follows: The Organ Grinder written by Bradley Jay Hill, performed by Bright Seed, and Produced by Brightseed and Hill.

Today we're going to discuss how to build a go to market strategy. Joining us is Mark Donnigan, who is a Marketing and Business Growth Consultant. In part 2 of our conversation, we discuss engineering's role in a marketing go-to-market plan. Show NotesConnect With:Mark Donnigan: Website // LinkedInThe MarTech Podcast: Email // Newsletter // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Today we're going to discuss how to build a go to market strategy. Joining us is Mark Donnigan, who is a Marketing and Business Growth Consultant. In part 2 of our conversation, we discuss engineering's role in a marketing go-to-market plan. Show NotesConnect With:Mark Donnigan: Website // LinkedInThe MarTech Podcast: Email // Newsletter // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Today we're going to discuss how to build a go to market strategy. Joining us is Mark Donnigan, who is a Marketing and Business Growth Consultant. In part 1 of our conversation, we discuss his market growth process. Show NotesConnect With:Mark Donnigan: Website // LinkedInThe MarTech Podcast: Email // Newsletter // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Today we're going to discuss how to build a go to market strategy. Joining us is Mark Donnigan, who is a Marketing and Business Growth Consultant. In part 1 of our conversation, we discuss his market growth process. Show NotesConnect With:Mark Donnigan: Website // LinkedInThe MarTech Podcast: Email // Newsletter // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

A Chief Marketing Officer is a catalyst for inspiration. They have the power to drive their organization forward with creativity and innovation. They can create an environment where employees are encouraged to develop new ideas and products.  Nowadays, a CMO is required to have more than just the average skill set. They must be able to personalize their messaging and cohesively fit it into the needs and demands of their audience. Additionally, they need to be more organized and creative in order to inspire their employees and resellers. They have to instill more motivation and leadership into their employees. They should have a clear understanding of their audience and be able to communicate with them effectively. Additionally, they should be able to identify and understand customer needs in order to drive business growth decisions. Being a high-growth CMO is a challenging role. A CMO must be able to lead their team, set and achieve goals, increase revenue, and more. So, what traits and qualities do a high-growth CMO needs to have?  In this episode, Mark Donnigan shares 5 vital traits of high-growth CMOs that you need to develop in order to become more successful.

In this episode, Mark Donnigan, the virtual CMO & Business Development Leader at Growth Stage Marketing, joins us to discuss the process of engineering go-to-market motions. Through this episode, Mark talks about: --> As a founder, how to take your new product to the market --> Why should the CMO you hire need to be a business strategist and domain expert? --> What can founders do to ‘prepare the market' before their new product is ready to go? -->Why do some companies fail to scale to the mass market - assuming there is a mass market for their product? --> Does a category define a brand, or does a brand define the category? and a lot more...

Here comes another value packed interview! For Episode 84, our host Bryan Whittington is joined by Mark Donnigan. Bryan and Mark go all in on a very hot and relevant topic these days - creating a community! You do NOT want to miss this episode! Connect with Mark: https://www.linkedin.com/in/markdonnigan/ Connect with Bryan: https://www.linkedin.com/in/brywhittington/ www.ebsgrowth.com

Before the pandemic, B2B customer engagement typically happened in person during sales calls, meetings, trade shows, demos, and more. With the shift to virtual communication, B2B companies must think about how to engage with customers and improve their experience online. In this episode, we'll hear about the importance of communities from Mark Donnigan, an expert in marketing for startups. We'll talk about how to apply those principles to enterprise businesses to connect your customers with each other and with your B2B brand. Discover the trickle-down effects of customer community, from better customer experience, satisfaction, and loyalty to drive business growth. About Mark Donnigan: Mark Donnigan is the CMO at Growth Stage Marketing. He designs and executes marketing programs and go-to-market strategies that build markets and establish disruptive innovation companies as a category king. With 20 years of experience as a transformative and strategic B2B marketing and business leader, Mark understands what's required to succeed in today's winner-takes-all market.

Happy Monday, Let's Talk Sales listeners! This week's guest is Mark Donnigan. Mark is the Virtual CMO and Business Development Leader at Growth Stage Marketing, helping founding teams structure their marketing and sales efforts and build high-impact plans so they can scale their growth. He has extensive experience running sales and marketing, with a focus on […] The post Building Credibility with Mark Donnigan appeared first on Criteria For Success.

Christopher Lochhead wrote a book called Play Bigger. He was the CMO of Mercury Interactive and a couple of other real high-flying Silicon Valley startups and just an awesome guy. But he has a saying. He likes to say, "Thinking about thinking is the most important thinking you do. A lot of what we're talking about is stepping back because it's so easy to default to, Hey, I just raised my Series A; I'm in MarTech, let's go get somebody out of drift or let's go get a sale. Let's go find someone in Salesforce marketing. Let's go get a director from Salesforce to run our marketing. Oh, they'll crush it for us. And it's a natural default, right? And then a person comes in and because they weren't clear... And usually that person, they don't know how to vet. So, they're told this and they're super excited. Oh, wow. This is my chance. Put a stamp on it, but they forget that I'm not going to be in Salesforce anymore. Meaning, I'm not going to have, first of all, the Salesforce brand and just all the power of that. I'm not going to have all the resources. ----more---- And so anyway, the questions are just so important. Our guest, Mark Donnigan spends a lot of time just going through with the CEOs and the founders he works with and just really talking it through. Listen to the rest of the interview with Mark. Part one of this interview is here: The Two-Way Interview with Mark Donnigan About Susan's Guest: Mark Donnigan designs and executes marketing programs and go-to-market strategies that build markets and establish disruptive innovation companies as a category king. With 20 years of experience as a transformative and strategic B2B marketing and business leader Mark understands what's required to succeed in today's winner takes all market. Leveraging marketing and growth tactics that work, Mark produces real business results for early and growth-stage technology and disruptive innovation startup companies. Being well versed in SaaS, software licensing, wholesale, and retail distribution models, he helps companies build nimble, highly efficient marketing teams that routinely outperform larger marketing groups. Mark is passionate about extracting the most value from every marketing dollar invested. He provides startup founders in the early stages of building their sales engine with high-impact marketing playbooks so that they can reach their revenue goals and scale sustainably.

 Find out how Category Design differentiates you from your competitors and have you as your market's top of mind Learn how to clearly define the problem you solve and communicate it Discover marketing tactics and strategies that meet your business objectives Resources/Links: Download your FREE PDF copy on How to Drive a Market: https://growthstage.marketing/category-design/ Summary Do you feel like you are putting all the marketing strategies in place and yet you are not meeting your financial objectives? Are you not having the revenue growth you thought you should be making? Do you want to find an opportunity to engage your market differently and be top of mind? Mark Donnigan designs and executes marketing programs and go-to-market strategies that build markets and establish disruptive innovation companies as a category king. Leveraging marketing and growth tactics that work, Mark produces real business results for early and growth-stage technology and disruptive innovation startup companies. In this episode, Mark shares how Category Design lets you cut through the noise and become your market's top of mind. Check out these episode highlights: 02:02 - Mark's ideal client: “My ideal clients are technical or product lead founders who are introducing technologies or deep tech-type solutions into the market. Anybody who's solving problems where, either advanced technology has been developed to address the problems, or they're bringing maybe some existing technologies into the market.” 02:51 - The problem Mark helps solve: “The problem I solve is the classic mindset of, ‘If I build it, they will come.' ‘My solution is so good.'” 03:47 - Typical symptoms that Mark's clients would be experiencing before working with them: “Most of the time they've launched, they have a product in market, they have customers, and they usually have to use the word you might say, product-market fit. The point is, depending on what your perspective is, you might think like 10 million, well, that's a pretty good start, right? The problem is you get to $10 million; you look around and say now what, I'm not seeing a clear path to get to 50 and 100 and really capture the market potential.” 05:45 - Common mistakes they make before they find Mark's solution: “The common mistakes are executing, frankly, a marketing playbook that was built for 10, 15, 20 years ago.” 07:25 - Mark's Valuable Free Action (VFA): “Clearly define the problem that you solve. And then every opportunity you have to talk about your solution, don't talk about your product, talk about the problem and the solution.” 07:52 - Mark's Valuable Free Resource (VFR): Check out Mark's Website: growthstage.marketing 08:10 - Q: Why in the world did I get into marketing? A: I'm a sales guy that started programming computers when I was 12, figured out that I wanted to be a rock star, went to music school, and so sales and marketing creativity. That's why I'm here. Tweetable Takeaways from this Episode: “Every opportunity you have to talk about your solution, don't talk about your product, talk about the problem you solve.” -Mark DonniganClick To Tweet Transcript (Note, this was transcribed using a transcription software and may not reflect the exact words used in the podcast) Tom Poland 00:09 Greetings everyone, a warm welcome to another edition of Marketing the Invisible.

 Find out how Category Design differentiates you from your competitors and have you as your market's top of mind Learn how to clearly define the problem you solve and communicate it Discover marketing tactics and strategies that meet your business objectives Resources/Links: Download your FREE PDF copy on How to Drive a Market: https://growthstage.marketing/category-design/ Summary Do you feel like you are putting all the marketing strategies in place and yet you are not meeting your financial objectives? Are you not having the revenue growth you thought you should be making? Do you want to find an opportunity to engage your market differently and be top of mind? Mark Donnigan designs and executes marketing programs and go-to-market strategies that build markets and establish disruptive innovation companies as a category king. Leveraging marketing and growth tactics that work, Mark produces real business results for early and growth-stage technology and disruptive innovation startup companies. In this episode, Mark shares how Category Design lets you cut through the noise and become your market's top of mind. Check out these episode highlights: 02:02 - Mark's ideal client: “My ideal clients are technical or product lead founders who are introducing technologies or deep tech-type solutions into the market. Anybody who's solving problems where, either advanced technology has been developed to address the problems, or they're bringing maybe some existing technologies into the market.” 02:51 - The problem Mark helps solve: “The problem I solve is the classic mindset of, ‘If I build it, they will come.' ‘My solution is so good.'” 03:47 - Typical symptoms that Mark's clients would be experiencing before working with them: “Most of the time they've launched, they have a product in market, they have customers, and they usually have to use the word you might say, product-market fit. The point is, depending on what your perspective is, you might think like 10 million, well, that's a pretty good start, right? The problem is you get to $10 million; you look around and say now what, I'm not seeing a clear path to get to 50 and 100 and really capture the market potential.” 05:45 - Common mistakes they make before they find Mark's solution: “The common mistakes are executing, frankly, a marketing playbook that was built for 10, 15, 20 years ago.” 07:25 - Mark's Valuable Free Action (VFA): “Clearly define the problem that you solve. And then every opportunity you have to talk about your solution, don't talk about your product, talk about the problem and the solution.” 07:52 - Mark's Valuable Free Resource (VFR): Check out Mark's Website: growthstage.marketing 08:10 - Q: Why in the world did I get into marketing? A: I'm a sales guy that started programming computers when I was 12, figured out that I wanted to be a rock star, went to music school, and so sales and marketing creativity. That's why I'm here. Tweetable Takeaways from this Episode: “Every opportunity you have to talk about your solution, don't talk about your product, talk about the problem you solve.” -Mark DonniganClick To Tweet Transcript (Note, this was transcribed using a transcription software and may not reflect the exact words used in the podcast) Tom Poland 00:09 Greetings everyone, a warm welcome to another edition of Marketing the Invisible. My name is Tom Poland beaming out to as always from little on the white sand next the big blue ocean Little Castaways Beach in Queensland, Australia, joined today by Mark Donnigan. Mark, good day, Sir, very warm welcome! Where are you hanging out? Mark Donnigan 0:25 Phoenix, Arizona in the desert. Phoenix, Arizona on the beach. Tom Poland 0:29 Right? Where you take your sinuses, right? Mark Donnigan 0:32

Mark Donnigan from Growth Stage Marketing says, "It's important that a professional team work together and all the executives should be rowing in the same direction and all these kind of truisms, which are all true but there are some very profound implications to how we market and how we sell, which gets wrapped up in go to market as a result of the fragmented buyer's journey that we are in today." He also reminds us, "think about how you talk about yourself, how you talk about what you do. And even as a full-time employee, in fact, I could argue that as a full-time employee, it matters as much or even more because it's so amazing when you hear one person who just describes themselves in kind of the HR job description. What do you do?" Listen to the first half of this interview to walk away walking a little bit taller, and able to go into interviews with more confidence. ----more---- About Susan's Guest: Mark Donnigan designs and executes marketing programs and go-to-market strategies that build markets and establish disruptive innovation companies as a category king. With 20 years of experience as a transformative and strategic B2B marketing and business leader Mark understands what's required to succeed in today's winner takes all market. Leveraging marketing and growth tactics that work, Mark produces real business results for early and growth-stage technology and disruptive innovation startup companies. Being well versed in SaaS, software licensing, wholesale, and retail distribution models, he helps companies build nimble, highly efficient marketing teams that routinely outperform larger marketing groups. Mark is passionate about extracting the most value from every marketing dollar invested. He provides startup founders in the early stages of building their sales engine with high-impact marketing playbooks so that they can reach their revenue goals and scale sustainably.

Chief marketing officers (CMOs) typically only stay with a company for only 24-25 months. That type of turnover at the top level of marketing departments is not good for marketers in leadership roles or with leadership aspirations.   Today's guest is Mark Donnigan, a marketing consultant. He talks about why CMOs need to think more like business strategists to better connect where marketing fits into the big picture within your organization rather than thinking about marketing as a set of tactics that are separate from what the rest of the business is doing.   Some of the highlights of the show include: Why marketing leaders need to understand the business objectives Gartner: Average buyer over 50% through buying journey before making contact MBA Playbook: Where CMOs go wrong by following a concrete buying cycle Solution: Spend time with CEO to connect with company's strategy and revenue Attribution: Avoid ROI issues by shifting from cost center to revenue perspective Pitfalls: When marketing leaders focus more on building, not understanding skills No Excuses: Marketing leaders need to be business-aware, business-oriented   Links: Mark Donnigan - Growth Stage Marketing Mark Donnigan on LinkedIn Category Design Presentation Founders Marketing Playbook Presentation Gartner Global Research and Advisory Company Play Bigger by Christopher Lochhead The Qualified Sales Leader by John McMahon HubSpot Ben Sailer on LinkedIn CoSchedule   Quotes from Mark Donnigan: “No longer is it sufficient in today's fragmented buyer journey to just basically build your whole program around a nice funnel.” “The average B2B buyer was...over 50% of the way through their buying journey before they even contacted the first vendor.” “You have the marketing tools to execute. There's no need to go to another marketing seminar, another martech seminar. Instead, spend time with the CEO.” “To be able to contribute in a sales meeting, you better know about the business.”

Welcome loyal listeners to another episode of The Loyalty Minute, I'm your host Rob Gallo and today I am excited to chat with Mark Donnigan. Mark designs and executes marketing playbooks that produce real business results for early and growth-stage technology and disruptive innovation startup companies. Welcome to the show Mark, thanks for joining me. --- Send in a voice message: https://anchor.fm/rob-gallo/message

“When you're out there shouting, and everyone else is shouting, is it a surprise that no one can hear?”   Listen & Learn: The four steps to the “flywheel” funnel. All about consensus creation. The one framework all marketers need to understand. How to boost demand-gen and topline revenue. Why marketers need to define the problem rather than talk about their product or service. Understanding the buyer journey Mark Donnigan is a startup trailblazer and a go-to-market engineer. He designs and executes marketing playbooks that produce real business results for early and growth-stage technology startup companies. He is well-versed in SaaS, software licensing, wholesale, and retail sales models, and builds nimble, highly efficient marketing teams that routinely outperform larger organizations. With his 20+ years of experience as a transformative B2B and enterprise marketing and business leader, Mark understands how to succeed in today's winner-takes-all market.   In a crowded marketplace, people-first strategies are what get our clients noticed. Call LORI JONES today to learn more. 303-678-7102.   TO FIND MARK DONNIGAN ON LINKEDIN, CLICK HERE.  

Today's guest, Mark Donnigan, designs and executes marketing playbooks that produce real business results for early and growth-stage technology and disruptive innovation startup companies. With his 20 years of experience as a transformative B2B marketing and business leader, Mark understands how to succeed in today's winner takes all market by building categories and thinking differently. On Mark's website he has a super valuable Category Creation Guide that is free and ungated! You won't get slathered with lead gen forms on my site! ----------------------- Want to learn more?  Contact HTTP://www.fanaticsmedia.com     

“The reason why companies fail is NOT because of the product or the technology. It's because they fail to understand that…they have to engineer their marketing.”   Listen & Learn: The essence of category design. Why some brands make the mistake of leading with parity. The role of marketing in demand generation. Understanding the buyer's journey. How the buyer's journey impacts the marketing funnel. Building touchpoints along the buyer's journey.   Mark Donnigan is a startup trailblazer and a go-to-market engineer. He designs and executes marketing playbooks that produce real business results for early and growth-stage technology startup companies. He is well-versed in SaaS, software licensing, wholesale, and retail sales models, and builds nimble, highly efficient marketing teams that routinely outperform larger organizations. With his 20+ years of experience as a transformative B2B and enterprise marketing and business leader, Mark understands how to succeed in today's winner-takes-all market.   Creating emotional epiphanies so your brand doesn't drown in a sea of sameness. Call LORI JONES today to learn more. 303-678-7102.   TO FIND MARK DONNIGAN ON LINKEDIN, CLICK HERE.

Mark Donnigan builds disruptive innovative startups. Mark's superpower is architecting go-to-market plans and marketing motions that drive real business outcomes.   Mark's experience spans 20 years as a transformative B2B marketing and business leader driving demand, brand development, and strategy for startup and growth-stage companies.   In today's episode, Mark and Mariya discuss how to effectively hire your first marketing person (and why they should not be from a big company).   Connect with Mark: https://growthstage.marketing

For the past year, everybody has been talking about the “new normal.” But there is also a new normal for marketing… And you either adapt or you go extinct. So says Mark Donnigan, Marketing & Business Growth Consultant at d-launch, who joined me on the podcast to go over how marketers should adapt to today's buyer's journey. What we talked about: How COVID has changed the buyer's journey Why adaptability matters more than ever Why you should focus on solving problems and the beneficiary of your solution This post includes highlights of our podcast interview with Jonathan Pogact, Mark Donnigan, Marketing & Business Growth Consultant at d-launch For the entire interview, you can listen to The B2B Revenue Executive Experience. If you don't use Apple Podcasts, we suggest this link.

For the past year, everybody has been talking about the “new normal.” But there is also a new normal for marketing… And you either adapt or you go extinct. So says Mark Donnigan, Marketing & Business Growth Consultant at d-launch, who joined me on the podcast to go over how marketers should adapt to today's buyer's journey. What we talked about: How COVID has changed the buyer's journey Why adaptability matters more than ever Why you should focus on solving problems and the beneficiary of your solution This post includes highlights of our podcast interview with Jonathan Pogact, Mark Donnigan, Marketing & Business Growth Consultant at d-launch For the entire interview, you can listen to The B2B Revenue Executive Experience. If you don't use Apple Podcasts, we suggest this link.

Mark Donnigan's experience spans 20 years as a transformative B2B and enterprise marketing and business leader driving demand, brand development, and strategy for startup, emerging, and growth-stage companies, backed by some of the largest VC firms in Silicon Valley. He has managed teams of 25 people, comprising marketing, SDR/BDRs, and account executives producing $29 million per year in revenue with P&L budget responsibility of $3 million annually, including mid-six figures for marketing investments. He builds disruptive innovative startups, with a focus on architecting go-to-market plans and marketing motions that drive real business outcomes. In this episode of the Product Science Podcast, we talk about how marketing can be a crucial component in finding product/market fit and propelling your startup to new stages of growth. We also touch on some traps organizations fall into that slow their marketing and product development down. Read the show notes to learn more: https://h2rproductscience.com/ep318-mark-donnigan

Conversation with Mark Donnigan, a marketing and growth consultant who works with startup founders to design categories and marketing playbooks that produce real business outcomes.

Today we're going to discuss how to build a go to market strategy. Joining us is Mark Donnigan, who is a Marketing and Business Growth Consultant. In part 2 of our conversation, we discuss engineering's role in a marketing go-to-market plan. Show NotesConnect With:Mark Donnigan: Website // LinkedInThe MarTech Podcast: Email // Newsletter // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Today we're going to discuss how to build a go to market strategy. Joining us is Mark Donnigan, who is a Marketing and Business Growth Consultant. In part 1 of our conversation, we discuss his market growth process. Show NotesConnect With:Mark Donnigan: Website // LinkedInThe MarTech Podcast: Email // Newsletter // TwitterBenjamin Shapiro: Website // LinkedIn // TwitterSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Hey everyone! Welcome back to another great episode of the Startup Junkies Podcast. This week, Jeff Amerine and Matthew Ward have a lively conversation with Mark Donnigan, founder of d-launch. d-launch is a marketing consulting firm Mark started in 2010 to help startup companies crush their business objectives through marketing architecture. In this episode, you will get to hear about Mark's impactful journey from sales to opening his own marketing consulting firm, his approach to picking your first marketing hire, and why there should not be a demand generation function. Tune in for these words of wisdom! Shownotes (1:01) Introducing Mark Donnigan (4:46) Go-to market engineering (10:51) The importance of a startup's first marketing hire (19:30) Demand generation marketing (23:35) The difference in B2C and B2B sales (31:00) Mark's advice for his past self   Links Caleb Talley Jeff Amerine Mark Donnigan d-launch     Quotes “The high failure rate of startups is due largely to the fact that - not because what was built didn't work, or didn't fulfill the promise...but it's because they failed to educate the market as to the problem that this invention would solve.” (5:52)   “In some ways, a lot of the B2C motions are, are, are, the tactics are different in B2B.  It's true, but at the end of the day, I'm still selling to a human.  And if that human doesn't connect with me, as another human even at the most basic relationship level, I'm not going to close the sale” (23:59)

TheNextCMO's latest podcast is with Mark Donnigan the Vice President of Marketing at Beamr. Mark is co-host of The Video Insiders podcast, where he shares up to date insights on the state of streaming technology and the business of video. Mark's journey into marketing leadership was a left turn from sales management and business development. But in today's “winner takes all” market reality we cover the importance of remote working, video marketing, category design and how essential the relationship between the CEO, CMO, and head of sales is for a business to thrive. Beamr- https://beamr.com The Video Insiders podcast- https://thevideoinsiders.com Mark Donnigan LinkedIn profile- https://www.linkedin.com/in/markdonnigan/ For more info about Plannuh, check out our website

Follow Mike on Instagram: https://www.instagram.com/michaeldavidhuey Follow Mark Donnigan on instagram: https://www.instagram.com/markdonnigan Connect with Mike: https://www.hefluence.com/ Connect with Mark Donnigan: https://www.linkedin.com/in/markdonnigan --- Send in a voice message: https://anchor.fm/michael-huey/message

Download the Innovid 2020 State of Connected TV ReportLearn about InnvoidTal Chalozin LinkedIn profileRelated episode: Direct-to-consumer streaming service launches and first impressionsListen to Episode 20 for more information on interactive advertising and video monetization technology--------------------------------------The Video Insiders LinkedIn Group is where thousands of your peers are discussing the latest video technology news and sharing best practices. Click here to joinWould you like to be a guest on the show? Email: thevideoinsiders@beamr.comLearn about Beamr--------------------------------------TRANSCRIPT: (edited lightly to improve readability)Tal Chalozin:       00:00          Innovid is what we call a video marketing platform. It's a technology platform sold to marketers, brands executives, and agencies that lets them do three things. First and foremost what is called an ad server. It's a technology that actually streams the ad to every website. So if a marketer, let's say Chrysler, or Proctor & Gamble or Best Buy, or others is advertising on YouTube or Hulu or Fox or NBC or New York Times there's a centralized platform that you can actually manage the campaign, upload the MP4's and actually do the streaming and make decisions on now on which video file to serve. So right now we're very fortunate to be the largest video ad server in the world and in many other countries in the United States and many other countries that we operate in. Tal Chalozin:       00:51          A little over a third of all video ads in the United States are being streamed by Innovid. So if you tune into every website and every app, let's say Hulu, one out of three ads, and as a matter of fact on Hulu, it's probably even higher than that. Almost one of every two ads would be, one's coming from Innovid every day. We stream roughly 450 years' worth of ads. And this is just ads content. So we stream a lot of videos. To complete the story of our platform. At a core it's an ad server. And then on top of that there are two applications. One is around creative and the other one is around measurement. Announcer:          01:31          The video insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation codec nerd and a marketing guy who knows what I-frames and macro blocks are. And here are your hosts, Mark Donnigan and Dror Gill. Dror Gill:          01:51          Today we have a very special guest and an old friend of mine Tal Chalozin who is the CTO of Innovid. Hi Tal. Welcome to The Video Insiders. Tal Chalozin:       01:59          Hello Dror. Hello Mark. Thanks for having me. It's a true honor. Mark Donnigan:      02:03          Yeah, welcome Tal. So tell us about Innovid. Tal Chalozin:       02:07          Innovid is a software company that I had the honor of starting together with my two friends and co founders, Zvika Netter our CEO and Zack Zigdon who runs all of our international business. And myself, it's a company that we started back in 2007. Before I explain what we do, just to take you back almost 13 years ago, this is the time after Google acquired YouTube and Hulu as a streaming site was kind of an inception mode. NBC and News Corp started this operation to bring streaming television into the internet. Tal Chalozin:       02:49          And what we said back then is that we believe that the future of television is over IP and to be streamed. We thought that when this would happen the one thing that we really want to tackle is the viewing experience around the advertising. Because it was clear that marketers and ad dollars take a very, very important part of the experience of television subsidizing content and creating the access to so many different people. But it's also clear that sitting through a pretty boring 30 second spot and that every person around the United States in a broadcast time window would see the exact same ad. It's kind of silly. And so we went on a journey to build a software that helps to create a better viewing experience around commercials. Tal Chalozin:       03:44          So we started with the technology, with technology that allows what is called in kind of layman terms virtual product placement. It was a computer vision technology that lets you process videos and reconstruct the 3D. So understanding occlusions and backgrounds and foregrounds and planes and allow you to render a product a 3D product in 3D images into the shot. And it looks like as if it was there while the content was shot while reproducing all the shades and lighting and again, occlusion and, and things like that. This was where we started. We got a bunch of patents. This is how we raised our A round back then. We got so many awards. It was awesome. But then what we learned is that it's amazing, but advertising is a business of scale for marketers to actually play. Tal Chalozin:       04:38          One of the main things that marketers gain out of television is a massive megaphone that lets you tell your story to millions, if not hundreds of millions of people in 30 seconds. So then we went on a journey to better learn this business and expanded more and more capability and fast forward to today. Innovid is what we call a video marketing platform. It's a technology platform sold to marketers, brands executives and agencies that lets them do three things. First and foremost what is called an ad server. It's a technology that actually streams the ad to every website. So if a marketer, let's say Chrysler or Procter and Gamble or Best Buy or others is advertising on YouTube or Hulu or Fox or NBC or New York times there's a centralized platform that you can actually manage the campaign, upload the MP4's and actually do the streaming and make decisions on which video file to serve to the individual that is streaming the content. Tal Chalozin:       05:48          So right now we're very fortunate to be the largest video ad server in the world. And in many other countries in the United States, many other countries that we operate in a little over a third of all video ads in the United States are being streamed by Innovid. At a core it's an ad server. And then on top of that, there are two applications. One is around creative and the other one is around measurement. Our headquarters in New York. There's 350 people, a big R&D center in Israel and then offices across the U S and in Europe. And in APAC. If you read the trades, it seems like the future of television has no ads. Disney Plus, Netflix, Amazon, Apple, all of the big services that made a lot of splash in the press toot the horn of no ads. Tal Chalozin:       06:43          This is very nice for marketing, but in reality advertising dollars pays the bills that makes so many pieces of content to be streamed. The subscription services could not really thrive on subscription alone, let alone when you're talking about a massive global service that would like to reach hundreds of millions of subscribers. You cannot do that only with subscribing. With subscription dollars or advertising is a very strong market and in the future will be that. Easy testament is that just last week NBC launched or Comcast launched there foray into that game called Peacock. And the main thing that they said is that, Hey there's so much noise around advertising, about no ads. This cannot work. We will include ads. Tal Chalozin:       07:36          And this makes to the second part of what I wanted to say about the future is that, but they put a lot of emphasis around ed experience. So it's not that you will see ads in the same way that you're used to watching television. There will still be ad breaks, but it will look and feel very, very different than what it used to be on television. And we play a very big role there and in other places. And we think that yes, the future of television is over the internet, over IP. The future of television is with ads, or at least in some capacity of it, but it would look and feel much different. Dror Gill:          08:14          I want to ask a question regarding the, the ad server component. And these ads go interleaved into content experiences sometimes before or after or during the actual streaming of the content. So how do you match the resolution and the quality of the ad that you provide to the actual content that is being streamed? Because I don't assume that somebody watching a 4K movie would like to be interrupted by like an, you know, an SD, low quality ad. It would probably be quite annoying. Tal Chalozin:       08:52          I have so many things to say about this stuff. First of all, before I answer exactly how we did it I can tell you that people think that the internet is so advanced in 2020 so all of this problem is practically solved. And there is no real problem to bring television over the internet and it's not really true. I'm sure you know you know, very well the general standard in the video ads industry right now is that we as the server that generated the file and hosts them, would create an XML template called vast V A S T and put multiple video renditions in a file and create a manifest that would have different renditions of and actually different encoders as well. Tal Chalozin:       09:44          Of the file. It used to be, we used to put FLV and other stuff. But right now it's all MP4 containers. But anyway, you put multiple renditions and then the actual player picks the right one and the player, essentially what it's doing is doing playlisting. So picking the right ad at the right time and there is a, in the last, the last few years, but honestly, just in the last year, there is a big change in the way video ads are being streamed. Moving from what used to be called CSAI client side ad insertion, AKA playlisting. So on the client you download some, some type of playlisting and then you just move between different files even if it's the main content - it doesn't matter the rendition, you would still switch between different files that you do progressive downloads for. Tal Chalozin:       10:45          Most of the very large sites and today apps are what is called SSAI server-side ad insertion. Essentially it doesn't matter what file we bring. You convert it into an HLS stream, create TS files, and then do kind of the, the term that everyone is using is manifest manipulation. So just manipulate the M3u8 and swap packets, TS files inside the M3u8. I hope that I don't need to explain everything that I'm just saying, but stop me if you want me to. So essentially let's say on Hulu, this is how it works. You will tune into a stream and you hit play on an episode of a, I don't know, The Good Wife on Hulu. What they will do, they will go, let's say this is 48 minutes of an episode or 21 minutes of an episode with multiple ads that need to be weaved throughout. So what they will do, they will do a server side call to all the different ads and then get either an MP4 and do just in time transcoding for it. Or, if it's pre-prepared, like a lot of the things that we do you would get the actual TS file and then just merge it into a single M3u8 with content TS files in the right rendition and the ads. Mark Donnigan:      12:09          So Tal, are you actually able to get the, you know, I'll call it the mezzanine file of the ad, and then you can create a high quality or at least the highest quality possible for the, you know, target resolution and bit rate or are you limited by the fact that sometimes, you know, you may get a mezzanine quality and other times it may just be a 1080p in which case Dror's example of like a 4K. You're just limited. I mean, you have the quality you have. So can you tell us, shed some light on that? Tal Chalozin:       12:43          It's a fascinating point. This is an uphill battle for us because we are, we're still an intermediary. We're not the post production shop at that makes the video file, so we're limited to whatever you would get. So yeah, the intention is to get a Pro Res or a mez file, mezzanine file, of the ad that allows us to do transcoding into whatever we want. But, that's not the reality all the time. In many cases we would get to your example, a 1080p is a good case. In some cases we get 720 and sometimes we even need to up convert it, which clearly is not really working. Tal Chalozin:       13:34          And the reality is that the 4K streaming of ad supported content is not a real thing as of right now. But, 1080p is definitely one that is. And again, we're in 2020 right now and you can open whatever app without naming names, but you can open one of the biggest apps out there and I'm sure you would get to an ad break and even an unaided eye can see that it's a totally different rendition of the ad, even different audio, let alone volume normalization. But even just the quality of the encoding is significantly different or lower than the actual content. And this is a common case or the state of the internet right now. Dror Gill:          14:24          But this is something you're trying to avoid? Tal Chalozin:       14:26          We're definitely trying to avoid the way that we're doing it is that if you think about it, there are two inputs to our system. One is the ad itself, literally, again the mez file, Pro Res, whatever container that is, an MP4. And then, what is called in ad terms a media plan. Media plan is saying that we are Chrysler, the campaign starts in this date and ends on this date, there is X number of million impressions on YouTube, then on Snapchat, then on Hulu, and then the full list. It's a very complicated meta data of the whole campaign. So those are the two inputs that we're getting. Historically that was just an upload. So in our system, you would go and just upload the files. Tal Chalozin:       15:13          More and more we're trying to get down to the source and create some type of an integration with the, with the DAM, the digital asset manager. Let's say, again, this is a Chrysler commercial, Chrysler 300 commercial. Someone actually did the post for it, and they do have the approved asset at the best quality possible. But those are not our customers. So sometimes we don't get access to that and we need to beg the customer to get that and try to explain what's the outcome if they don't get it. So what we're trying to do is to get down to the source as close as possible. So then that post-production shop would actually have an API to us, or even if they upload, they would upload the source and not have a downsample of it. Mark Donnigan:      16:05          So our audience, are largely encoding engineers, video engineers, and we just hear over and over again incredible frustration about this. Dror and I were just talking to a very large live sports streaming service last week and the person responsible for encoding was lamenting that whenever there's issues with quality, it's because he can't do any better. It's a source issue! The high quality asset exists. Why can't we get access to it so that we can provide an incredible advertising experience. And I'm just wondering, how do we fix this? Tal Chalozin:       16:50          How do we fix that? As more hours per day continues to pour into the connected, let's call it the connected television space, and as more and more ad dollars flow in there, and then more and more people cut their cord or shave their cord or are cord nevers and haven't even been exposed to traditional television, this becomes the norm and not the new thing. It's essentially a supply chain or a workflow problem because as you said, the file is there. It's not that someone is shooting on an SD camera and now you, you're stuck with a shitty file. People are using RED cameras to shoot it. So yeah, so it's more of a workflow problem. And this is what we set out to do is to just remove the clutter and connect everything in an industry that wasn't connected. Ads on television, still are being delivered predominantly through FedEx with cassette tapes that are being sent to local TV stations. Tal Chalozin:       17:50          This is still a thing. We're moving from this world and now talking about getting a mezzanine or 4K file. I'll tell you about one thing that I'm very keen on, is that another thing is getting the raw asset is one thing. And then another thing, if you look at it, there's multiple parties on the internet that are getting an asset and transcoding it. So let's say that we get the video file. Probably Facebook got the video file as well, maybe not through Innovid. And they also transcoded the video file and then YouTube or Tik Tok got the video file somehow. And then sometimes clients would use Innovid. Sometimes you would go directly into YouTube and upload the raw file. And maybe NBC would get it through some other distribution channel to the broadcast side. Tal Chalozin:       18:44          And then when they run it online, they would take the broadcast file and transcode it as well. So there was multiple people or organization that got the raw footage and then they're in charge of transcoding. This is pretty stupid. It should be some type of a centralized repository because there is an ID to every file and there is an initiative called the Ad ID to make sure that there will be a unified numbering system, and a catalog. And by virtue of that, meta data and tracking just in the ad space so in every ad and then not only did you have a catalog, you can access all different resolutions in a centralized place. So then if YouTube wants a a downsampled version, then you just pick the resolution you want. You don't take the raw and then encode it as well. Tal Chalozin:       19:32          There's an initiative. There are several companies trying to do that. It's kind of a hurding cats type of an initiative. But it's almost a necessity because unless you do that, you will always have those artifacts. Mark Donnigan:      19:46          Yeah, that's right. And that Ad ID in your experience does that travel, I'll use the word seamlessly, you know, between these various systems or is that even an issue of keeping that ad ID intact? Tal Chalozin:       19:59          You know that it is a meta data but in reality again, we are one of the largest platforms that actually accesses files and stream them out and encode them. Most people that do encoding do not carry on all the meta data. That's one thing. Second thing is that most people, actually, most platforms don't even look into that meta data. So don't even expose that or do anything with it. Tal Chalozin:       20:22          Several encoders do not put it in there. So right now, yes, it is there, but it's not fully available. So the solution that is used mostly right now, which you would laugh, is putting it in the actual file name. So literally as an unstructured text on the file before the dot and before you put an underscore and then the the actual file, which clearly doesn't carry through anywhere. So that's the reality again, right now in 2020. It's almost like Dror do you remember Yossi Vardi's example of pigeons carrying DVDs in order to transfer a lot of files, large files? Dror Gill:          21:04          He also did another experiment. He took a snail and he stuck a USB drive on the back of the snail. And then he had two computers connected with a crossed ethernet cable and he was trying to see how the data will go faster through the cable or the snail that is moving slowly between the computers with the USB drive on his back. And I'm sorry to say, but the snail won! Tal Chalozin:       21:28          The industry from the outside seems like, again, it all problems are solved, but it's far from it. You know, the Superbowl is coming up very soon and Fox is going to air the Superbowl and like every year you can access it in streaming as well. And it's still a discussion every year. Is the internet already for that? The term for ad serving in real time in the world of television is called DAI dynamic ad insertion. Every broadcaster that gets the right to stream the Superbowl is asking, are we ready or are we safe to do DAI for the ads or to play it safe are we gonna take the broadcast feed and then just retransmit? I Can tell you a funny story, that last year we did a really cool experiment. Tal Chalozin:       22:19          CBS had the rights for the Superbowl and they use a system that takes the SCTE tone and converts it into an ID3 tag for digital systems. And then on the ID3 we put the marker of the ads, we put the actual Innovid URL of the the ad that is about to play. Originally the system was architected for measurement. So you can do measurements from the client side. So there is something on the client side, gate the ID3 tag and then fire that just do an HTTP get call that URL in order to track track the ads from the client in the most accurate way. But then what we did last year together with CBS is add the ability to also run overlays on top of the video. Tal Chalozin:       23:09          So that URL was not just for measurement, but also downloaded graphics to be displayed as a kind of, as a transparent layer on top of that on the device itself. So if you stream live stream. This is not VOD or anything like that. You do live stream of the Superbowl. Last year many devices on CBS Sports had a small SDK that again, took the SCTE tone converted to an ID3 tag, get a URL for a PNG file or whatever that is rendered in near real time. And then every house on the United States gets something else. We did an experiment together with Pringles. The whole commercial was some type of a game with Pringles. So you would get a message that is tailored to you. Tal Chalozin:       24:00          So, it literally featured the name of your city on it. And then it allows it to use your remote, let's say Apple TV. You can use your remote to left and right to swipe and play some, some kind of a funky game as the ad was playing. So funny thing again, this is 2019. You would imagine that we would have that technology available. This is not rocket science. We're talking about a lot more advanced things on the internet. But even that was super revolutionary and this year this capability will not be available because the way that Fox works is different. But that count is super cutting edge. Mark Donnigan:      24:40          Now Tal, I know that you're working very closely with Roku, so why don't you share with us what you're doing with them. Mark Donnigan:      24:49          Share what you can and tell us about what's happening on the Roku platform because I think that's very important to all of us in, in streaming media streaming video. Tal Chalozin:       25:00          Roku is a streaming device. It is divided into two parts of their platform. One is a device a streaming stick and streaming box. But, Roku first and foremost is an operating system that runs on that device or licensed to TV manufacturers, to TV OEMs. And right now there's eleven OEMs that carries that. Anything from TCL, or Insignia, all the way to LG, and on some SKUs from Sharp as well. And by numbers, Roku is the largest television operating system right now in the United States. The most amount of TV's purchased in 2019 was Roku powered or TVs or streams were powered by Roku. Tal Chalozin:       25:47          So this is larger than Amazon Fire, way way larger than Apple TV or Xbox or PlayStation or whatnot. So this is, this is Roku. Back in the early, early days of Roku this dating back to, to 2014 or 15, we did the first advertising oriented deal with Roku to create a small library and SDK that would be part of their firmware that many years later, the name is Roku ad framework, or RAF. Which is a set of libraries that lets app developers, Roku app developers get access to to stuff they need to run ads inside the app without a lot of work that allows us to create a technology for like, for example, interactive television, something that can be done in a very scalable way because now every app on Roku has the ability to render ads that can have overlays. Tal Chalozin:       26:47          You can press the remote and you can purchase things or send things to your phone or whatever activity you would like. So this is the first thing we've done with Roku and enabled that technology at a mass scale. This is many, many years before Roku was a big success. But at the end of last year, in September we, together with Roku, we announced kind of the second, second act of the innovation on the future of television, which is around measurement. I mentioned at the beginning, the top of the, of the show that we have three parts to our platform, the ad server, which we talked a lot about, different tools around creative. And the third one would be measurement capabilities. On the measurement side this is an area that the television industry, we talked a lot about things that require innovation. Tal Chalozin:       27:41          Measurement is maybe at the top of the list cause right now measurement on television is dominated by a company called Nielsen, which I'm sure many people know that the way they measure television because of lack of connectivity is by putting a people meter or a device in people's home. In very, very few households in the United States that act as a sample or as a panel which presumably should represent every household in the United States. So there's roughly 20,000 families in the United States that represent the television ecosystem, which there is north of 100 million households in the United States. And maybe 80 or 90 million households that are watching broadcast television and they're being paneled by 20,000 that essentially measuring what do people actually watch. Tal Chalozin:       28:42          So, we want to change that. We, and many other important, an important point is that many other companies are, are at it. Because, it's obvious this needs to be changed. But we teamed up with Roku that every one of the devices that carries their operating system, so every one of those TVs that have Roku as an operating system have a small chip called ACR. Stands for automatic content recognition that essentially knows what you're watching. So it records everything that hits the glass. And it doesn't matter if it hits the glass because it's an app on the Roku platform, let's say Hulu or YouTube or Netflix, or you plugged in via HDMI, your set top box or you plugged in an antenna to to the TV or even you have a DVR or VHS plugged into your television. Tal Chalozin:       29:32          Doesn't matter if it's rendered on the screen, then Roku would know what it is. They do a second by second or almost a frame by frame to a catalog. And then know what exactly you're watching and at what time code. We can talk about privacy as well, which is a very important part of it. But this is all opted in. You don't have to contribute this data, but most people do. And then we get this data. We don't care about the individual household, but we can use that as you don't, you don't need a panel anymore where every television is telling you what exactly you're watching. So we are, we're on a mission to reinvent that television measurement in a much better way. Dror Gill:          30:15          That's really amazing. So the television is actually watching what you are watching. Even if it's not streamed through that Roku platform, it's watching everything that is projected to the screen and not only you know, like recording the pixels or they're actually using this automatic content recognition system. Analyzing and knowing what content, what piece of content this is, whether it's a live broadcast or a video on demand. It could be a DVD or a VHS, time shifted or it's an ad. Exactly. Mark Donnigan:      30:51          Where is that fingerprint happening Tal? Tal Chalozin:       31:01          And by the way, a disclaimer, I don't work for Roku and I don't know any internal data about Roku. We have a strong partnership with them. So Roku is unique technology. And by the way, other TV manufacturers are doing the same thing. This is not limited to Roku. Vizio who made a lot of noise around that as well. And many others, Sony and Toshiba and others. Are using similar technologies. What's on the device is mainly picking up multiple pixels, hashing it together and sending it to the cloud. The matching to the catalog is not happening on the device. There's clearly no need for that. And there are several companies that create this catalog and does essentially the pattern matching between the set of temporal data of that set of frames, consecutive frames to a catalog to know exactly what you're watching. Tal Chalozin:       31:55          Is it - what show? What episode? Is it an ad? So one thing is to know the catalog. The other one is to know what is on right now in every... It's a very complicated problem, because sometimes you are you, you may be watching it live. Again, tuning into, I dunno, ABC, but right now because that show is a local show, you would watch it streamed by the Kansas city, Missouri ABC affiliate and it's not a national show. So you can't really match it to a catalog and know is it live or not live? And then when it comes to ads, it gets even more complicated because some of the ads are inserted in real time. So you need to know that that ad is inserted in real time so then it doesn't impact the idea of the stream. You didn't really change the channel. It's just dynamic insertion. Dror Gill:          32:48          So doing all of this measurement, I think it probably puts a lot of responsibility on your part of the value chain on the software that you create, on the reports that you generate. Because based on this I guess is how the content providers get paid right. For showing those ads, as you said. Tal Chalozin:       33:12          We are what is called the system of record for billing. So I mentioned that roughly a third of the ads are being transacted by us. This is a very rough number because the dollars don't go through us. We're just creating the billing. We are the actual counter of something like $5 billion of of ad dollars. So again, YouTube and Snapchat and New York times and NBC and Fox and TubiTV and many other channels and apps are being paid based on our numbers. And in order for that, we need to do a lot of filtration, detecting what is fraud, and making sure there's no false positives, and and many other things like that. And for it, we go through an audit process. So Ernst & Young is the auditor and there's an organization called the Media Rating Council that we go through an audit every year to make sure that what we say we do, we actually do. Tal Chalozin:       34:12          And there's no there's no problems in the counting. And yeah, it happens all the time that we are counting, but also clearly broadcasters or apps would count for their own use as well. And sometimes, unfortunately, the numbers are not the same. So we would say that P&G ran 10 million ads and the broadcaster, NBC, Discovery, what have you, would say that actually it's 10 and a half million ads. So then they need to get paid more. But the way that the contract is written is that Innovid numbers because we're unbiased is what is what will dictate the payment. So you're like the gold standard in measurements. But it's a very interesting, a very interesting world. Tal Chalozin:       35:08          It's an ever changing world. So counting ads 10 years ago and counting ads today is a very, very different business. Mark Donnigan:      35:14          There's a lot of studies and I think you even have one that you can cite if you'd like to that say very clearly that consumers are not opposed to ads. This whole notion that people "hate ads" is actually not true. What they hate is a bad or an irrelevant experience. If the platform happens to know that I'm looking for a new car and I get served a great car ad, guess what? And especially if it piques my interest, that's actually a good experience. Tal Chalozin:       35:48          100%. Yeah. We always use exactly the same term that you mentioned. People don't hate ads, they just hate bad ads. And that's absolutely true. And when you ask people, when you again, when you read the trades, it looks like ads are a very gloomy thing. Tal Chalozin:       36:06          And then you go to platforms like, in my mind, Instagram is the best ad experience ever made. When you see ads on Instagram, it's significantly better. And it's not disruptive at all. You have your thumb there and you can continue scrolling. And then many, many people choose to actually watch that. So completely reverse model. It's not that I'm forced to watch the ad. I literally can continue scrolling the same way that I'm scrolling there. But people literally are choosing to watch that because it's good ads. Mark Donnigan:      36:43          This has been a really amazing discussion and you know we have to do a part two. Yeah, there are few issues we did not cover and we must cover them and it's really been fascinating. Yeah, absolutely. Thanks for joining us Tal. Tal Chalozin:       36:57          I'd love to, thank you so much. Thanks, Mark. Thanks Dror. Thanks everyone that listened. Thanks Beamr. Announcer:          37:04          Thank you for listening to The Video Insiders podcast, a production of Beamr limited. To begin using Beamr's codecs today. Go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Click to watch SPIE Future Video Codec Panel DiscussionRelated episode with Gary Sullivan at Microsoft: VVC, HEVC & other MPEG codec standardsInterview with MPEG Chairman Leonardo Charliogne: MPEG Through the Eyes of it's ChairmanLearn about FastDVO herePankaj Topiwala LinkedIn profile--------------------------------------The Video Insiders LinkedIn Group is where thousands of your peers are discussing the latest video technology news and sharing best practices. Click here to joinWould you like to be a guest on the show? Email: thevideoinsiders@beamr.comLearn more about Beamr--------------------------------------TRANSCRIPT:Pankaj Topiwala: 00:00 With H.264 H.265 HEVC in 2013, we were now able to do up to 300 to one to up to 500 to one compression on a, let's say a 4K video. And with VVC we have truly entered a new realm where we can do up to 1000 to one compression, which is three full orders of magnitude reduction of the original size. If the original size is say 10 gigabits, we can bring that down to 10 megabits. And that's unbelievable. And so video compression truly is a remarkable technology and you know, it's a, it's a marval to look at Announcer: 00:39 The Video Insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation codec nerd and a marketing guy who knows what I-frames and macro blocks are. And here are your hosts, Mark Donnigan and Dror Gill. Speaker 3: 00:39 Dror Gill: 01:11 Today we're going to talk with one of the key figures in the development of a video codecs and a true video insider Pankaj Topiwala. Hello Pankaj and welcome to The Video Insiders podcast. Pankaj Topiwala: 01:24 Gentlemen. hello, and thank you very much for this invite. It looks like it's going to be a lot of fun. Mark Donnigan: 01:31 It is. Thank you for joining Pankaj. Dror Gill: 01:33 Yeah, it sure will be a lot of fun. So can you start by telling us a little bit about your experience in codec development? Pankaj Topiwala: 01:41 Sure, so, I should say that unlike a number of the other people that you have interviewed or may interview my background is fair bit different. I really came into this field really by a back door and almost by chance my degree PhD degree is actually in mathematical physics from 1985. And I actually have no engineering, computer science or even management experience. So naturally I run a small research company working in video compression and analytics, and that makes sense, but that's just the way things go in the modern world. But that the effect for me was a, and the entry point was that even though I was working in very, very abstract mathematics I decided to leave. I worked in academia for a few years and then I decided to join industry. And at that point they were putting me into applied mathematical research. Pankaj Topiwala: 02:44 And the topic at that time that was really hot in applied mathematics was a topic of wavelets. And I ended up writing and edited a book called wavelet image and video compression in 1998. Which was a lot of fun along with quite a few other co authors on that book. But, wavelets had its biggest contribution in the compression of image and video. And so that led me finally to enter into, and I noticed that video compression was a far larger field than image compression. I mean, by many orders, by orders of magnitude. It is probably a hundred times bigger in terms of market size than, than image compression. And as a result I said, okay, if the sexiest application of this new fangled mathematics could be in video compression I entered that field roughly with the the book that I mentioned in 1998. Mark Donnigan: 03:47 So one thing that I noticed Pankaj cause it's really interesting is your, your initial writing and you know, research was around wavelet compression and yet you have been very active in ISO MPEG, all block-based codecs. So, so tell us about that? Pankaj Topiwala: 04:08 Okay. Well obviously you know when you make the transition from working on the wavelets and our initial starting point was in doing wavelet based video compression. When I started first founded my company fastVDO in 1998, 1999 period we were working on wavelet based video compression and we, we pushed that about as much as we could. And at that, at one point we had what we felt was the world's best a video compression using wavelets in fact, but best overall. And it had the feature that you know, one thing that we should, we should tell your view or reader listeners is that the, the value of wavelets in particular in image coding is that not only can you do state of the art image coding, but you can make the bitstream what is called embedded, meaning you can chop it off at anywhere you like, and it's still a decodable stream. Pankaj Topiwala: 05:11 And in fact it is the best quality you can get for that bit rate. And that is a powerful, powerful thing you can do in image coding. Now in video, there is actually no way to do that. Video is just so much more complicated, but we did the best we could to make it not embedded, but at least scalable. And we, we built a scalable wavelet based video codec, which at that time was beating at the current implementations of MPEG4. So we were very excited that we could launch a company based on a proprietary codec that was based on this new fangled mathematics called wavelets. And lead us to a state of the art codec. The facts of the ground though is that just within the first couple of years of running our company, we found that in fact the block-based transformed codecs that everybody else was using, including the implementers of MPEG4. Pankaj Topiwala: 06:17 And then later AVC, those quickly surpassed anything we could build with with wavelets in terms of both quality and stability. The wavelet based codecs were not as powerful or as stable. And I can say quite a bit more about why that's true. If you want? Dror Gill: 06:38 So when you talk about stability, what exactly are you referring to in, in a video codec? Pankaj Topiwala: 06:42 Right. So let's let's take our listeners back a bit to compare image coding and video coding. Image coding is basically, you're given a set of pixels in a rectangular array and we normally divide that into blocks of sub blocks of that image. And then do transforms and then quantization and than entropy coding, that's how we typically do image coding. With the wavelet transform, we have a global transform. It's a, it's ideally done on the entire image. Pankaj Topiwala: 07:17 And then you could do it multiple times, what are called multiple scales of the wavelet transform. So you could take various sub sub blocks that you create by doing the wavelet transfer and the low pass high pass. Ancs do that again to the low low pass for multiple scales, typically about four or five scales that are used in popular image codecs that use wavelets. But now in video, the novelty is that you don't have one frame. You have many, many frames, hundreds or thousands or more. And you have motion. Now, motion is something where you have pieces of the image that float around from one frame to another and they float randomly. That is, it's not as if all of the motion is in one direction. Some things move one way, some things move other ways, some things actually change orientations. Pankaj Topiwala: 08:12 And they really move, of course, in three dimensional space, not in our two dimensional space that we capture. That complicates video compression enormously over image compression. And it particularly complicates all the wavelet methods to do video compression. So, wavelet methods that try to deal with motion were not very successful. The best we tried to do was using motion compensated video you know, transformed. So doing wavelet transforms in the time domain as well as the spatial domain along the paths of motion vectors. But that was not very successful. And what I mean by stability is that as soon as you increase the motion, the codec breaks, whereas in video coding using block-based transforms and block-based motion estimation and compensation it doesn't break. It just degrades much more gracefully. Wavelet based codecs do not degrade gracefully in that regard. Pankaj Topiwala: 09:16 And so we of course, as a company we decided, well, if those are the facts on the ground. We're going to go with whichever way video coding is going and drop our initial entry point, namely wavelets, and go with the DCT. Now one important thing we found was that even in the DCT- ideas we learned in wavelets can be applied right to the DCT. And I don't know if you're familiar with this part of the story, but a wavelet transform can be decomposed using bits shifts and ads only using something called the lifting transform, at least a important wavelet transforms can. Now, it turns out that the DCT can also be decomposed using lifting transforms using only bit shifts and ads. And that is something that my company developed way back back in 1998 actually. Pankaj Topiwala: 10:18 And we showed that not only for DCT, but a large class of transforms called lab transforms, which included the block transforms, but in particular included more powerful transforms the importance of that in the story of video coding. Is that up until H.264, all the video codec. So H.261, MPEG-1, MPEG-2, all these video codecs used a floating point implementation of the discrete cosign transform and without requiring anybody to implement you know a full floating point transform to a very large number of decimal places. What they required then was a minimum accuracy to the DCT and that became something that all codecs had to do. Instead. If you had an implementation of the DCT, it had to be accurate to the true floating point DCT up to a certain decimal point in, in the transform accuracy. Pankaj Topiwala: 11:27 With the advent of H.264, with H.264, we decided right away that we were not going to do a flooding point transform. We were going to do an integer transform. That decision was made even before I joined, my company joined, the development base, H.264, AVC, But they were using 32 point transforms. We found that we could introduce 16 point transforms, half the complexity. And half the complexity only in the linear dimension when you, when you think of it as a spatial dimension. So two spatial dimensions, it's a, it's actually grows more. And so the reduction in complexity is not a factor of two, but at least a factor of four and much more than that. In fact, it's a little closer to exponential. The reality is that we were able to bring the H.264 codec. Pankaj Topiwala: 12:20 So in fact, the transform was the most complicated part of the entire codec. So if you had a 32 point transform, the entire codec was at 32 point technology and it needed 32 points, 32 bits at every sample to process in hardware or software. By changing the transform to 16 bits, we were able to bring the entire codec to a 16 bit implementation, which dramatically improved the hardware implementability of this transfer of this entire codec without at all effecting the quality. So that was an important development that happened with AVC. And since then, we've been working with only integer transforms. Mark Donnigan: 13:03 This technical history is a really amazing to hear. I, I didn't actually know that Dror or you, you probably knew that, but I didn't. Dror Gill: 13:13 Yeah, I mean, I knew about the transform and shifting from fixed point, from a floating point to integer transform. But you know, I didn't know that's an incredible contribution Pankaj. Pankaj Topiwala: 13:27 We like to say that we've saved the world billions of dollars in hardware implementations. And we've taken a small a small you know, a donation as a result of that to survive as a small company. Dror Gill: 13:40 Yeah, that's great. And then from AVC you moved on and you continued your involvement in, in the other standards, right? That's followed. Pankaj Topiwala: 13:47 in fact, we've been involved in standardization efforts now for almost 20 years. My first meeting was a, I recall in may of 2000, I went to a an MPEG meeting in Geneva. And then shortly after that in July I went to an ITU VCEG meeting. VCEG is the video coding experts group of the ITU. And MPEG is the moving picture experts group of ISO. These two organizations were separately pursuing their own codecs at that time. Pankaj Topiwala: 14:21 ISO MPEG was working on MPEG-4 and ITU VCEG was working on H.263, and 263 plus and 263 plus plus. And then finally they started a project called 263 L for longterm. And eventually it became clear to these two organizations that look, it's silly to work on, on separate codecs. They had worked once before in MPEG-2 develop a joint standard and they decided to, to form a joint team at that time called the joint video team, JVT to develop the H.264 AVC video codec, which was finally done in 2003. We participate participated you know fully in that making many contributions of course in the transform but also in motion estimation and other aspects. So, for example, it might not be known that we also contributed the fast motion estimation that's now widely used in probably nearly all implementations of 264, but in 265 HEVC as well. Pankaj Topiwala: 15:38 And we participated in VVC. But one of the important things that we can discuss is these technologies, although they all have the same overall structure, they have become much more complicated in terms of the processing that they do. And we can discuss that to some extent if you want? Dror Gill: 15:59 The compression factors, just keep increasing from generation to generation and you know, we're wondering what's the limit of that? Pankaj Topiwala: 16:07 That's of course a very good question and let me try to answer some of that. And in fact that discussion I don't think came up in the discussion you had with Gary Sullivan, which certainly could have but I don't recall it in that conversation. So let me try to give for your listeners who did not catch that or are not familiar with it. A little bit of the story. Pankaj Topiwala: 16:28 The first international standard was the ITU. H.261 standard dating roughly to 1988 and it was designed to do only about 15 to one to 20 to one compression. And it was used mainly for video conferencing. And at that time you'd be surprised from our point of view today, the size of the video being used was actually incredibly tiny about QCIP or 176 by 144 pixels. Video of that quality that was the best we could conceive. And we thought we were doing great. And doing 20 to one compression, wow! Recall by the way, that if you try to do a lossless compression of any natural signal, whether it's speech or audio or images or video you can't do better than about two to one or at most about two and a half to one. Pankaj Topiwala: 17:25 You cannot do, typically you cannot even do three to one and you definitely cannot do 10 to one. So a video codec that could do 20 to one compression was 10 times better than what you could do lossless, I'm sorry. So this is definitely lossy, but lossy with still a good quality so that you can use it. And so we thought we were really good. When MPEG-1 came along in, in roughly 1992 we were aiming for 25 to one compression and the application was the video compact disc, the VCD. With H.262 or MPEG-2 roughly 1994, we were looking to do about 35 to one compression, 30 to 35. And the main application was then DVD or also broadcast television. At that point, broadcast television was ready to use at least in some, some segments. Pankaj Topiwala: 18:21 Try digital broadcasting. In the United States, that took a while. But in any case it could be used for broadcast television. And then from that point H.264 AVC In 2003, we jumped right away to more than 100 to one compression. This technology at least on large format video can be used to shrink the original size of a video by more than two orders of magnitude, which was absolutely stunning. You know no other natural signal, not speech, not broadband, audio, not images could be compressed that much and still give you high quality subjective quality. But video can because it's it is so redundant. And because we don't understand fully yet how to appreciate video. Subjectively. We've been trying things you know, ad hoc. And so the entire development of video coding has been really by ad hoc methods to see what quality we can get. Pankaj Topiwala: 19:27 And by quality we been using two two metrics. One is simply a mean square error based metric called peak signal to noise ratio or PSNR. And that has been the industry standard for the last 35 years. But the other method is simply to have people look at the video, what we call subjective rating of the video. Now it's hard to get a subjective rating. That's reliable. You have to do a lot of standardization get a lot of different people and take mean opinion scores and things like that. That's expensive. Whereas PSNR is something you can calculate on a computer. And so people have mostly in the development of video coding for 35 years relied on one objective quality metric called PSNR. And it is good but not great. And it's been known right from the beginning that it was not perfect, not perfectly correlated to video quality, and yet we didn't have anything better anyway. Pankaj Topiwala: 20:32 To finish the story of the video codecs with H.265 HEVC in 2013, we were now able to do up to 300 to one to up to 500 to one compression on let's say a 4K. And with VVC we have truly entered a new realm where we can do up to 1000 to one compression, which is three full orders of magnitude reduction of the original size. If the original size is say, 10 gigabits, we can bring that down to 10 megabits. And that's unbelievable. And so video compression truly is a remarkable technology. And you know, it's a, it's a marvel to look at. Of course it does not, it's not magic. It comes with an awful lot of processing and an awful lot of smarts have gone into it. That's right. Mark Donnigan: 21:24 You know Pankaj, that, is an amazing overview and to hear that that VVC is going to be a thousand to one. You know, compression benefit. Wow. That's incredible! Pankaj Topiwala: 21:37 I think we should of course we should of course temper that with you know, what people will use in applications. Correct. They may not use the full power of a VVC and may not crank it to that level. Sure, sure. I can certainly tell you that that we and many other companies have created bitstreams with 1000 to one or more compression and seeing video quality that we thought was usable. Mark Donnigan: 22:07 One of the topics that has come to light recently and been talked about quite a bit. And it was initially raised by Dave Ronca who used to lead encoding at Netflix for like 10 years. In fact you know, I think he really built that department, the encoding team there and is now at Facebook. And he wrote a LinkedIn article post that was really fascinating. And what he was pointing out in this post was, was that with compression efficiency and as each generation of codec is getting more efficient as you just explained and gave us an overview. There's a, there's a problem that's coming with that in that each generation of codec is also getting even more complex and you know, in some settings and, and I suppose you know, Netflix is maybe an example where you know, it's probably not accurate to say they have unlimited compute, but their application is obviously very different in terms of how they can operate their, their encoding function compared to someone who's doing live, live streaming for example, or live broadcast. Maybe you can share with us as well. You know, through the generation generational growth of these codecs, how has the, how has the compute requirements also grown and has it grown in sort of a linear way along with the compression efficiency? Or are you seeing, you know, some issues with you know, yes, we can get a thousand to one, but our compute efficiency is getting to the, where we could be hitting a wall. Pankaj Topiwala: 23:46 You asked a good question. Has the complexity only scaled linearly with the compression ratio? And the answer is no. Not at all. Complexity has outpaced the compression ratio. Even though the compression ratio is, is a tremendous, the complexity is much, much higher. And has always been at every step. First of all there's a big difference in doing the research, the research phase in development of the, of a technology like VVC where we were using a standardized reference model that the committee develops along the way, which is not at all optimized. But that's what we all use because we share a common code base. And make any new proposals based on modifying that code base. Now that code base is always along the entire development chain has always been very, very slow. Pankaj Topiwala: 24:42 And true implementations are anywhere from 100 to 500 times more efficient in complexity than the reference software. So right away you can have the reference software for say VVC and somebody developing a, an implementation that's a real product. It can be at least 100 times more efficient than what the reference software, maybe even more. So there's a big difference. You know, when we're developing a technology, it is very hard to predict what implementers will actually come up with later. Of course, the only way they can do that is that companies actually invest the time and energy right away as they're developing the standard to build prototype both software and hardware and have a good idea that when they finish this, you know, what is it going to really cost? So just to give you a, an idea, between, H.264 and Pankaj Topiwala: 25:38 H.265, H.264, only had two transforms of size, four by four and eight by eight. And these were integer transforms, which are only bit shifts and adds, took no multiplies and no divides. The division in fact got incorporated into the quantizer and as a result, it was very, very fast. Moreover, if you had to do, make decisions such as inter versus intra mode, the intra modes there were only about eight or 10 intra modes in H.264. By contrast in H.265. We have not two transforms eight, four by four and eight by, but in fact sizes of four, eight, 16 and 32. So we have much larger sized transforms and instead of a eight or 10 intra modes, we jumped up to 35 intra modes. Pankaj Topiwala: 26:36 And then with a VVC we jumped up to 67 intro modes and we just, it just became so much more complex. The compression ratio between HEVC and VVC is not quite two to one, but let's say, you know, 40% better. But the the complexity is not 40% more. On the ground and nobody has yet, to my knowledge, built a a, a, a fully compliant and powerful either software or hardware video codec for VVC yet because it's not even finished yet. It's going to be finished in July 2020. When it, when, the dust finally settles maybe four or five years from now, it will be, it will prove to be at least three or four times more complex than HEVC encoder the decoder, not that much. The decoder, luckily we're able to build decoders that are much more linear than the encoder. Pankaj Topiwala: 27:37 So I guess I should qualify as discussion saying the complexity growth is all mostly been in the encoder. The decoder has been a much more reasonable. Remember, we are always relying on this principle of ever-increasing compute capability. You know, a factor of two every 18 months. We've long heard about all of this, you know, and it is true, Moore's law. If we did not have that, none of this could have happened. None of this high complexity codecs, whatever had been developed because nobody would ever be able to implement them. But because of Moore's law we can confidently say that even if we put out this very highly complex VVC standard, someday and in the not too distant future, people will be able to implement this in hardware. Now you also asked a very good question earlier, is there a limit to how much we can compress? Pankaj Topiwala: 28:34 And also one can ask relatively in this issue, is there a limit to a Moore's law? And we've heard a lot about that. That may be finally after decades of the success of Moore's law and actually being realized, maybe we are now finally coming to quantum mechanical limits to you know how much we can miniaturize in electronics before we actually have to go to quantum computing, which is a totally different you know approach to doing computing because trying to go smaller die size. Well, we'll make it a unstable quantum mechanically. Now the, it appears that we may be hitting a wall eventually we haven't hit it yet, but we may be close to a, a physical limit in die size. And in the observations that I've been making at least it seems possible to me that we are also reaching a limit to how much we can compress video even without a complexity limit, how much we can compress video and still obtain reasonable or rather high quality. Pankaj Topiwala: 29:46 But we don't know the answer to that. And in fact there are many many aspects of this that we simply don't know. For example, the only real arbiter of video quality is subjective testing. Nobody has come up with an objective video quality metric that we can rely on. PSNR is not it. When, when push comes to shove, nobody in this industry actually relies on PSNR. They actually do subjective testing well. So in that scenario, we don't know what the limits of visual quality because we don't understand human vision, you know, we try, but human vision is so complicated. Nobody can understand the impact of that on video quality to any very significant extent. Now in fact, the first baby steps to try to understand, not explicitly but implicitly capture subjective human video quality assessment into a neural model. Those steps are just now being taken in the last couple of years. In fact, we've been involved, my company has been involved in, in getting into that because I think that's a very exciting area. Dror Gill: 30:57 I tend to agree that modeling human perception with a neural network seems more natural than, you know, just regular formulas and algorithms which are which are linear. Now I, I wanted to ask you about this process of, of creating the codecs. It's, it's very important to have standards. So you encode a video once and then you can play it anywhere and anytime and on any device. And for this, the encoder and decoder need to agree on exactly the format of the video. And traditionally you know, as you pointed out with all the history of, of development. Video codecs have been developed by standardization bodies, MPEG and ITU first separately. And then they joined forces to develop the newest video standards. But recently we're seeing another approach to develop codecs, which is by open sourcing them. Dror Gill: 31:58 Google started with an open source code, they called VP9 which they first developed internally. Then they open sourced it and and they use it widely across their services, especially in, YouTube. And then they joined forces with the, I think the largest companies in the world, not just in video but in general. You know those large internet giants such as Amazon and Facebook and and Netflix and even Microsoft, Apple, Intel have joined together with the Alliance of Open Media to jointly create another open codec called AV1. And this is a completely parallel process to the MPEG codec development process. And the question is, do you think that this was kind of a one time effort to, to to try and find a, or develop a royalty free codec, or is this something that will continue? And how do you think the adoption of the open source codecs versus the committee defined codecs, how would that adoption play out in the market? Pankaj Topiwala: 33:17 That's of course a large topic on its own. And I should mention that there have been a number of discussions about that topic. In particular at the SPIE conference last summer in San Diego, we had a panel discussion of experts in video compression to discuss exactly that. And one of the things we should provide to your listeners is a link to that captured video of the panel discussion where that topic is discussed to some significant extent. And it's on YouTube so we can provide a link to that. My answer. And of course none of us knows the future. Right. But we're going to take our best guesses. I believe that this trend will continue and is a new factor in the landscape of video compression development. Pankaj Topiwala: 34:10 But we should also point out that the domain of preponderance use preponderant use of these codecs is going to be different than in our traditional codecs. Our traditional codecs such as H.264 265, were initially developed for primarily for the broadcast market or for DVD and Blu-ray. Whereas these new codecs from AOM are primarily being developed for the streaming media industry. So the likes of Netflix and Amazon and for YouTube where they put up billions of user generated videos. So, for the streaming application, the decoder is almost always a software decoder. That means they can update that decoder anytime they do a software update. So they're not limited by a hardware development cycle. Of course, hardware companies are also building AV1. Pankaj Topiwala: 35:13 And the point of that would be to try to put it into handheld devices like laptops, tablets, and especially smartphones. But to try to get AV1 not only as a decoder but also as an encoder in a smartphone is going to be quite complicated. And the first few codecs that come out in hardware will be of much lower quality, for example, comparable to AVC and not even the quality of HEVC when they first start out. So that's... the hardware implementations of AV1 that work in real time are not going to be, it's going to take a while for them to catch up to the quality that AV1 can offer. But for streaming we, we can decode these streams reasonably well in software or in firmware. And the net result is that, or in GPU for example, and the net result is that these companies can already start streaming. Pankaj Topiwala: 36:14 So in fact Google is already streaming some test streams maybe one now. And it's cloud-based YouTube application and companies like Cisco are testing it already, even for for their WebEx video communication platform. Although the quality will not be then anything like the full capability of AV1, it'll be at a much reduced level, but it'll be this open source and notionally, you know, royalty free video codec. Dror Gill: 36:50 Notionally. Yeah. Because they always tried to do this, this dance and every algorithm that they try to put into the standard is being scrutinized and, and, and they check if there are any patents around it so they can try and keep this notion of of royalty-free around the codec because definitely the codec is open source and royalty free. Dror Gill: 37:14 I think that is, is, is a big question. So much IP has gone into the development of the different MPEG standards and we know it has caused issues. Went pretty smoothly with AVC, with MPEG-LA that had kind of a single point of contact for licensing all the essential patents and with HEVC, that hasn't gone very well in the beginning. But still there is a lot of IP there. So the question is, is it even possible to have a truly royalty free codec that can be competitive in, in compression efficiency and performance with the codec developed by the standards committee? Pankaj Topiwala: 37:50 I'll give you a two part answer. One because of the landscape of patents in the field of video compression which I would describe as being, you know very, very spaghetti like and patents date back to other patents. Pankaj Topiwala: 38:09 And they cover most of the, the topics and the most of the, the tools used in video compression. And by the way we've looked at the AV1 and AV1 is not that different from all the other standards that we have. H.265 or VVC. There are some things that are different. By and large, it resembles the existing standards. So can it be that this animal is totally patent free? No, it cannot be that it is patent free. But patent free is not the same as royalty free. There's no question that AV1 has many, many patents, probably hundreds of patents that reach into it. The question is whether the people developing and practicing AV1 own all of those patents. That is of course, a much larger question. Pankaj Topiwala: 39:07 And in fact, there has been a recent challenge to that, a group has even stood up to proclaim that they have a central IP in AV1. The net reaction from the AOM has been to develop a legal defense fund so that they're not going to budge in terms of their royalty free model. If they do. It would kill the whole project because their main thesis is that this is a world do free thing, use it and go ahead. Now, the legal defense fund then protects the members of that Alliance, jointly. Now, it's not as if the Alliance is going to indemnify you against any possible attack on IP. They can't do that because nobody can predict, you know, where somebody's IP is. The world is so large, so many patents in that we're talking not, not even hundreds and thousands, but tens of thousands of patents at least. Pankaj Topiwala: 40:08 So nobody in the world has ever reviewed all of those patent. It's not possible. And the net result is that nobody can know for sure what technology might have been patented by third parties. But the point is that because such a large number of powerful companies that are also the main users of this technology, you know, people, companies like Google and Apple and Microsoft and, and Netflix and Amazon and Facebook and whatnot. These companies are so powerful. And Samsung by the way, has joined the Alliance. These companies are so powerful that you know, it would be hard to challenge them. And so in practice, the point is they can project a royalty-free technology because it would be hard for anybody to challenge it. And so that's the reality on the ground. Pankaj Topiwala: 41:03 So at the moment it is succeeding as a royalty free project. I should also point out that if you want to use this, not join the Alliance, but just want to be a user. Even just to use it, you already have to offer any IP you have in this technology it to the Alliance. So all users around the world, so if tens of thousands and eventually millions of you know, users around the world, including tens of thousands of companies around the world start to use this technology, they will all have automatically yielded any IP they have in AV1, to the Alliance. Dror Gill: 41:44 Wow. That's really fascinating. I mean, first the distinction you made between royalty free and patent free. So the AOM can keep this technology royalty free, even if it's not patent free because they don't charge royalties and they can help with the legal defense fund against patent claim and still keep it royalty free. And, and second is the fact that when you use this technology, you are giving up any IP claims against the creators of the technology, which means that if any, any party who wants to have any IP claims against the AV1 encoder cannot use it in any form or shape. Pankaj Topiwala: 42:25 That's at least my understanding. And I've tried to look at of course I'm not a lawyer. And you have to take that as just the opinion of a video coding expert rather than a lawyer dissecting the legalities of this. But be that as it may, my understanding is that any user would have to yield any IP they have in the standard to the Alliance. And the net result will be if this technology truly does get widely used more IP than just from the Alliance members will have been folded into into it so that eventually it would be hard for anybody to challenge this. Mark Donnigan: 43:09 Pankaj, what does this mean for the development of so much of the technology has been in has been enabled by the financial incentive of small groups of people, you know, or medium sized groups of people forming together. You know, building a company, usually. Hiring other experts and being able to derive some economic benefit from the research and the work and the, you know, the effort that's put in. If all of this sort of consolidates to a handful or a couple of handfuls of, you know, very, very large companies, you know, does that, I guess I'm, I'm asking from your view, will, will video and coding technology development and advancements proliferate? Will it sort of stay static? Because basically all these companies will hire or acquire, you know, all the experts and you know, it's just now everybody works for Google and Facebook and Netflix and you know... Or, or do you think it will ultimately decline? Because that's something that that comes to mind here is, you know, if the economic incentives sort of go away, well, you know, people aren't going to work for free! Pankaj Topiwala: 44:29 So that's of course a, another question and a one relevant. In fact to many of us working in video compression right now, including my company. And I faced this directly back in the days of MPEG-2. There was a two and a half dollar ($2.50) per unit license fee for using MPEG-2. That created billions of dollars in licensing in fact, the patent pool, MPEG-LA itself made billions of dollars, even though they took only 10% of the proceeds, they already made billions of dollars, you know, huge amounts of money. With the advent of H.264 AVC, the patent license went not to from two and a half dollars to 25 cents a unit. And now with HEVC, it's a little bit less than that per unit. Of course the number of units has grown exponentially, but then the big companies don't continue to pay per unit anymore. Pankaj Topiwala: 45:29 They just pay a yearly cap. For example, 5 million or 10 million, which to these big companies is is peanuts. So there's a yearly cap for the big companies that have, you know, hundreds of millions of units. You know imagine the number of Microsoft windows that are out there or the number of you know, Google Chrome browsers. And if you have a, a codec embedded in the browser there are hundreds of millions of them, if not billions of them. And so they just pay a cap and they're done with it. But even then, there was up till now an incentive for smart engineers to develop exciting new ideas in a future video coding. But, and that has been up the story up till now. But when, if it happens that this AOM model with AV1 and then AV2, really becomes a dominant codec and takes over the market, then there will be no incentive for researchers to devote any time and energy. Pankaj Topiwala: 46:32 Certainly my company for example, can't afford to you know, just twiddle thumbs, create technologies for which there is absolutely no possibility of a royalty stream. So we, we cannot be in the business of developing video coding when video coding doesn't pay. So the only thing that makes money, is Applications, for example, a streaming application or some other such thing. And so Netflix and, and Google and Amazon will be streaming video and they'll charge you per stream but not on the codec. So that that's an interesting thing and it certainly affects the future development of video. It's clear to me it's a negative impact on the research that we got going in. I can't expect that Google and Amazon and Microsoft are going to continue to devote the same energy to develop future compression technologies in their royalty free environment that companies have in the open standards development technology environment. Pankaj Topiwala: 47:34 It's hard for me to believe that they will devote that much energy. They'll devote energy, but it will not be the the same level. For example, in developing a video standards such as HEVC, it took up to 10 years of development by on the order of 500 to 600 experts, well, let's say four to 500 experts from around the world meeting four times a year for 10 years. Mark Donnigan: 48:03 That is so critical. I want you to repeat that again. Pankaj Topiwala: 48:07 Well, I mean so very clearly we've been putting out a video codec roughly on the schedule of once every 10 years. MPEG-2 was 1994. AVC was 2003 and also 2004. And then HEVC in 2013. Those were roughly 10 years apart. But VVC we've accelerated the schedule to put one out in seven years instead of 10 years. But even then you should realize that we had been working right since HEVC was done. Pankaj Topiwala: 48:39 We've been working all this time to develop VVC and so on the order of 500 experts from around the world have met four times a year at all international locations, spending on the order of $100 million per meeting. You know so billions of dollars have been spent by industry to create these standards, many billions and it can't happen, you know without that. It's hard for me to believe that companies like Microsoft, Google, and whatnot, are going to devote billions to develop their next incremental, you know, AV1and AV2 AV3's. But maybe they will it just, that there's no royalty stream coming from the codec itself, only the application. Then the incentive, suppose they start dominating to create even better technology will not be there. So there really is a, a financial issue in this and that's at play right now. Dror Gill: 49:36 Yeah, I, I find it really fascinating. And of course, Mark and I are not lawyers, but all this you know, royalty free versus committee developed open source versus a standard those large companies who some people fear, you know, their dominance and not only in video codec development, but in many other areas. You know, versus you know, dozens of companies and hundreds of engineers working for seven or 10 years in a codec. So you know, it's really different approaches different methods of development eventually to approach the exact same problem of video compression. And, and how this turns out. I mean we, we cannot forecast for sure, but it will be very interesting, especially next year in 2020 when VVC is ratified. And at around the same time, EVC is ratified another codec from the MPEG committee. Dror Gill: 50:43 And then AV1, and once you know, AV1 starts hitting the market. We'll hear all the discussions of AV2. So it's gonna be really interesting and fascinating to follow. And we, we promise to to bring you all the updates here on The Video Insiders. So Pankaj I really want to thank you. This has been a fascinating discussion with very interesting insights into the world of codec development and compression and, and wavelets and DCT and and all of those topics and, and the history and the future. So thank you very much for joining us today on the video insiders. Pankaj Topiwala: 51:25 It's been my pleasure, Mark and Dror. And I look forward to interacting in the future. Hope this is a useful for your audience. If I can give you a one parting thought, let me give this... Pankaj Topiwala: 51:40 H.264 AVC was developed in 2003 and also 2004. That is you know, some 17 years or 16 years ago, it is close to being now nearly royalty-free itself. And if you look at the market share of video codecs currently being used in the market, for example, even in streaming AVC dominates that market completely. Even though VP8 and VP9 and VP10 were introduced and now AV1, none of those have any sizeable market share. AVC currently dominates from 70 to 80% of that marketplace right now. And it fully dominates broadcast where those other codecs are not even in play. And so they're 17, 16, 17 years later, it is now still the dominant codec even much over HEVC, which by the way is also taking an uptick in the last several years. So the standardized codecs developed by ITU and MPEG are not dead. They may just take a little longer to emerge as dominant forces. Mark Donnigan: 52:51 That's a great parting thought. Thanks for sharing that. What an engaging episode Dror. Yeah. Yeah. Really interesting. I learned so much. I got a DCT primer. I mean, that in and of itself was a amazing, Dror Gill: 53:08 Yeah. Yeah. Thank you. Mark Donnigan: 53:11 Yeah, amazing Pankaj. Okay, well good. Well thanks again for listening to the video insiders, and as always, if you would like to come on this show, we would love to have you just send us an email. The email address is thevideoinsiders@beamr.com, and Dror or myself will follow up with you and we'd love to hear what you're doing. We're always interested in talking to video experts who are involved in really every area of video distribution. So it's not only encoding and not only codecs, whatever you're doing, tell us about it. And until next time what do we say Dror? Happy encoding! Thanks everyone. 

Learn about Unified Patents hereCheck out Unified Patents Objective PAtent Landscape OPAL toolRead the Independent economic study for HEVC royaltiesShawn Ambwani LinkedIn profileRelated episode: VVC, HEVC & other MPEG codec standardsThe Video Insiders LinkedIn Group is where thousands of your peers are discussing the latest video technology news and sharing best practices. Click here to join --------------------------------------Would you like to be a guest on the show? Email: thevideoinsiders@beamr.comLearn more about Beamr-------------------------------------- TRANSCRIPT (edited slightly for readability)Narrator: 00:00 The Video Insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation codec nerd and a marketing guy who knows what I-frames and macro blocks are. Here are your hosts, Mark Donnigan and Dror Gill. Mark Donnigan: 00:19 Well, welcome back to The Video Insiders. Dror, how you doing today? I'm doing great. How are you Mark? I am doing awesome. As always. I am super pleased to welcome Shawn Ambwani who is co-founder of Unified Patents and Shawn is gonna tell us all about what Unified Patents does and we're going to dive into, you know, just a really tremendous discussion. But Shawn, Welcome to the podcast! Shawn Ambwani: 00:46 Hey guys. Thanks Mark. Thanks for, for allowing me to participate on your wonderful podcast. I look at this as similar to 'All Things Considered' and 'How I built this', two of my favorite podcasts. Mark Donnigan: 01:01 Those are awesome podcasts by the way. What an honor? Yeah. Wow. The level that I expect you guys to be at in traffic very shortly. That's right. Well, we hope so to. Well, why don't you introduce yourself you know, and give us a quick snapshot of your background and then let's let's hear about what Unified Patents is doing. Shawn Ambwani: 01:23 It's kind of a, I have an interesting I mean some might say not so interesting, but I think it's interesting background related to this area since, you know, the first startup that I did and the second one were all related to MPEG4. So I co-founded a company called Envivio, which way back when was actually one of the original MPEG4 companies when they just had simple profile actually out there doing encoders and decoders. And then I went to a Korean company called NexStreaming, which actually still exists, which is doing encoders as well, but more for the mobile space and decoders. So it's an area I'm quite familiar with. I wasn't really being an attorney back then. Now I'm kind of more of an attorney than I was back then, but I tried to avoid being an attorney as much as possible in general. Shawn Ambwani: 02:16 And basically I helped co-found a company called Unified Patents. And what unified patents does is it gets contributions from member companies as well as it allows small companies to join for free and they participate in joining what we call zones. And these different zones are intended to protect against what we consider unsubstantiated or invalid patent assertions. And the goal of these zones is to deter those from occurring in the first place. So if you imagine the kind of a technology area, let it be content or let it be video codec in this case or other things as having a bunch of companies that have a common interest in maintaining, you know, patents and ensuring patents that are asserted in that space are valid, which means that no one invented the idea beforehand. And also that it's fairly priced and you know, people are explaining the rationale behind what they're doing and they're not basically just attempting to get people to settle out, not because the assertions are valid or good, but simply because the cost of litigation is so high when it comes to patents. Shawn Ambwani: 03:35 And we want to deter that type of activity because there's been a lot of investment in that activity so far. In fact, most litigation's are by NPE's and so Unified started by doing those zones and, and, and we've have a bunch of them now. We just launched an open source zone in fact, but with you know, Linux foundation and OIN and IBM and others and the video codec zone was something that we were thinking about for a long time. It's something that I'm very familiar with from my past dealings with MPEG LA and other pools. And it was a big issue I think. And it has been a constant issue, which is how do you deal with multiple pools or multiple people asking for money in a standard? How do you deal with the pricing of it? Especially if you're smaller entities and you don't have the information that may be larger companies might have. Shawn Ambwani: 04:26 How do you deal with that and how do you deal with all the invalid assertions that are being done or declarations that occur in this area? How do you figure out who you have to pay? And how much you have to pay. All these add a level of complexity to deploying these standards, which makes adoption harder and creates the uncertainty that causes people to go to proprietary solutions, which I think is a negative in the end. So that's why Unified Patents really created this area and created the video codec zone. And basically we've been pretty, I think, successful so far now actually going through and doing each one of the things we said we were going to do. Dror Gill: 05:06 So what, what are those things basically when you set up a zone and, and want to start finding those patterns that may be invalid how do you go about doing that? What is the process? Shawn Ambwani: 05:18 Yeah, so I mean there's two major things in the SEP zones that it's not, it's not just about finding invalid patents, although I can tell you it's relatively easy to find invalid patents in any of these zones. That's not a difficulty. The hard part is figuring out which ones to go to or which ones are going to be the most interesting to go after. And that takes a lot of art. Essentially identifying them, finding out good prior art that we feel comfortable with, hiring good counsel. There's all kinds of weighing mechanisms that go into it, who the entity is, how it came about, how old the patents are, where it came from. All of these variables go into that kind of equation of when we decide. What's kind of unique about the way we work is we work independently of our members, so our members are funding these activities and some join for free. Shawn Ambwani: 06:11 So we have a number of members of the video codecs and we use all this money and information in our activities to basically go back and decide what to do, with the objective of deterring you know, what we consider bad assertions in this space. And then that's one part of it. The other part of it is that SEP's are all about, you know, an area called FRAND, fair, reasonable, non-discriminatory. And part of all of that involves negotiation. And so what we provide are tools to allow companies to negotiate we think in a fair and more transparent way to licensors as to pricing. But also explaining why the pricing is the way it is. Because one of the problems that we've had in the big picture is that a lot of these licensors have been asking for money, whether their own pools, whether outside of pools, whatever. Shawn Ambwani: 07:12 But no one can really explain why the price is what it is. And I think that leads to a lot of people to just stop paying or stop wanting to get into licensing discussions. And that's not beneficial for the market. And so by explaining how the price comes out the way it is and providing a very, we consider, solid methodology for it, it allows our members but also licensors to better understand who owns what and how much value is in the standard. So what they should reasonably expect to get for that technology and how much licensees reasonably should expect to pay in order to deploy the technology. Mark Donnigan: 07:55 Now my question, you know, Shawn is when you are getting into these conversations with the parties or party that, you know, owns this IP and I'm speaking more around sort of the pricing and the model and that sort of thing. Are you then...is that information available to your members or is it more that you're sort of helping facilitate, helping bring some rationality, you know, so that then that body can turn around and make public: "Hey, great news!" We've decided that all digitally distributed content doesn't carry, you know, a royalty cost. What exactly, I guess what my question is, what exactly is, is your role then in informing the market? Shawn Ambwani: 08:40 I think that, well, I mean there's a number of things to talk about, but what's I think most important is that we, you know, we don't know necessarily what the right price is. We hired an outside economist to look into that and he came back with a pricing range in you know, a report that we gave the highlights too and there was some press over and it's on our website but you can also look at it through a number of particles and basically he came back with a price of between 8 cents and 28 cents I believe if I'm accurate. Is what he believed the estimate to be for the value of the technology including everything. And it ranged based on I think the device and like other factors and stuff like that. Now that high level information we provided publicly and in fact we provided the information on who made the report when it was created and what it was based on. Shawn Ambwani: 09:38 And we even provided kind of the overall methodology of how it was done, which is basically being used at a very high level. They used MPEG LA's AVC license as the starting point or the foundation for deciding what HEVC in this case, which is what he was looking at, pricing should be based on his expert analysis. And then he modified that based on switching costs based on the cost of bandwidth, the cost of storage and quality and other factors basically that are valuable. So, that's where we went. Now, what's important to understand is that we published that information so anyone could take a look at the, at the high level. And the methodology pretty much tells you the roadmap of where we started and how we ended up where we are. The other part is how do you decide who you have to pay and how much each person gets. Even assuming that you figure out that, let's say it's 25 cents, that you think the royalty rate should be for it. And I'm not saying that's the number, but everyone can decide on whatever number they feel comfortable. Our expert created this report and we published it. Other people can create other reports and I'm sure they have their own kind of versions. But what's important for us is that, you know, people should explain why they came out with their pricing. And unfortunately in pools and licensing organizations in general, that just doesn't happen. Dror Gill: 11:05 So basically you're finding economical reason behind a certain price for for this technology. In this case, HEVC. And now companies who want to use HEVC, how do they use this number? Because they have your number, which is the total, and then they have royalty rates that are asked that, you know, certain patent pools are asking and they add up to a different number that could be a higher number. So do they just you know, divide the number that they think is the right one among the different patent pools and pay them the amount they think they should pay or do they just use it as a negotiating tool when they talk to them and, and you know, and negotiate the actual world, the rates that they will have to pay? Shawn Ambwani: 11:52 By providing a lot of this information. Some of it publicly like economic report in some format. The hope is that smaller entities instead of rolling over when licensing people come by and say, Hey, take it or leave it, they really have an ability to make a fair response, a good faith response with information that allows them to then basically justify why they came up with a price and really push back and say, listen, you know, this is what my methodology came out to. Now. It could be right, could be wrong. You know, in the end in FRAND negotiations, I have to make a good faith offer. That's really the intent. So that's an important aspect of pushing back on this kind of, we think less information that is occurring in the marketplace and more fragmentation. And I think they're all interrelated because of the less information you have more fragmentation. Cause if everyone could agree on a price and everyone agreed that this is the fair value for the technology, there really wouldn't be multiple pools in my opinion or multiple licensors, because everyone would know what the number is. And so why would you separate? Dror Gill: 13:07 But basically you're saying that even if a patent pool set, the royalty rates and those royalty rates in some cases are public, at least for some of the patent pools, this is not what a licensor would pay. This is just kind of a starting point for a negotiation and you're providing tools for this type of negotiation. Shawn Ambwani: 13:23 We also think that validity is a big issue because none of these entities look at validity when they're incorporating patents into their pools or into their licensing. It's really up to the licensees or the people who are potentially taking the license to have the responsibility to go out and figure that out, which can be very, very costly. Dror Gill: 13:44 You assume they're valid, right? If they're licensing patents to you, you assume that they're licensing valid patents. Right? These are kind of, you know, respectable patent holders and patent pools. Why would they license something that's not valid? Shawn Ambwani: 13:57 I mean, it's a great point. I mean, the argument would be that they want to license patents. Mark Donnigan: 14:03 That's their business at the end of the day. Yeah. Shawn Ambwani: 14:08 Right. So, you know, if you had a car and you're trying to sell a car, you're going to accentuate the good things about the car. Not that it's a rebuilt or something like that or you know, like it's, it's been, you know, it's been in a crash or accident like, yeah. Like you're going to show what you want to show. Right. And that's natural in any of these cases. The unfortunate fact is that it's very costly to figure out that stuff and there's no really organization you'd think a licensing organization like MPEG LA or others. And I'm not saying MPEG LA is doing a bad job necessarily, I'm just pointing them out as an example, would do a better job of vetting to some degree on that type of activity. But they don't, and I think there's a number of reasons Mark Donnigan: 14:53 Why do they want to do that? I almost liken this to the 500 channel cable bundle of which there's about 15 high quality channels and there's 485 that are anywhere from just a, you know, not, not relevant, not interesting to, you know, to even lower quality than that, but, but you know, but Hey, I got a 500 channel bundle, right? So I feel like, wow, it must be worth $100 a month, you know, or whatever. Shawn Ambwani: 15:23 The idea that that licensing organizations like MPEG LA or (HEVC) Advance or other ones like that aren't doing it to the benefit of their licensors. It just seems ridiculous to me. I mean the people on their, on their management and the people who are actually owning that organization, typically it's managed and owned and administrative fees are paid to licensors. And traditionally the money flows one way from licensees to licensors. It's for the benefit of the licensors. And the rules that they put in are essentially to make sure that those guys are protected. They have no incentive in general of saying people's patents are invalid. And, and that's just a bad fact pattern for them. If basically they get back and say, Hey, listen this patent... Yeah, no, it's bad. Mark Donnigan: 16:16 Exactly. So, so in that context then it completely makes sense that they don't vet you know, at the level that you are and why, you know, Unified Patents needs to exist, you know, is because we need this sort of independent third party. I guess. I, I, you know, that's, that's out there doing this work. Now, Shawn, one of the things that I noticed is you're acting both against NPE's, so, non-practicing entities, and against SEP's. So standard essential patents. What are the issues with SEP's? Shawn Ambwani: 16:51 Well, I mean the general assumption has been, and I don't know where this assumption came from, was that standard essential patents or people who declare their patents to be standard essential are more likely to be valid than other patents. And in the real world where there's litigation and there's challenges and things get checked out or vetted essentially, adversarially, the reality is that standard essential patents in all the studies that have been done fair, far worse, than normal patents do on average. And you know, it's not shocking actually when you think about it. Obviously there's a lot of self selection here, but part of the reason why is, you know, when you're submitting into pools or in when you're getting these patents, when part of a standardization body or doing other activities, there's a lot of other people involved and it's usually built on other ideas that people have had in the past. Shawn Ambwani: 17:59 And it's not surprising that a lot of these patents have underlying ideas that had been done in the past or other people had brought up previously. Sometimes they weren't accepted, sometimes they were or sometimes they were put on hold. Who knows? But there's a lot of prior art oftentimes in these areas. These aren't open fields, these aren't brand new innovations that typically come up. And so that's not surprising. Now, you know, there's also a general belief that standard essential patterns are more valuable. And I think, you know, that's a pretty, I would say, you know, I dunno if it's absolutely valid, but it's not unreasonable to believe that if you declare a patent, as standardized, if you look at the average patent and compared to that patent, it's probably your, it's probably more valuable, at that point. Because you basically said it's part of a standard that people are probably going to adopt at that point versus a patent in general, which you never know most of the time, whether anyone's going to use that patent. Shawn Ambwani: 19:02 I mean the vast majority of patents are never actually used in any way whatsoever. They're not enforceable because they're just ideas that people have most of the time, and these patents are arguable more likely than not to be in a standard and that standard might or might not actually get used in the end. Inherently you get - they're more valuable. The problem is there's tons of over declaration that occurs in this area. There's very little incentive. I mean some places there's more of an incentive than others, but the way MPEG works specifically is that you can do blanket declarations and so you don't have to declare specific patents. And, other standards, you have to basically declare each individual patent that you have. So, I mean, there's all kinds of trade offs, and all these different things, but the reality is that no one really knows exactly how many patents need to be licensed. And that just creates a lot of uncertainty. And you know, a lot of companies who are trying to make money, not off products but off of doing licensing thrive on uncertainty because that's where they can make money. Is basically by, you know, saying, okay, well who knows what can happen, but if you take care of me now, I can make sure that I'm not going to cause you issues. Dror Gill: 20:23 Right. And that's why uncertainty is in the middle of FUD, fear, uncertainty, doubt, which is one of those tactics and uncertainty is definitely a big part of that. Shawn Ambwani: 20:33 Yeah. I mean, the other thing is that companies in general, it seems like a one way street a lot of the time, which is pretty unfortunate in that although I'm not sure if I have a good solution, you know, a lot of companies, the licensors have a way of getting together, agreeing on a price and then licensing through an organization like MPEG LA or others to do that type of activity or Velos (Media), or whatever it is. They choose, you know, they can select a price, they can work together, agree on a price. And the reason why they can do that according to the DOJ is because it's a different product than what's available before. So it decreases uncertainty by making it easier for people to take a license of convenience for that specific technology area. Dror Gill: 21:21 Otherwise, it might might've been considered price setting, Shawn Ambwani: 21:24 Right? Yeah. It would be considered price. It would be considered price setting. But in this case, the argument is always that you can always go to each individual company and get a license or negotiate a separate license. This is a license of convenience for this technology area from all these companies for one price. And that makes it a lot easier for people on both sides to be able to know exactly how much they're going to be getting and how much they're going to have to pay for clearing this risk. Which makes sense. I fundamentally have no problem with pools and what they do. The, the issue comes up is that a lot of these pools, A) don't talk about the pricing, they don't look at the validity. They don't really have a great essentially checking on top of it. And they're very much incentivized to help out the licensors, not the licensees figure stuff out. And what ends up happening is over time you kind of, and you have companies also that are not interested in making products, which is unfortunate. They're just interested in making money off of their licensing. Which is unfortunate because there's a lot of games that can be played in the standardization world to get your stuff in and then get your patents in basically. Mark Donnigan: 22:44 Well, it ultimately, it, it stops innovation. I mean, at the end of the day, you know, and one thing, and Dror and I have talked about this on episodes and we've certainly talked about this a lot, you know, privately within Beamr is, you know, it's a little bit mystifying as well because okay, so HEVC clearly was set back as a result of, of many issues. But you know, largely what we'd been talking about for the last 35 minutes and the adoption of HEVC. And yet these people, as you point out, the licensors, they don't make money if nobody's using the technology. So, so what's mystifying to us is that this is not, you know, it's not like somehow they're getting paid still. You know, even though the adoption of the technology is not in place or it's not being used, they're not getting paid. And so it seems like at some point, you know, a rational actor would stand up and say, wait a second, I'd rather get something rather than nothing! But, it's almost like they, they're not acting that way. Dror Gill: 23:46 But, but it did happen. They did reduce the royalty rate. Yes, yes, yes. Certainly. And they did come to their senses and they did put a cap and then initially it was uncapped and they did remove royalties from content. And you know, they did a lot of things in the right direction after the pressure from the market when they realized they're not going to get anything. And when AV1 started to happen, you know, and they were pressured by that, by a competing codec that was supposedly a royalty free and didn't have these issues. So I think the situation is improved. But you've launched a specific zone. It's called the video codec zone, but basically right now it deals only with HEVC. Shawn Ambwani: 24:33 A lot of these patents that we've challenged relate not just to HEVC but potentially to AV and other codecs like AVC as well. Cause there's such overlap between these things. That's why we generically call it a video codec zone. So, obviously a lot of the things that we've looked at in like the economic report and everything else and landscape, a lot of the focus has been on HEVC. Dror Gill: 24:59 So you examined the HEVC and and you saw this situation that you have three patent pools. One of them hasn't even announced the royalty rates and, and you have a lot of independent patent holders who claim to have standard essential patents for HEVC. And this is kind of your, you're opening a, a situation. So what, what was the first thing that, that you did, how did you start to, to approach the HEVC pattern topic and what actions did you take? Shawn Ambwani: 25:34 Like I said, we've done a bunch of different stuff. We had a submission repo called open, which where we collated all the prior art, not prior art, but submissions into the standard for HEVC and AVC and other standards from MPEG so people can make it easily searchable. In fact, 50% of the priority art that we got for our patent challenges came from the submission repo, which is great, which is basically, you know, previous submissions to the same standard. We have OPAL, which is our landscape tool. And then, you know, obviously we have OPEN which is our evaluation report that I mentioned for HEVC. And then we did a bunch of reviews of validity and challenged a bunch of patents in different licensing entities. I mean, Velos, I think they don't consider themselves a pool. Just to be clear. Dror Gill: 26:29 Because they actually own the patents. They've licensed those patents on their own? Shawn Ambwani: 26:34 Well, I think they just don't consider themselves tackling a patent pool in the way that MPEG LA and HEVC Advanced does simply that would throw them into a different bucket and they would have all kinds of requirements on them that they don't want basically. So you know when the DOJ kind of made the rules or kind of the lawyers decided what the right rules are to make it work, you know like you've got to show your stuff. Basically you got to show your price, you've got to make sure it's reasonable or it's, you know, like there's, there's no most favored nation clause. I mean there is a most favorite nation (MFN) let me rephrase this. So all these things to make sure that everything is very transparent in order to allow this kind of companies to get together and set a price for how much they want to license for it, which typically would have huge anti-competitive or antitrust issues. Right. They made all these rules and Velos I think would not consider themselves technically a patent pool like those guys because that would make them have the similar requirements. Dror Gill: 27:40 So they're like an independent patent holder? Shawn Ambwani: 27:42 I don't really know what they call themselves. I've definitely never heard them say that they're a patent pool. I've heard other people call them a patent pool. I probably have at some point, but I don't really know if they actually consider themselves a patent pool Dror Gill: 27:55 Because I noticed that your litigation was against the patent holders. Companies like GE and KBS and against Velos Media itself. Yeah. Shawn Ambwani: 28:07 Yeah. Well Velos is you know, an unusual beast in that it owns a number of patents that got transferred to it as well as it provides licenses to the people who participated. You know, the other patent holders in general are much more traditional in their patent pool type activity in that the patent holders are different from the people who are doing the licensing. Dror Gill: 28:28 And you're not suing the patent pools like MPEG LA and HEVC Advanced or not your targets? Shawn Ambwani: 28:32 Well, they don't own patents directly, so really nothing to do as far as I know. I mean, you could say, you know, part of it is we're challenging them to a certain degree on their pricing and kind of their whole model of not looking at validity by challenging some of their patents as well as, you know, putting them on notice that as they get more patents in, we might challenge further patents for validity. So why don't they do it ahead of time? I mean, the idea that, you know, validity is a victimless crime if you don't check for validity, it doesn't hurt anyone. It's just not true in my opinion. It's just not true because you are hurting the people who actually innovated. There's a set amount of money that goes to everyone. If you have a bunch of patents and they're just like, you're checking for essentiality before you allow a patent in, you check for validity because there's a bunch of patents that just aren't valid that shouldn't be, they should not be making money off of. It just incentivizes people to get more invalid patents in the same space that they can stick into a pool to get a bigger share of it, like a giant game. Right? Mark Donnigan: 29:43 Yeah, that's a really good point. I'm wondering what is the cost to test for essentiality? Is some of this just sort of practical like it's just either too time consuming or costly to test? Yeah. I mean Shawn Ambwani: 29:58 Esentiality is often times more expensive than validity in some cases, but I mean they do test for essentiality. The companies pay to have their own patents tested often times for essentiality, but there is no test for validity that they enforce. So no one actually does it. You know, if they did ask for it, I'm sure people in some cases would pay for it, but more importantly, people who didn't think their patents would be found valid, probably wouldn't submit them in the first place then. Then there would be, there'd be huge disincentive for people who had that risk of that happening. They just wouldn't submit it, which you know, obviously it's going to hurt the pool because they get less patents. And at the same time, the hope is that people will think twice before they submit stuff they know is crap. Anyway. Mark Donnigan: 30:43 So what is, what is your bar for determining low quality? I mean, what does that process look like? Shawn Ambwani: 30:52 We have a bunch of patents that come into our hopper that we're constantly looking at in every single zone that we're in and we're constantly looking and seeing if it's a valid patent or not. And there's multiple ways of doing that. We have crowd sourcing that we do for that. We just pay people, you know, in order to do prior art experts for example, to do prior art searches. You can prior art search infinitely long these, there's no stopping. You know, what you can do. But you know, in the end there's only so much you can reasonably expect to find. And so from my perspective, you know, there's definitely been situations where we've looked at patents and we've said, okay, we don't think we have good prior art. We're just not going to do anything about it. And that's okay. In fact, I mean it's okay if a licensing entity or licensor has a valid patent, that's perfectly fine with me. Shawn Ambwani: 31:49 I think if they have a valid patent, they should be able to make money off of it. I have no problem with, it should be a fair amount if it's in a standard based on FRAND principles, but in general, people should be able to make money off of a valid patent. The problem is is that a lot of people are making a lot of money, in my opinion, off of a lot of bad patents. And invalid patents, which hurts the people who actually do have good patents because they're getting crowded out, which is sad because that really is the disincentive for innovation then is when the people actually are innovating aren't making money off of it because they're getting crowded out by the people who are just playing a good game. Dror Gill: 32:25 You described earlier the, the process with a standard setting bodies such as MPEG where you declare your patents but you only, you can declare them as, as a pool or as a bunch of patents and not specifically, and then you can basically, Dror Gill: 32:40 You know, create a pool and charge as much as you want if it's under the FRAND principles. Do you think there's anything broken in the standard setting process itself? Do those committees need to do something else in order to make sure that when they create a standard, the situation of royalties of, of the situation of, of IP which is essential to that standard is more well known that you have less uncertainty in that IP? Shawn Ambwani: 33:09 Yeah, I'm not sure. I mean there's always ways of like tweaking the system. Every standards body has different ways of managing it. I mean the only really clean way of doing it is saying it's royalty free and having anyone who participates in the standard agree that it's royalty free. Anything above that, just you know, you can play all these different types of rules and machinations and 3GPP has their own and other people, organizations have their own. But in the end it ends up being the same issue of you know, under declaring over declaring - issues with essentiality, validity, all kinds of other things. So I'm not sure if you, unless you go to that binary level, how much, you know, changing that up is going to change things fundamentally. I think the more fundamental thing is that, you know, the idea that I think the fundamental reason why you have these patent pools and other things like that was to clear risk and decrease uncertainty. Unfortunately I'd say uncertainty is actually increasing in some of these cases not decreasing by all these different groups asking for money at this point, which is unfortunate. Dror Gill: 34:17 No, that's a very interesting insight, really. Mark Donnigan: 34:19 Hey, thanks for joining us, Shawn. This was really an amazing discussion and we definitely have to have a part two. Shawn Ambwani: 34:26 All right, well, thanks for your time, gentlemen. I really appreciate it. Dror Gill: 34:28 Thank you. Narrator: 34:30 Thank you for listening to The Video Insiders podcast, a production of Beamr Imaging limited. To begin using Beamr's codecs today, go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

The NAB Streaming Experience website can be found hereLearn about NAB Streaming Summit hereDan Rayburn LinkedIn profileRelated episode: What happens when content owners go directThe Video Insiders LinkedIn Group is where over 1,600 of your peers are discussing the latest news and sharing information of interest. Click here to joinWould you like to be a guest on the show? Send an email to: thevideoinsiders@beamr.comLearn more about Beamr TRANSCRIPTION (Note: This is machine generated and may have been lightly edited)Dan Rayburn: 00:00 There's seven, eight years ago when we were all playing in this arena and trying to really figure out the business model today, this is big business. We have tens of billions of dollars at stake. This stuff has to work. It has to be right and there is a lot of pressure on these new conglomerates to make sure that the video workflows, they're building out work properly because it is truly the future of their business. And I think the great way to really drive that point home is just remember all the services that Dan Rayburn: 00:25 launched say five years ago in the market. When the services came out, one, there was no investor day because investors didn't care what you were launching because at the time you weren't spending that much money and it was still a newish experience from a quality standpoint. Today, every single service that's launching is having an investor day where before the service is even out they're projecting to investors of when these services will become profitable. Talk about a shift in our industry. Announcer: 00:53 The video insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation codec nerd and a marketing guy who knows what I frames and macro blocks are. And here are your hosts Mark Donnigan and Dror Gill. Mark Donnigan: 01:07 Welcome to another super exciting episode of the video insiders. We have Dan Rayburn with us again. Uh, this is part two. Yeah, it's amazing. And his first interview was one of the most popular ones on our podcast. We have to say it's the most downloaded and we have a lot to talk to talk about because since the last podcast episode where Dan was interviewed here on The Video Insiders, a lot has happened in the OTT space. I mean, really a lot. Yeah, a lot. So Dan, you know, welcome back to The Video Insiders. Thanks guys. Thanks for having me. You know me, I always have lots to talk about, so I love chatting about the industry. Well you do, you are an easy guest to host, that's for sure. Dan Rayburn: 01:53 I've always got stuff to say, right? I have an opinion on everything, but uh, it's, it's an exciting time in the space. And since we last talked to your point, we've got, we've got Disney out, we've got Apple plus out, we've got some new announcements from, from NBC regarding Peacock. We've, got a lot going on in the industry right now. A lot of confusion as well though. Dror Gill: 02:11 So let me ask you, when you, when you to watch some TV in the evening, can you really focus on the content or are you always looking for kind of artifacts, HDR levels? You know, stuff like that. Dan Rayburn: 02:26 I really don't want to think about the business because I do so much of that when I am reviewing the services from a business or content standpoint, you know, to your, to your point in terms of yeah, I am constantly looking at bitrates. I am looking at, okay, what's coming through my router because I want to see what the maximum stream is that I'm getting from the Mandalorian. You know, I probably have 40 different streaming services here at home and I've got anywhere between 10 and 12 TV's set up. Just sort of a lab environment and plus all the iPads, iPhones, MacBooks, like it's ridiculous, like a Best Buy here. I'm not the average consumer obviously, but, uh, I think like the average consumer in many cases, we are all looking at where the content is. So I've got some friends who, huge Rick and Morty fans and the new Rick and Morty season is out and this and that. Dan Rayburn: 03:10 And I said to him, you know, Hey, next year you're going to be able to stream this. And they're like, yeah, but I can't figure out where. Well, that's a great point. They can't figure out where, because where it's currently is and where it's currently at right now is going to be removed because AT&T has said that this is going to be exclusively under the new HBO Max brand. The average consumer isn't going to know that. So we're still going to have content fragmentation problems. So as a consumer, I think that's the biggest thing that we look at is just what content do we want to watch? Mark Donnigan: 03:39 You bring up something really interesting Dan, and this is a huge hole that I see in reporting on all the new services. It seems like so much of the press is writing about, you know, this service killing the next service. Dan Rayburn: 03:53 The problem is, look, the problem is the vast majority of people who are writing about our industry don't actually use the product. Mark Donnigan: 04:00 Yeah. They don't have 12 TV set up in their house, you know, like you do. Dan Rayburn: 04:03 They don't have one. All of these major platforms that are either telcos, carriers, wireless operators, content owners, distributors, whatever, whatever you want to call them, they're all creating brand new digital platforms for the future. And by that I mean this, when you think of what's taking place in the market right now with mergers, Viacom, CBS, Pluto, right, CBS All Access, CBS sports, CBS news, they are now all going to be converging and building out a new platform for all of these different products and services. That's one. Now throw in NBC sports, NBC news, Playmaker, Peacock, a, what do they also own? New England sports network. One of the other sports things. Throw all those guys in. That's now a brand new stack in the ecosystem. Now let's move on to AT&T. AT&T, Warner, Turner, HBO Max. That's now a whole system. Oh, and I forgot SKY when you're talking about NBC, you got to throw SKY in there too. So think about some of the largest companies that we have out there that are now creating a brand new stack end-to-end to fuel all these different new properties that they have. The biggest thing that you need there when you do that is what? Expertise, Mark Donnigan: 05:22 there might be some cues here because of course news just came out literally a couple of days ago. Fox signed with AWS a a very large deal. Now, um, I was reading some analysis on this and you know, it's because 21st Century Fox when they were acquired by Disney, you know, so there was a split, right? So the studio was acquired by Disney and all of those technical services actually went with you know, 21st Century Fox and of course then being a part of Disney. And then with BAM, now you've got this huge, you know, service organization that's available. And then here was Fox, the TV studio, the sports, you know, the sports side of Fox that needed a complete, you know, service provider. And it appears that they have selected AWS for even more than just, you know, on demand instances. So it's even more than a data center play. Um, and, and so that would seem to give credence to what your saying that, you know, BAM is far more than just a streaming service that, you know, there's, there's a lot of technical expertise and services they're providing Disney. Dan Rayburn: 06:36 Yeah, there's a huge amount and people, you know, really don't understand. I think a lot of people, even in our industry, don't understand what goes into all these services. Just the amount of beacons that are deployed, right. Just the amount of APIs you have to check. All the QoS and QoE reporting that has to come in and the analytics. And that's before you're doing any advertising. So anything advertising based obviously has more complexity tying into all the ad flows. And if you're doing live, okay, now you're talking about stream stitching for inserting ads into a live stream- that adds complexity. You have to think about latency and different ways to do chunked encoding. There's things you can tweak with HLS. There's just so much going on with these workflows and platforms that you really have to have that expertise. And some companies, you know think of Discovery, right? Dan Rayburn: 07:19 We heard from Discovery six, seven months ago when they announced they were going to hire 200 people to build a new streaming department to run all of Discovery's properties. So in some cases you have companies like that go, we want to own this, we want to build it, we'll bring it in house and it'll take them some time to get to market with that expertise. But they'll get there. And then you have other companies like Fox here where they signed that deal with Amazon and you know what they're really using AWS for is a couple of different things on the Cloud Front side, it's to deliver Thursday night football. Amazon already does live football. They kicked off the Premier League a what, two days ago? Three days ago from when we're talking now. So Amazon obviously has expertise in live streaming. The Premier League went off well with no major hitches. Dan Rayburn: 08:01 You did have some users complaining about latency, but that wasn't a problem Amazon was trying to fix just like we saw with the past Superbowl. That wasn't something where they were like, okay, we want to get latency to the same as broadcast. That was not the goal. So I don't see that as a problem. So they're using AWS for video workflows, editing and graphic storage, but also for this new product AWS calls Local Zone that puts cloud computing hardware closer to the edge and the edge is a broad term. Netflix has also signed on to be one of one of the first customers for this new AWS Local Zone service as well. So, it's super important, you know, we as consumers, we all want a good quality service and we expect it and now we're paying for it. So today this is big business. We have tens of billions of dollars at stake. This stuff has to work. It has to be right. And there is a lot of pressure on these new conglomerates to make sure that the video workflows, they're building out work properly because it is truly the future of their business. Dror Gill: 08:58 We're done with experimenting. Now we need to show the money. Dan Rayburn: 09:01 That's right. And you're spending a lot of money to do this. Look at how much money Disney's lost so far just on Hulu and then the acquisition of BAMTech. But they've already said to investors, here's one, we're going to make it back. Here's where we're going to become profitable. So you saw AT&T do that in their, in their HBO Dan Rayburn: 09:17 Max day. And NBC just announced they're going to have an investor day in January for Peacock. We're in a different era. Mark Donnigan: 09:22 Now for, you know, almost the first time, what is being done in engineering and R&D, can actually move a stock price. You know, meaning that the decisions that are made, whether that's technology choices, um, you know, codecs, certain stacks, architectures... If It doesn't work, like the stock is gonna move. And when the stock moves, it has the attention of everyone, you know, up until this point, you know? Yeah. The tech blogs, you know, would, would "dis a service" for an outage or for, you know, poor quality or you know, so yes it would get coverage but it never moved a stock price. You know? Or maybe there was a one day blip and you know, but, but basically it was kind of a non-event. Now that is no longer the case. Right? Dan Rayburn: 10:13 The bottom line is you have to think about profitability. And it's interesting that we're talking about this at a time when, if you think about Uber and WeWork, and some of these other services, what are investors clamoring for now? Profits, forget all this Amazon model of getting big, fast and burn as much money as possible. Thank God we seem to be getting out of that from a investment standpoint right now and in the streaming space, even more so also, look who's getting into the space? AT&T I think right now is the most heavily indebted US company right now. I mean it's insane how much debt that they have. So you also have companies, some of these that are already very deeply in debt that investors want to see anything new that they get into where they're spending billions of dollars to do it. They better turn a profit pretty quickly. Dror Gill: 11:03 But, but uh, Dan, let's look at the other side of the coin. A company that has tons of money, um, in the, in the bank and now they need to find some creative ways to use it in order to get those profits, uh, coming in again. And of course we're talking about Apple. Um, after selling a, you know, so many devices and now they, they've realized that services would be a much larger part of their revenues moving forward. So they, they really in a, in a spending mode and uh, the real question is will they be successful in catching up to the existing services and competing with all this new stuff that is coming out? Dan Rayburn: 11:47 Well, see I don't think they have to catch up though. That's the difference cause their, their business model is different. That's the other thing is people don't look at the business model of these services. You know, if you think about Apple services revenue, it was twelve and a half billion dollars, um, in the last quarter, which is pretty amazing. Their services business grew 13% year over year, so they're certainly doing a good job there. And Apple TV Plus, you know, the whole deal of that is just drive more usage on Apple's platform and services. But the unique thing with Apple of course is, well they own the hardware as we know, but they also own the OS. They own the browser, they own the store, they own the entire ecosystem. What does Netflix own? They don't own anything except content, right? So it's two different business models and everybody throws these, these folks in together and people go, Apple didn't have a successful launch. Dan Rayburn: 12:38 Well they did. They weren't trying to license back catalog. They weren't trying to launch with a hundred shows. That wasn't the goal of their platform because they're driving revenue in different ways. So it's the same way right now that Roku doesn't make a lot of money on their hardware, their seeding it out in the market to obviously drive the advertising business and the Roku channel, you know, the platform business. And Amazon pushing out Amazon fire TVs is what, $20 on black Friday for those sticks. They're not making much money in that either. So I think it's always bad when you see all these services compared to one another in the media and this horrible term streaming war because it's not a war- hate that term. Uh, and a lot of these services are not competing with one another. They don't see each other as competition. Apple is not trying to do the same business model as a Netflix, nor do they need to because it's a different type of company. Mark Donnigan: 13:31 It's excellent you brought up Roku. I'm looking at their Q3 numbers. They just came out like three weeks ago, um, or early November, I believe. And they're advertising revenue for the period was just under 180 million, 179.3 million. It was up 79% from the previous year's quarter, almost double and their device revenue was up 11% so that's good. But it was 81 million. So the point is their advertising was more than double their device revenue, you know, and their, and their numbers are showing on the advertising platform side, you know, just tremendous growth. And of course that's ultimately what they're really reporting around. I mean, yes, their device revenues are significant enough, you know, they're reporting that. Dan Rayburn: 14:22 Yeah, they shifted their business model. Right. I mean Anthony was smart. Keep in mind, Anthony came out of, came out of Netflix, that's where the Roku was born. Yeah, it was incubated there. Initially. Right. And that's where they got some of the money from and, and they realized longterm, I'd say two things were really smart. In the beginning Roku realized Netflix realized they didn't want to be invested in any one hardware company because then they couldn't be Switzerland. They couldn't be neutral. So that was smart to diverse, diversify from the Roku investment that they have. But then Roku also realized, they were smart to realize the writing on the wall here, you're not going to compete longterm on the hardware side. Hardware pricing always gets pushed down and back then if you remember all the different devices, I mean at one point we had 20 different streaming players in the market. Dan Rayburn: 15:03 It was ridiculous how many were out there. Even Vizio had one. Uh, but then really, I think what changed was when Amazon came into the market. Because we all know Amazon pushes pricing, pricing down on everything and we're, we're at a point soon of where I, I, this isn't official, Amazon hasn't told me this, but I will pretty much bet anything that at some point you're going to sign up for prime and you're going to get a stick for free because at $19 now on black Friday, this thing is getting close to being free. And if you're in the hardware business, do you want to be competing with Amazon on something like that? Absolutely not. So Roku realized that Roku had to become not a hardware device, but a platform. And the key thing there was obviously them getting their platform into smart TVs and especially a lot of smart TVs that are not the high end ones, not that TCL, doesn't make some good "high-ender" TVs. Dan Rayburn: 15:54 But you know, the average Roku enabled TV that's being sold is probably $300. Hisense, TCL, some of the others. So they're getting more of them out there. And, and that's really what Roku has become is that platform and their, their latest acquisition of Dataxu. You know, that's interesting because that is a platform that basically will allow Roku advertisers to better plan and optimize their ad spend across TV and OTT providers. And, and that's really smart of Roku. Uh, because this is the future of the company. You're talking about a company that's doing over a billion dollars a year now, in 2019, if I remember that number correctly. So you have to think about how Roku can capture a larger share of the market because as well known as a brand that Roku is, they still have a very small percentage of total households in the U S when you look at the numbers of, they don't call it consumers anymore, devices. Dan Rayburn: 16:55 Um, you know, which is good because like I have a bunch of devices in my house, but I'm one person. So they're growing, but that's something that they have to continue to do. Their monthly active users has to continue to go up. But yeah, Roku is in a really interesting spot in that regard. Their, their stock is incredible in terms of how much volatility it has in any given day or week. Sometimes. Uh, I think the Roku channel is an interesting thing where, you know, they go out and they're starting, they start offering content for free just like Tubi and Pluto and you know, IMDB TV by Amazon and that market is getting very crowded. And frankly, I don't quite understand that market because the content on those platforms is just so old and outdated. I really don't know who's clamoring to see Gilligan's Island. Mark Donnigan: 17:37 Well, Dan, so how should services be measured, you know, from a QoS standpoint? Dan Rayburn: 17:43 Uh, boy, that's a great question. Uh, I think first and foremost you have to look at what the methodology is. Methodology is the key for anything. So, you know, as an analyst, I don't frankly care about opinions so much. I care about data. I think companies should base how a service is doing, whether that's financially, whether it's technically, whether how it's scaling. They should base that on data because data can't be argued with really in most cases. Uh, so I, I think first and foremost is the methodology. And I think what you have to understand there is different companies have different ways of measuring performance. When I go out and do surveys to CDN customers on how they measure, some go, I care all about time to first frame or startup time. Others go, no, I only care about rebuffering. Some go, well, to me latency is most important. Dan Rayburn: 18:25 Well, none of those are more important than the other. It depends on who the customer is. And what their business model is. So as an industry we have to continue to think about these services as, as isolated services as opposed to every throwing everybody in this group of, Oh, you're a video service, you should measure your video quality this way. Not necessarily. So I think methodology first and foremost is most important. I think sharing that methodology is key as well. Uh, but, but I think you should always value a service based on quality over quantity. And we hear that a lot. The opposite of that in the advertising side where everybody talks about how many ads were delivered. But the question I always then ask is to a brand, would you rather deliver fewer ads and have a better viewing experience or do you just care about how many ads you pushed out there? Dan Rayburn: 19:16 And we have to think about that the same way on services that are not ad based. So I think what we obviously know from consumers from all these reports that we've seen, and frankly I don't think we need any more. I don't know why people keep pushing out more reports saying that if the video doesn't start up quickly, consumers are unhappy. Yeah, thanks. We know that. I think measuring quality has to first and foremost come down to what is the experience that you want a consumer to have with your content. That's the first thing. Once you define that experience, now how do you actually decide how to achieve that? Well, there's different ways to do that. We know that some of the basic ones are startup time. We know that customers get frustrated when something takes long to start. We also know rebuffering is a huge issue as well, which is obviously why we use adaptive bit rate encoding hopefully to relieve those issues. Dan Rayburn: 20:03 But it's interesting when you look online you don't see a lot of complaints honestly around rebuffering you see more with just initial startup time, but the biggest complaint you see actually doesn't have to do with the video. It has to do with just getting to the video. So you're having all these other issues in the stack before it actually gets to delivering the video bits and those are the things that really have to be solved. Those are the things that really have to be scaled because scaling the video is not that hard for someone like Disney Plus. Disney Plus launches that day, let's say it was 10 million actual individual subscribers and let's say they were all watching at the same time, 10 million streams across the five CDNs that Disney was using. That's not a big deal at all. It's 2 million streams a CDN, that's nothing. That's not hard, so people always think it's the CDN. Dan Rayburn: 20:56 I think when you're determining quality first and foremost you have to have a good understanding internally at your company, what you think good quality is to you for your service based on your business model, based on your consumers and also based on the type of device they're watching on is the vast majority of your content on mobile. And the reason I say that is as an example, when Quibi comes out next year, it's a hundred percent mobile focused. Do you think their methodology to measure quality should be the same as a Netflix? Because we know everything's going to be viewed on a small screen in short form content for Quibi. It's a different way to measure. I think there's lots of good services out there to help you measure there. There's, there's newer ones coming to the market in terms of what's being measured. You've got services that are measuring how well API's are doing versus how well streaming servers are doing versus ad servers and ad platforms and exchanges. Dan Rayburn: 21:43 And then you think of their traditional stuff that's been out there in terms of telcos and carriers, last mile providers, how they're doing transit providers. When you put all that together, it gives you a much better holistic view of what QoS looks like across the internet from end to end, from glass to delivery. Uh, but we still have a ways to go in terms of really showcasing that. And unfortunately none of these companies after the fact ever share any sort of methodology and they don't ever share any kind of numbers. You know, I worked on those Superbowl was CBS this year and I can't talk to the, you know, the numbers. I know, but you know, it's too bad. CBS doesn't put out from their Conviva dashboard and Mux and all the other services being used here was the rebuffering rate because you know what, it was really, really, really low. Like why not put that out? It shows a great quality service. Mark Donnigan: 22:32 You made a good point earlier that it's very interesting that now, all these big companies are actually staging investor days, or investor conferences around their services, which is like has never happened previously. I wonder if this methodology is going to begin to make it in, you know, to some of the public disclosures, you know, in some way? Dan Rayburn: 22:55 Sounds great. But, come on, if you deal with investors, you know that you start talking even bit rate calculations with them and they can't figure it out. Right? I mean, so no, investors aren't worried about that stuff. They don't understand it. Um, I mean it's amazing how many people just just on LinkedIn alone, let alone the media, was comparing the success of Disney Plus based on the metric of when Netflix launched and it just, it boggles your mind, right? Because I stuck up on LinkedIn just real quickly, and this is all factual information you can easily look up, which you know, the media doesn't want to do. The year Netflix launched, there was only 34 million iPhones in the market. That's it. Now, smart TVs didn't exist at all. And two years later, in 2012, only 12 million were connected to the internet. And at the end of the first year of Netflix, Apple had sold 7.5 million tablets. So now you're going to compare Disney Plus launching in an era with over a billion iPhones alone and I don't know how many Apple iPads, smart TVs, and you're going to compare that and go, we've now deemed this a success because it's beaten something that launched nine years earlier. Yeah. The methodology is flawed, and forget bandwidth. I mean bandwidth back then compared to now. It's night and day. Mark Donnigan: 24:22 I was there. I was there in 2007 we were just launching VUDU and you know, on a dedicated set top box because that was the way that we could bring a guaranteed experience to the home. You know, it wasn't because, you know, VUDU wanted to be in the hardware business. Uh, and ultimately, you know, the company of course pivoted, you know, to an app on devices. But um, I can, I can remember having to think that that the average broadband capacity in the US in most markets was around two megabits. Dan Rayburn: 24:57 It was a different time, comparing something that long ago. But here's the biggest thing. The media doesn't write for accuracy like we talked about before. They write for one thing, headlines. So the moment you say this kills Netflix and this crushes Netflix or this did better than Netflix, what happens? People click on it because everybody's heard of Netflix. Cause the only way these guys make money is page views. So that's a whole different discussion. We're not going to get into, cause that's a whole different podcast. But the entire model for news on the internet is broken. And has been broken for years. When, it's based on just here's how many page views you have. So let's cram out more articles that are 800 words or less instead of actually telling us. Mark Donnigan: 25:38 So I think it's a interesting, you know, to talk about devices and since we are talking a little bit about history now, you know, there was a time where it was really critical that you got your service on a device and I'm kind of, you know, using "air quotes" there. Um, because if you were on a device that was widely sold, then you, you know, you had, um, you had an ecosystem you're a part of now with SDKs and API APIs and, and it's far more ubiquitous, you know, HTML5 apps and things like that, you know, with the app stores being clearly defined. Um, you know, basically you need to be in the, uh, Apple app store. You know, you need to be in the Google, uh, store, you know, for Android. Um, you need to be on about half a dozen connected TV platforms and then Roku and you've covered like 99% of the market. Right. Um, so what's your perspective of, you know, even like Nvidia launching, you know, the, Shield TV. Dan Rayburn: 26:43 And you know, just the role of devices. What are you, um, uh, you know, what are you seeing there? Well, you know, I think over time devices play less of an important role. And the only reason I say that is to your point, it's really about the platform now and it's about ecosystems and people pick certain devices or services because I'm already in the Apple ecosystem already. And the Android ecosystem, I already have a, you know, an Xbox one. Typically people who have an Xbox one is they're not going to then go out and buy a PS4 just because of a new service. So what we've seen over the years is no longer have services launched with exclusives on platforms. Like we saw when HBO Now launched, it was only available for the first 90 days on Apple TV. That's actually a disservice to the service. Dan Rayburn: 27:28 It's getting in fewer people's hands. So I think the devices we have in the market, I don't see that changing at all. Right. I think you have the major devices between Xbox, PlayStation, Chromecast, Apple TV, Roku, Amazon. Uh, who am I missing? Those are the seven major ones. I look at something like the Shield TV, which now has two new models from Nvidia, which I've, I've tested and played with. Yeah, it's a good device cause it's super fast. And the fact that it's built on Android, you know, you, you can go in there and you can install a Plex server on it, which works really well. It's a great device for Plex media server. Uh, but who's the video really targeting with the device? It's $200. Dror Gill: 28:09 People who like a nice design. I mean look at the shield TV. It's a cylinder shape. It looks exactly like the Roku Sound Bridge come to think of it. Dan Rayburn: 28:18 So the lower end model does, that's the one that's $149, the $199 model, which has storage in it and two USB ports. The original one you're talking about has no USB port, so you can't add additional storage, which is kind of a problem. Uh, you know, $200. Your really targeting the person who wants to build something at home. The enthusiasts, right? That's who you're targeting. I think that's great. Like there's nothing wrong with that, but I, you know, I questioned like, is that Nvidia's core business? No, it's not. But since they're making the chip inside, I get it. Their cost to produce that hardware is probably much cheaper than others because they're not paying for the chips since they own it. Um, but I don't think the hardware changes going forward. I, I do think we've seen an amazing amount of progress with smart TVs over the last five or six years. Dan Rayburn: 29:04 They actually work. Um, if you remember five or six years ago, you never wanted to launch an app on your smart TV cause you didn't know how long it would take a load. Now they work really well. They're pretty seamless. I mean, the new LG device that I just got the remote's really well thought out. It's smart. Uh, it's clean and simple. There's not a lot of bloatware on it. That's the other thing is a lot of these smart TVs used to have so much bloatware, especially Samsung, they've gotten much better at reducing that with removing what used to be mandatory ads. So I think the smart TV has gotten much better there. And I think for a lot of people that continues to be a device that grows down the line because it's all integrated into one. And that's also part of the, the reason Amazon came out with the cube and now the second generation cube, you know, really cool device that is voice-based and will automatically, when you say turn on Hulu, will know how to change your input know how to turn on your TV. It can also control your lights. We're starting to see more streaming services on these platforms that are being combined into the connected home. Dror Gill: 30:05 Right. And you see this with a, with Nvidia shield TV, right? It connects to your, uh, um, uh, nest to the Phillips Hue, to Netgear, all of that. Dan Rayburn: 30:14 I think that that's the future where some of this is going is they're no longer these companies and platforms and no longer looking at streaming services as an isolated service. It's one of multiple services in your house. It provides entertainment or lighting or something of that nature. And the Cube is a really cool device. I've spent a lot of time with the Cube. Um, we recently at the NAB streaming summit in October, we had one of the executives on stage doing a fireside chat with me. Really talking about the technology that went into it. And audio is really hard and I don't think people understand in the audio side just how hard it is to do things on the voice side and actually have it work on the back end and have it worked quickly and in real time. Uh, I would say right now Amazon is by far leading the market when it comes to the technology that they have for voice enabled applications. And you see that with the Cube, especially from first gen, the second gen, and on black Friday the price was down to $90. What do you think is going to be next year? Right. It's probably going to be 70 bucks, you know, just keep dropping. So yeah, I think that's pretty neat to see in our industry, just how streaming is now thought of as a one of many things in the home that we're using for entertainment. Yeah. Dror Gill: 31:24 And, and people are using voice actually they got used to talking to their devices? Dan Rayburn: 31:28 Well, from what we're hearing and the data we've been given. Dan Rayburn: 31:30 Hulu at the show said that uh, people who were using voice to find content tied into Amazon's products were watching 40% more Hulu and it makes sense because people know how to use their voice and they know what to say. When you're doing a search in, um, one of these services, do you put in the title? If the title is not perfect, what you put in, do you still get the right results? Many times? No. Whereas with your voice, it's much more natural in terms of how you're going to search for content. Dror Gill: 32:00 The LG remote, you mentioned earlier, it has like a single button. Then you talk to the remote and it automatically searches on all the applications that you haven't stalled on the TV and finds the content very simple. Dan Rayburn: 32:12 Also, if you don't want to do that, the pointer system's very simple. If you don't instead want to have to type stuff in, they give you flexible options, which I like as consumers, we will all want options and I think options are good. The downside to options obviously is too much choice, too much confusion, not sure what the business model is. And that's why a lot of consumers are going to jump amongst these services in 2020 because when you can try them for a week or 30 days, why wouldn't you? Mark Donnigan: 32:38 Well, Dan, I know you were telling us before we started recording about something really exciting you're doing at the NAB show, um, around devices. So, um, why don't you tell us, you know, what you got planned. Dan Rayburn: 32:52 Yeah. So this, this is pretty cool. Um, and we're going to have some, we're going to have some information on the website up pretty soon and you'll see me announce it sort of everywhere. Dan Rayburn: 32:59 But one of the problems I've always seen at conferences talking about our industry is we're all there talking about video, but nobody is showing it. We're talking about devices, but nobody's getting hands on with them. Nobody can see these platforms in action. And the three of us on the phone, we eat, sleep and breathe this industry. So we see all this stuff. We use all this stuff, but we're not the average consumer. We're not the average industry participant. So my idea here was the NAB show is, is the largest collection of people in the video world. Maybe not all streaming, obviously a lot of traditional broadcast, but those are the people we actually have to educate even more than people in our industry. So what we're going to do in April is for anybody who walked into the North hall lobby, if you remember, there wasn't really much in the North hall lobby. Dan Rayburn: 33:45 There's some little booths and some other things. Well, we're going to take over the North hall lobby and we're going to call it the streaming experience. And we're building out 12 living room style, uh, seating with large screen TVs. And every single TV in all 12 locations is going to be XBox, PS4, Apple TV, Chromecast, Roku, uh, what did I forget? It's basically gonna be every hardware device in the market today of the seven that we talked about earlier. And then on each one of those, there's going to be 50 different OTT platforms that you can test and these will be pay services, these will be AVOD services, these will be authenticated services. Think like a CBS sports or something like that. And any attendee to the NAB show can walk right in and say, you know, I really wanted to see what Netflix, HDR looks like here compared to you know, Amazon HDR or I want to see what bundling of content looks like. Dan Rayburn: 34:41 I want to see what UI and UX is compared to these services. I want to see how the ad supported services are doing pre-roll. I want to see what live sporting personalization looks like. I want to actually test an Amazon Cube and see how good it is in terms of understanding voice recognition. So we're calling it the streaming experience. We're going to have it out for three days. It's going to be a place where people can also just come to get questions answered about these platforms. I'm going to personally have my folks manning every single one of the stations. Uh, and in addition we're going to be giving away every single piece of hardware that we are installing during the event. We're going to be giving that away after. So, it's about $10,000 in gear, not including the TVs, which those are rentals, but everything else, uh, that we're buying, we're going to be giving away. Dan Rayburn: 35:33 So you're going to be able to get into some amazing raffles, some really good gear. And then in addition to that, we are also going to have a location in the middle of that area. The streaming pavilion, Oh, sorry. Streaming experience where you're going to be able to also test these streaming services on phones and tablets. Oh, that is awesome. And because we have to bring that experience in as well we can't only think large screen and if all works out, hopefully we might even have 5G demos. So these services working across 5G. So think of every service in the market, you know, all the live linear services, the on demand services, the free services, the authenticated services. I basically challenged people to come to the streaming experience and find a service that we don't have on on those devices and we'll have, we will have services from other countries. Dan Rayburn: 36:26 It's not just going to be the US I won't have everything. Obviously there's, there's some of these services that only work based on certain geo-fencing and certain locations. But we also already have some OTT providers who were saying, Hey, we're going to give you special accounts so that the services work for you as a demo even if it's not available in that region. So we have a lot of OTT companies that are working with us. We've got some that are partnering with us on a sponsorship level to really promote the service. And the other thing we're going to do is for the companies that really wants some feedback, we're going to have a, an attendee who comes up and let's say they use Hulu's service for a couple minutes and then they walk away before they walk away. We're going to say, Hey, fill out this quick card that has five questions on it. Dan Rayburn: 37:09 Would you buy this feature functionality? And then we're going to dump all that data back to the OTT platforms. Because now they're going to collect thousands, hopefully of real world feedback from customers who are using the service or thinking about using the service. So we want this to become a focal point for the show where people can come and just talk about these services, see them, compare them, test them. Win Some of this product, uh, get your questions answered. And then also use it as a way to collect data for the industry to share with the platform providers what is actually taking place. So I don't know of any other show that's doing it. It's something that I've been wanting to do for quite some time at this size and scale. And when you have the NAB behind it and once they start promoting it and we've got dedicated bandwidth for it. Dan Rayburn: 37:55 So we're making sure the experience is really good and I'm curating the entire thing so I am going to make sure everything works beforehand. We're there days in advance, I've already bought all the devices for the, for the event for months prior, right were we had them like it's about 2,600 accounts you have to set up across all the devices. It's a big undertaking. This is, this is serious, but it's going to be a good as we're calling it experience. So whether you're in the advertising market and you want to see what ads look like or you're in the compression business and you want to look at artifacting from one service to another, you want to look at 4K and lighting and HDR. You want to come. I think UI and UX is super important. So all those people that come to the NAB show that are doing design or creative UI and UX will come compare how they work and work between mobile and larger screen. So really whatever industry you're in and the NAB gets a lot of different people from different verticals and industries and regions of the world, this is going to be relevant to you in some way, shape or form and you're going to be able to see it free of charge. Dror Gill: 38:59 This really sounds amazing Dan. It's kind of a combination of a, of a playground that everybody wants to play with and also a way to experience, uh, all of this tests, right? And, and the way to experience a lot of things that you don't have access to because nobody can buy all of that gear and get access to all of those services at the same time. So you can really come in and experiment and see video quality as you said, UX, advertising, integration, everything. And also be able to talk to people who are, who are experts in this and can walk you through it. And the fact that you're feeding back the information and the comments from, uh, from the visitors, you know, back to the services is, is really a great service to the industry because then you can finally get those comments and uh, and information back. Dan Rayburn: 39:49 And we're also going to share it with the industry as a whole. We're definitely going to share here are some of the highlights we've seen from what consumers have been saying. And the other way I'm looking at this too is it educates two other portions of the market that are really important. It educates the media because now it's going to happen is when somebody wants to do an interview with Hulu who speaking at the show and you know, wants to talk about the platform. Somebody from Hulu is going to be able to walk them to the streaming experience and actually show it to them, which means hopefully they actually get the coverage accurate. So it's really important that the media sees the stuff. And second, the other market that we have at the show is investors. There's a lot of investors at the NAB show, institutional investors, and they don't get to see this stuff. Dan Rayburn: 40:29 So when they're making predictions about stock and about revenue and loss and capex and OPEX and all these other things that they use to determine success or failure of companies, the best way to do that is to actually see the product in action. So now you're also going to have investors who are going to be able to get hands on with this stuff even from a high level, which is going to benefit them. So I think overall it just benefits the industry. It benefits the platform providers, the consumers, the media, the investors. Those are really the five vertical markets that I'm trying to target. Dror Gill: 40:57 We need something like this. Um, you know, as an installation permanently somewhere. Dan Rayburn: 41:02 Yeah, maybe. I mean, I'm doing this with the NAB and that's, that's the exclusive, you know, group I'm working with now. I'm certainly not going to bring this to other conferences, but this is something that you're going to see now moving forward at NAB show in Vegas for sure. New York is much more difficult to do this only because of unions, some other, some other rules around that. But, uh, in Vegas, this is, you know, this is DnaB also planting a stake in the ground going, listen, you know, last year you walked into the North hall lobby and it was still so much of a focus on broadcast and traditional TV. Well, users are in for a, you know, wake up when they walk in this time and go, wow, what is all this streaming stuff? Mark Donnigan: 41:38 This is an amazing service that you're providing Dan. Uh, and we're gonna promote it and encourage everyone, uh, you know, our customers and those that are in, you know, in our sphere of influence, uh, to check it out, you know, really, cause this is, this is amazing. Dan Rayburn: 41:52 I'm excited for it. It's a lot of work and it's a huge undertaking. It is a lot of work. Yeah. It scares me at times. Just cause to do it right. It's, it's a lot of work. Um, but I'm going to have a good, I'm going to have a good team. I'm going to be flying in some, uh, some of my buddies from the special operations community who are, who are tech guys and they're, they're going to come help me in the booth and whatnot. And, uh, it's, it's going to be a good three days. Well, Mark Donnigan: 42:18 Dan, uh, this is, uh, you've been yet another amazing interview. Thank you so much for coming on the video insiders. Dan Rayburn: 42:26 Thank you for having me again. As you know, I can talk all day about this stuff. So it's a good thing you have to edit this down into something shorter. Mark Donnigan: 42:30 The next time we have you on, uh, I think, uh, will time, the timing will be good with some new, uh, things you have going. Dan Rayburn: 42:41 There'll be some other new things in the new year that I can't talk about now, but yeah, yeah. The, the, the idea of wanting to inform the market more and providing more resources for the community. That's, that's something that's coming up. Dror Gill: 42:51 Great. So thanks again. Thanks again for joining us today. Dan Rayburn: 42:54 Thank you guys. Announcer: 42:55 Thank you for listening to The Video Insiders podcast. A production of Beamr Imaging, Ltd. To begin using Beamr's codecs today, go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Download: Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff published by Technion–Israel Institute of Technology, Haifa, Israel. Authors: Yochai Blau , Tomer Michaeli.Today's guest: Tamar ShohamRelated episode: E32 - Objectionable Uses of Objective Quality MetricsThe Video Insiders LinkedIn Group is where we host engaging conversations with over 1,500 of your peers. Click here to joinLike to be a guest on the show? We want to hear from you! Send an email to: thevideoinsiders@beamr.comLearn more about Beamr's technology TRANSCRIPTION (Note: This is machine generated and may have been lightly edited)Tamar Shoham: 00:00 No matter which application or which area of video compression and you know, image compression, video compression that we're looking at.Tamar Shoham: 00:08 There is finally a growing awareness that without subjective testing you cannot validate your results. You cannot be sure of your quality of the video because at least for now it's still people watching the video. At the end of the day, we don't have our machines watching, at least not quite yet.Dror Gill: 00:42 Hello everybody, and welcome to another episode of The Video Insiders. With me is my co-host Mark Donnigan. I'm really excited today because we have a guest that was on our podcast before today. She's coming back for more. So I would like to welcome Beamr's, own VP of Technology Tamar Shoham to the show. Hi Tamar. Welcome to The Video Insiders again.Tamar Shoham: 01:06 Hi Dror, Hi Mark, great to be here again.Dror Gill: 01:08 And today we're going to discuss with Tamar a topic which has been a very hot lately and this is a topic of a video quality measurement. And I think it's something that's a very important to anybody in in video. And we have various ways to measure quality. We can look at the video or we can compute some some formula that will tell us how good that video is. And this is exactly what we're going to discuss today. We're going to discuss objective quality measurement and subjective quality measurement. So let's start with the objective metrics and Tamar can you give us an overview of what is an objective metric? And what are the most common ones?Tamar Shoham: 01:55 Fortunately the world of video compression has come a long way in the last decade or so. It used to be very common to find video compression evaluated using only PSNR. So that's peak signal to noise ratio, which basically is just looking at how much distortion MSE (mean square error) there is between a reconstructed compressed video. And the source. And while this is, you know, in a very easy to compute metric and it does give some indication of the distortion introduced its correlation with subjective or perceptive quality is very, very low. And even though everybody knows that most papers I'd say up till about a decade ago started with, you know, PSNR is a very bad metric, but it's what we have. So we're going to show our results on a PSNR scale. I mean, everybody knew it. It wasn't a good way to do it, but it was sort of the only way available.Tamar Shoham: 02:58 Then objective metrics started coming in. So there was SSIM the structural similarity which said, Hey, you know, a picture isn't just a group of pixels, it has structures and those are very important perceptually. So it attempts to measure the preservation of the structure as well as just the pixel by pixel difference. Then multi-scale SSIM came on the arena, sorry. And it said, well, it's not only the structure at a particular scale, we want to see how this behaves on different scales of the image. So that's multi-scale SSIM and it's, it's actually not a bad metric for getting an impression of how distorted your video is. Netflix did a huge service to the industry when they developed and open source their VMAF metric a few years back. And this was a breakthrough for two reasons. The first is almost all the metrics used before to evaluate the quality of video were image metrics. And they were actually only measuring the, the per image quality. We're not looking at a collection of images, we're looking at video. And while there were a few other attempts, there was WAVE I think by, by Alan Bovik's group and a few other attempts.Dror Gill: 04:28 So VMAF basically takes several different objective metrics and combines them together. And this combination is controlled by some machine learning process?Tamar Shoham: 04:40 VMAF was a measure that incorporated a temporal component from day one. So that's one place that really helped. The second place is when you're Netflix, you can do a lot of subjective testing to verify and as a part of the process of developing the metric and verifying it and calibrating it and essentially the way they did it by using existing powerful metrics such as VIF and and adding as we said, the temporal component and additional components but then fusing them altogether. That's where the F from VMAF comes from. Fusing them together using an sophisticated machine learning neural network, a base model. So that was a big step forward and we now do have an objective measure that can tell us, you know, what the quality of the video is across a large spectrum of distortion. They did a large round of subjective testing and a graded the quality of distorted videos using actual users. And then they took the results of a few existing metrics. Some of them were shaped slightly for their needs and added a pretty simple temporal component and then took for each distorted video the result of these metrics and essentially learned how to fuse them to get as close as possible to the subjective MOS score for that data.Mark Donnigan: 06:19 One of the questions I have Tamar is the Netflix library of content that they use to, to train VMAF, you know, entertainment focused, kind of, you know, major Hollywood movies, but there's things like live sports. Does that mean that VMAF works equally well, you know, with something like live sports, which I actually don't know, maybe they trained, you know, Netflix trained, but that's certainly not a part of their regular catalog. Or do we know if there's some content that, you know, maybe it needs some more training, or it's not optimized for?Tamar Shoham: 06:54 Yeah. So, so Netflix and being very upfront about the fact that VMAF was developed for their purposes, using clips from their catalogs and using AVC encoding with the encoder that they commonly use to create these clips that were distorted and evaluated subjectively and used to create the models, which means that a, it may not apply as well across the board for all codecs or all configurations. And all types of content. That's something that we actually hope to discuss with Netflix in the immediate future and maybe work together to make VMAF even better for the entire industry. Another issue with VMAF, and it's interesting that you know that you mentioned live in sports is that it's computational complexity is very high. If you are Netflix and you're doing offline optimization and you've got all the compute power that you need, that that's not a limitation.Tamar Shoham: 07:56 It's a consideration and it's fine. But if you want to somehow have a more live feedback on your quality or be able to optimize and evaluate your quality with reasonable compute power VMAF is going to pose a bit of a problem in, in that respect. In any case these are all objective metrics and as I said, you know, they go from the bottom of the scale in both performance required to compute them and reliability or correlation with subjective opinions and up to VMAF, which is probably today top of the scale for a correlation with subjective quality. But it's also very heavy to compute. But all of these metrics have one thing in common. Unfortunately, there are a number, they measure distortion. They're not a subjective estimation or evaluation of perceptual quality. That's a good point. Yeah. So I recently had the pleasure of hearing a very interesting PhD dissertation by Yochai Blau at the Technion under the supervision of professor Tomer Michaeli.Tamar Shoham: 09:10 And the title of his work is the perception distortion tradeoff. And what he shows there is he shows both experimentally with two sets of extensive experiments that they performed and mathematically using modeling of perceptual, indication of quality and statistical representations for that versus the mathematical model of various distortion metrics. And he shows in that work that it's sort of mutually exclusive. So if you're optimizing your solutions specifically, for example, a neural net based image processing solution. If you're optimizing for distortion, you're going to have a less acceptable perceptual result. And if you're optimizing for perception, you're inherently going to have to slightly increase the distortion that you introduce. And there's like a convex hall curve, which finds this trade off. So mathematical distortion, you know, a minus B, no matter how sophisticated your distance metric is, is inherently opposing in some ways to perception. Because our HVS, our human visual system is so sophisticated and does so much mathematical operation on the data that the distance between the points or some transform or wavelit done on these points can never fully represent what our human visual system does to analyze it. And, and that's, I mean, a fascinating work. I think it's the first time that it's been proven mathematically that this convex hall exists and there is a bound to how well you're going to do perceptually if you're optimizing for distortion and vice versa.Dror Gill: 11:07 And I think we also see this in in, in video compression. For example, in the open source x264 codec and also other codecs. You can tune the codec to give you better PSNR results or better SSIM results, you can use the tune PSNR or SSIM flag to to actually optimize or configure the encoder to make decisions which maximize those objective metrics. But, it is well known that when you use those flags, subjective quality suffers.Tamar Shoham: 11:43 Yup. Yup. That's an excellent point. And, and continuingly that x264 and most other codecs generally have a PSY or a psycho-visual rate distortion mode. And as you said, it's well known that if you're going to turn that on, you're going to drop in your PSNR and your objective metrics. So it's something that has been known. You know what the reason I, I'm, I'm very vocal about this work at the Technion is it's the first time I'm aware of that it's being proven mathematically, that there's like a real model to back it up. And I think that's very exciting because it's something, you know, we've known for a while and, and now it's actually been proven. So we've known it, but now we know why! For the nerds among us, we can prove that if it's even mathematical, it's not just a known fact.Tamar Shoham: 12:28 This is coming up everywhere. And there's growing awareness that, you know, the objective metrics and perception are not necessarily well correlated. But I think in the last month I've probably heard it more times than I've heard it in the five years before that it's like there's really an awareness. Just the other day when, when Google was presenting Stadia, Khaled (Abdel Rahman), one of the leads on the Stadia project at Google specifically said that, you know, they were testing quality and they were doing subjective testing and they had some modifications that every single synthetic or objective measure they tried to test said there was no difference and yet every single player could see the difference in quality.Dror Gill: 13:21 Hmm. Wow.Tamar Shoham: 13:23 No matter which application or which area of video compression and you know, image compression, video compression that we're looking at. There is finally a growing awareness that without subjective testing, you cannot validate your results. You cannot be sure of your quality of the video because at least for now, it's still people watching the video. At the end of the day, we don't have our machines watching it for us quite yet.Dror Gill: 13:49 And Tamar, I think this would be a good point to discuss how user testing is done subjective testing, but actually it's, it's user opinion testing is done. I know there are some there are some standards for that?Tamar Shoham: 14:05 Right. So I think one of the reasons we're seeing more acceptance of subjective testing in recent years is that, originally there were quite strict standards about how to perform a subjective testing or visual testing of video. And it started with the standard or the recommendation by ITU, which is BT.500 which was the gold basis for all of this. And it defines very strict viewing conditions, including the luminance of the display, maximum observation angle, the background chromosome chromaticity. So you have to do these evaluations against a wall that's painted gray in a specific shade of gray. You need specific room illumination, monitor resolution, monitor, contrast. So it was like if you wanted to really comply with this objective testing of the standard, there was so much overhead and it was so expensive that although there were companies, you know, that specialized in offering these services, it wasn't something that a video coding developer would say, Oh, okay.Tamar Shoham: 15:16 You know, I'm going to test now and see what the subjective opinion or what the subjective video quality of my video is. And I think two things happened that helped this move along. One is that more standards came out, or more recommendations, which isn't always a good thing, but in this case, the newer documents were less normative, less constraining and allowed to do easier user subjective testing. In parallel, I think people started to realize that, okay, it doesn't have to be all or nothing. Okay. If I'm not going to do a rigorous BT.500 test, that doesn't mean I don't want to collect subjective opinions about what my video looks like, and that I won't be able to learn and evolve from that. At Beamr, we have a very convenient tool which we developed called Beamr View, which allows to compare two videos side by side, played back in-sync and really get a feel for how the two videos compare.Tamar Shoham: 16:23 So while the older metrics were a very, very rigorous in their conditions and it was very difficult to do testing subjective testing and confirm with these standards, at some point we all started realizing that it doesn't have to be black and white. It doesn't have to be either you are doing BT.500 subjective testing by all those definitions or you know, you're just not doing any subjective testing. And Using our tool Beamr View, which I presume many of you in the industry use. We often compare videos side by side and try to form a subjective opinion of, you know, comparing two video coding and configurations or checking our perceptually optimized video to make sure it's perceptually identical to the source, et cetera. And then the idea came along saying, okay, you know what if we took this Beamr View tool and added a bit of API and some backend and made this into a subjective test that was trivial for an average user to do in their own time on their own computer.Tamar Shoham: 17:27 Okay. Because if it's really easy and you're just looking at two videos playing back side by side and someone else is taking care of opening the files and doing the blind testing so you don't know which video is on which side and you just have to, you know, press a button and say, Oh, A looks better, the left looks better or the left looks worse. That makes the testing process very, very easy to do. So at this point we developed what we nicknamed VISTA. VISTA basically takes Beamr View, which is a side-by-side video viewer and it corresponds with a backend that says, okay, these are the files I want to compare. And the user just has to look at it and say, Hmm, I don't know yet. Replay. Hmm, yeah, a definitely, you know the left definitely looks worse. So I'm going to press that button and then you get fed the next pair.Tamar Shoham: 18:25 So we're making this visual side by side comparison really, really easy to do. And that was the first step to making large scale subjective testing, a reality that we could actually use. And if before you know, Oh gee, I've got two configurations and I want to know which looks better, you know, you have to go and pay a company to do BT.500 testing and get results two weeks down. Well now at least we had a tool that we could use internally in the company, get a few of our colleagues together and say, okay, you know, run this test session. Let me know what you think. And while it's true that this wasn't scaling yet, you know, so we would collect five or 10 opinions over our set of 10 or 20 clips that we were testing. We could always complete these evaluation with objective measures. But no matter how many objective measures you measure, okay, you're always going to get, for example, going back to the example we mentioned before, if you're turning on psy-RD, you can run a thousand tests. PSNR is going to be lower.Dror Gill: 19:35 Yeah. And that's the problem. I mean, the advantage of objective metrics is that they can scale infinitely, right? You can run thousands of servers on AWS that would just compute the objective metrics all day, but they're not very accurate. They don't reflect real user opinions. And on the other hand, if you want to run subjective testing or user testing at scale, you have a problem because either it costs very much, you need to go to those dedicated labs. Or if you do it internally, you know, with a few people in the company it's not a large enough sample. And another problem with doing it with your colleagues is that they are not average users. Most of them are either golden eyes or after working for a few years in a video company, they become golden eyes. And you want people that don't just compare videos all day. People who are really average users.Tamar Shoham: 20:30 Exactly. So, so you highlighted on, on the exact three problems that we set out to then solve. So, so we have this tool that you know, allowed for very easy comparison of video, but how are we going to scale it? How will we be able to do that cheaply and how would we get an average user because average users are very slippery beings. Even someone that is an average user today, after they've watched hours and hours and days of video and comparisons, they start to pick up on the nuances between the compressed or the processed video and the input. And, and then they're broken. They're not an average user anymore. But at some point you just want to know, okay, what will the average user think, so we took this VISTA you know, how do you solve today a problem of I need lots of average user? Crowdsourcing. And we specifically went with mechanical Turk, which gives you access to practically an endless supply of average users.Dror Gill: 21:35 Amazon mechanical Turk. If somebody doesn't know this, a platform, it's basically in the same way that you can launch up a computer servers on the internet, you can launch actual users, right? You can set up a task and, and people, real people from all over the world can bid on this task and perform it for you.Tamar Shoham: 21:54 Yeah. And it's really amazing to see the feedback we get from some of these users because there are all kinds of tasks from Amazon mechanical Turk and some of them, you know, might not be as entertaining, here, we're just paying people to look at videos and express an opinion. So we also try where possible, where we have control over the content selected to choose videos that you know, are visually pleasing or interesting. And people tend to really enjoy these tasks and give quite good feedback on, you know, cool. And we, we also have a problem with repeat users or workers that want to do our tasks again and again. And it's actually interesting to watch the curve of how they become more and more professional and can detect more and more mild artifacts as they repeat the process. So we're actually adding now some screening process that, you know, understand before we're looking at a user who has a very present, they have opinions or if they are still an average user, but we do need to, to verify our users.Tamar Shoham: 23:08 So we, because I mean, this is mechanical Turk, so you know, how do you know if the user is doing the test thoroughly, or if they're just choosing randomly they have, they have to go through the entire process of the test. So they have to play the videos at least once. But you know, what, if they're just choosing randomly or have even managed to configure some bumps to take this test for them? Yeah. So we prevent that by interspersing user validation tests where we, these are sort of like control questions where we know there is a visible degradation on one of the sides but it's not obvious or you know, something that PSNR would pick up on and only users that get those answers right will be included in the pool and only their answers will be incorporated. So, you know, what we do is we launch these hits, which is the name of a task on Amazon mechanical Turk and people register through the hits, complete them, get paid and we start collecting statistics. So first we weed out any sense where either if there were problems with the application and they didn't complete or they just chose not to complete or they didn't answer the user validation questions correctly. And then we have our set for statistical analysis and then we can start looking at and collecting the information and collecting the opinions and very cheaply, very quickly, get a reliable subjective indication of what the average user thought of our pairs of videos.Mark Donnigan: 24:51 This is really interesting Tamar. I'm wondering, do we have some data on how, how this does correlate this average user, these average user test results, do they correlate pretty close to what a "golden eye", you know, would also pick up on, I mean, you did mention that some of these people have become quite proficient, so they're almost becoming trained just through completing these tasks. But you know, I'm curious if someone is listening and maybe they're saying, okay, this sounds really interesting, but my requirements are you know, for someone who maybe fits more of a Goldeneye profile, are we finding that these quote unquote average users are actually the results line up pretty closely to what a golden eye might see?Tamar Shoham: 25:46 So it depends on the question you're posing? So when we start a round like this of testing, the first thing you need to do is pose a question that you want to answer. For example, I have configuration A of the encoder and configuration B. Configuration B is a bit faster. Okay. But I want to make sure it doesn't compromise perceptual quality. Okay. So that's one type of question. And in that type of question, what you're trying to verify is, are the videos in set A perceptually equivalent to the videos in set B? And in that case, okay, you may not want the opinion of a golden eye because even if a golden eye can see the difference, you might be better off as a streaming content provider to go with a faster encode that 95% of your viewers won't be able to distinguish between them.Tamar Shoham: 26:44 So, sometimes you really don't want to know what the golden eye thinks you want to know what the average viewer is going to do. But, we actually can control the level of I guess how professional or how, what shade of gold our users are. And the way we can do that is by changing the level of degradation in the user validation pairs. So if we have a test where we really only want to include results from people who have very high sensitivity to degradation in video, we can use user validation pairs where the degradation is subtle and if they pick up on all of those user validation pairs, then we know that the opinion that they're offering us, you know, is valid. I need to emphasize, maybe I didn't make it clear these user validations are randomly inserted along the test session.Tamar Shoham: 27:38 The user has no idea that there is anything special about these pairs. Do we know of any other solution that works like this? Have you come across anything? So we've come across another similar solution. It's called subjectify.us. It's coming out of MSU, Moscow State University. And I presume everyone in the field, you know, has heard of the MSU video comparison reports that they give out annually to compare different implementations of video codecs. And it seems that they went through the same path that we did saying, okay, you know, we've got metrics but we really need subjective opinions. And they have a solution that is a actually a service that you can apply to and pay for where you give them your images or a video that you want to compare and they perform similar kinds of tests for you. In our solution.Tamar Shoham: 28:45 We, we have many components that are specific to the kind of questions that we want to answer that might be a bit different, but it's actually very encouraging to see similar solutions coming out of other institutions or companies. Because you know, it means that this understanding is finally dawning on everyone that A), you do not have to do BT.500 compliant testing to get interesting subjective feedback on what you do. B), this should be incorporated as part of codec development codec optimization. And, and you know, we're not at the days where you can publish a paper and say I brought the PSNR down and therefore it is by definition good. No, it has to look good as well.Dror Gill: 29:42 And I think, the MSU, the latest report, I'm not sure about the previous ones. They have two reports comparing codecs. One of them is with objective metrics and one of them with subjective. So I guess they developed this external tool subjectify.us first to internally so they can use it when comparing the codecs in the test they do. And then they decided to to make it available to the industry as well.Tamar Shoham: 30:05 Yeah. And, you know, I, don't see it as competition at all. You know, I see it as synergy of all of us figuring out how to work correctly. You know, in this field of video compression, video streaming and a recognition that sure, we want to make better objective metrics or numerical metrics because that's an indispensable tool. But it can never be the only ruler that we measure by. It's just not enough. You need the subjective element and the more, you know, solutions out there to do this. I think it's great for the video streaming community.Mark Donnigan: 30:47 What if somebody wanted to build their own system because this isn't a commercial offer. Although we've had many, many of our customers suggest that we should offer it that way, but it, at this time, you know, we're, we're not planning to do that. So how, how would someone get started, you know, if they listen and say, wow, that's a brilliant idea, you know, mechanical Turk and, but how would I build this?Tamar Shoham: 31:13 Okay. So, so I think the answer is in two parts. The, the important part if you're going to do video comparison is the player and the client. And that's something that if you're starting from scratch is going to be a bit challenging because over the years we've invested a lot of effort in our Beamr View player. And you know, there aren't a lot of equivalent tools like that. You need a very reliable client that can accurately show the video frames side by side in motion, you know, if that's what you're testing, frame synchronized and aligned. And I mean we, we originally did our first round of subjective testing, which actually was BT.500 compliant as we did it in a facility that had all the required gray paints with wall and calibrated monitors. Yeah. And, and we did that for images and building a client for that.Tamar Shoham: 32:13 That was for Beamr's JPEGmini product, building a client that compares two images is quite easy and straightforward. Okay. But building a client that reliably displays side by side video and synchronized playback is, is might be the biggest obstacle to, you know, some companies saying, that's cool, I want to do this. Then you have the second part, which is, you know, the backend creating the test sets. We put a fair bit of thought into how to create tests sets that, you know, we can really rule out unreliable users easily and get good coverage over the video comparisons that we want to make to be able to collect reliable statistics. So, that's like a coding task.Mark Donnigan: 33:03 But the point is there's logic that has to be built around that. And you have to really put thought into, you know, how you are going to either weight or screen out someone's result.Tamar Shoham: 33:15 Definitely, definitely. And, and then you get, I mean, so you have the part of, you know, having a client, you have the design of the test sets on the back end and the part that's you know, building these test sets so that you get a good result. And then you have the third part, which is collecting all the data and doing a, you know, making sense of it, doing a statistical analysis, understanding you know, what the confidence intervals are for the results that you've collected. If maybe you need to collect more results in order to be happy with with your confidence level. And so there are, you know, elements here, some of them are design and understanding how to build it and some of them are coding challenges. And then you have the client, which you know, you need to create. So it's, it's not a trivial thing to build from scratch given the components that we had in the understanding that we had. It, it was quite, quite doable with reasonable investment. And you know, now we're reaping the benefitsDror Gill: 34:17 And, and it's really amazing. You know, for me, for each time we, we want to test something or to to check some of our codec parameters, which one is better or compare two versions of an encoder or etc. You know, you can launch this test and basically overnight, I mean, the next morning you can come in and you'll have 100, 200 user opinions, whatever your budget is averaged that, give you a, an answer, give you a real answer based on user opinions, which one is better...Tamar Shoham: 34:53 It's an invaluable tool. So literally, if before, you know, we would be able to look at two or three clips and saying, yeah, I think this is a good algorithm and you know, this makes it look better. Now, as you said, overnight, you can collect data on dozens of clips over dozens of users and get an opinion and really integrate it into your development cycles. So it really is very, very useful.Mark Donnigan: 35:21 And you know, there's an application that comes to mind. I'm curious if, if we have used the tool for this or we know someone who has and that is for determining optimal ABR ladders. And I'm just curious, is that an application for VISTA?Tamar Shoham: 35:38 So, I mean, as I said before, basically it's a matter of selecting your question before you start designing your test. And what we have built over a brief time and maybe haven't mentioned yet, we call it auto VISTA says that, okay, if I have a question. Okay. I can go from question to answer. Basically by, you know, pulling the big lever on the machine because we have a fully automated system that says this is the encoder I wanted to test with this configuration. Okay. That's A the second encoder or configuration or something I wanted to test is B, you know, take these configuration files, take, these are the inputs I want to work on and do the rest. Okay. And it will set up EC2 instances on Amazon AWS and perform the encodes and create the pairs and send that to the backend and create the test sessions and start a launch around the testing and enable, you know, access to the database to collect the results.Tamar Shoham: 36:51 So with that, you can basically, it's just about posing the question. So if the question you want to answer is, I have, you know, I can either get this layer or I can get that layer, you know, which of them looks better, then yes, you can use VISTA to, you know, create a set that corresponds to one ABR ladder, create a set that corresponds to another and you would need to build the pairs correctly. Okay. For this comparison, what you consider a pair, but that that's again, just in technicalities. Basically for any task that says, I want to compare that pair, does set A look like set B or does set A look better than set B. Okay. Those are the two kind of questions that we can answer. And you know, we've, we've invested a fair bit of effort in making it as easy to use as possible so that it's practical to use it really in answering our development question.Mark Donnigan: 37:52 Well, I think we just exposed to the entire industry what our secret weapon is!Tamar Shoham: 37:59 You know, better than that Mark. It's just one of our secret weapons!Mark Donnigan: 38:03 And you know, I think we should give you an opportunity to give an invitation because I think you are wanting to pull together your own episode and interview?Tamar Shoham: 38:15 This is a shout out to all you women video insiders and we know you're out there. So if you'd like to come on for either a regular podcast interview on the amazing things you are doing in streaming media, then we're very, very happy for you to reach out to either Dror, Mark, or myself, so we can arrange interview. And if some of you don't feel comfortable or are not allowed to expose your trade secrets on the air, then we're thinking of also looking in to do a specific special episode on what it means to be a woman in the video insiders world. Thank you, Tamara, for joining us on this really engaging episode. Thanks so much for having me again.Narrator: 39:01 Thank you for listening to The Video Insiders podcast, a production of Beamr Imaging Ltd. To begin using Beamr's codecs today. Go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Join the conversation by jumping into The Video Insiders LinkedIn Group. If you would like to be a guest on the show, send an email to thevideoinsiders@beamr.com For more podcast episodes, click here. Learn more about Beamr's technology. Learn more about RealNetworks. Today's guest: Reza Rassool.

Join the conversation by jumping into The Video Insiders LinkedIn Group. If you would like to be a guest on the show, send an email to thevideoinsiders@beamr.com For more podcast episodes, click here. Learn more about Beamr's technology. Learn more about Source Digital. Today's guest: Hank Frecon.

Join the conversation by jumping into The Video Insiders LinkedIn Group If you would like to be a guest on the show, send an email to thevideoinsiders@beamr.com For more podcast episodes, click here Learn more about Beamr's technology Today's guest: Mark Kogan

E07: The Video Insiders talk with a pioneering software development company who is at the center of the microservices trend in modern video workflows. Featuring Dom Robinson & Adrian Roe from id3as. Beamr blog: https://blog.beamr.com/2019/02/04/microservices-good-on-a-bad-day-podcast/ Following is an undedited transcript of the episode. Enjoy, but please look past mistakes. Mark & Dror Intro: 00:00 The Video Insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation Kodak nerd and a marketing guy who knows what I-frames and Macroblocks are. And, here are your hosts, Mark Donnigan and Dror Gill. Mark Donnigan: 00:22 Well, welcome back to the Video Insiders. It's so great to be here. Dror, how are you doing? Dror Gill: 00:29 I'm doing great and I'm really excited to do another episode of the Video Insiders. I would say this is probably the best part of my day now doing the Podcast. Although, watching video all day isn't bad at all. Mark Donnigan: 00:45 That's not a bad job. I mean, hey, what do you tell your kids? Dror Gill: 00:49 So, exactly, this is [crosstalk 00:00:52]. I work part-time out of my home office and my daughter comes in after school and she sees me watching those videos and she says, "Dad, what are you doing?" So, I said, I'm watching videos, it's part of my work. I'm checking quality, stuff like that. Then she says, "What? That's your work? You mean they pay you to do that? Where can I get a job like that? You get paid to watch TV." Dror Gill: 01:18 Now, of course, I'm not laid back on a sofa with some popcorn watching a full length movie, no. I'm watching the same boring video clip again and again, the same 20, 30 seconds segments, and I'm watching it with our player tool, with Beamr view and typically one half is flipped over like butterfly mode. And then, you're pausing on a frame and you're looking for these tiny differences in artifacts. So, it's not exactly like watching TV in the evening, but you get to see stuff, you get to watch content, it's nice but could get tiring after a while. But, I don't think I'll ever get tired of this Podcast Mark. Mark Donnigan: 02:04 No, no. I know I won't. And, I think going back to what you do in your day job watching video, I think our listeners can relate to. It's a little bit of a curse, because here you are on a Friday night, you want to relax, you just want to enjoy the movie, and what do you see? All of the freaking artifacts and all the ... And, you're thinking that ABR recipe sure could have been better because I can see it just switched and it shouldn't have, anyway, I think we can all relate to that. Enough about us, let's launch into this episode, and I know that we're both super excited. I was thinking about the intro here, and one of the challenges is all of our guests are awesome, and yet it feels like each guest is like this is the best yet. Dror Gill: 02:56 Yeah. Really today we have two of really the leading experts on video delivery. I've been running into these guys at various industry events and conferences, they also organize conferences and moderate panels and chair sessions, and really lead the industry over the top delivery and CDNs and all of that. So, it's a real pleasure for me to welcome to today's Podcast Dom and Adrian from id3as, hi there? Adrian Roe: 03:26 Hey, thank you very much. Dom Robinson: 03:27 Hey guys. Adrian Roe: 03:27 It's great to be on. Dom Robinson: 03:28 How are you doing? Dror Gill: 03:29 Okay. So, can you tell us a little bit about id3as and stuff you do there? Adrian Roe: 03:34 Sure. So, id3as is a specialist media workflow creation company. We build large scale media systems almost always dealing with live video, so live events, be that sporting events or financial service type announcements, and we specialize in doing so on a very, very large scale and with extremely high service levels. And, both of those I guess are really crucial in a live arena. You only get one shot at doing a live announcement of any sort, so if you missed the goal because the stream was temporary glitch to that point, that's something that's pretty hard to recover from. Adrian Roe: 04:14 We've passionate about the climate and how that can help you build some interesting workflows and deliver some interesting levels of scale and we're primary constructors. Yeah, we're a software company first and foremost, a couple of the founders have a software background. Dom is one of the original streamers ever, so Dom knows everything there is to know about streaming and the rest of us hang on his coattails, but have some of the skills to turn that into one's a note, so work for our customers. Dror Gill: 04:46 Really Dom, so how far back do you go in your streaming history? Dom Robinson: 04:50 Well, anecdotally I sometimes like to count myself in the second or third webcasters in Europe. And interestingly, actually one of the people who's slightly ahead of me in the queue is Steve Clee who works with you guys. So, did the dance around Steve Clee in the mid '90s. So, yeah, it's a good 20, 23 years now I've been streaming [inaudible 00:05:12]. Dror Gill: 05:11 Actually, I mean, we've come a long way and probably we'll talk a bit about this in today's episode. But first, there's something that really puzzles me is your tagline. The tagline of id3as is, good on a bad day. So, can you tell us a bit more about this? What do you mean by good on a bad day? Adrian Roe: 05:33 We think is probably the most important single facet about how your systems behave, especially again in a live context. There are hundreds or possibly even thousands of companies out there who can do perfectly good A to B video encoding and transcoding and delivery when they're running in the lab. And, there's some great tools, open source tools to enable you to do that, things like FFmpeg and so on. What differentiates a great service from a merely good service though is what happens when things go wrong. And especially when you're working at scale, we think it's really important to embrace the fact that things will go wrong. If you have a thousand servers running in your x hundred events at any one particular time, every now and then, yeah, one of those servers is going to go up in a puff of smoke. Your network's going to fail, or a power supply is going to blow up, or whatever else it may be. Adrian Roe: 06:31 And so, what we think differentiates a great service from a merely good one is how well it behaves when things are going wrong or ranji, and partly because of the technology we use and partly because of the background we come from. Technically, when we entered the media space, so as a company that was about eight years ago, obviously Dom's been in the space forever, but as a company it's been eight years or so, we came to it from exactly that angle of how can we ... So, our first customer was Nasdaq delivering financial announcements on a purely cloud based system, and they needed to be able to deliver SLAS to their customers that were vastly higher than the SLAS you could get for any one particular cloud service or cloud server. And so, how you can deliver a fantastic end to end user experience even when things inside your infrastructure are going wrong, we think is much more important than merely, can you do an A to B media chain? Mark Donnigan: 07:27 That's interesting Adrian. I know you guys are really focused on micro services, and maybe you can comment about what you've built and why you're so vested in data architecture. Adrian Roe: 07:39 With both things, there's nothing new in technology. So, Microservices as a phrase, I guess has been particularly hot the last, I don't know, three, four years. Mark Donnigan: 07:49 Oh, it's the buzzy, it's the buzzy word. Dror loves buzzy words. Dror Gill: 07:54 Micro services, buzz, buzz. Mark Donnigan: 07:54 There we go. I'm afraid you have to hear the rap, you have to hear his rap. I'm telling you it's going to be number one on the radio, number one on the charts. It's going to be a hit, it's going to be viral, it's going to be [inaudible 00:08:08]. Adrian Roe: 08:09 So, our approach to Microservices I'm afraid is grounded in the 1980s, so if we're going to do a rap at that point, I'd need to have a big bouffant hair or something in order to do my Microservices- Mark Donnigan: 08:18 And new eyes. Dom Robinson: 08:21 You left your flares in my house dude. Adrian Roe: 08:23 Oh, no, my spare pairs are on, it's okay. Actually, a lot of that thinking comes from the Telco space where when we were starting to get into ... In a past life I used to build online banks and big scale systems like that, but one of the things that was interesting when we came to media is actually if you've got 500 live events running, that's a big system. The amount of data flowing through that with all the different bit rates and so on and so forth is extremely high. Those 500 events might be running on a thousand servers plus in order to give you a full scale redundancy and so on and so forth, and those servers might well be spread across three, four, five different data centers in three, four, five different continents. Adrian Roe: 09:14 And, there are some properly difficult problems to solve in the wider space rather than specifically in the narrow space of a particular single element to that workflow. And, we did some research a while back, we said actually other people must have faced some of these challenges before. And, in particular the Telco space has faced some of these challenges for a long time, and people get so used to just being able to pick up the phone and have the call go from A to B, and the technology by and large works so well that you don't really notice it's there, which is actually another good strap line I think, technology is so good you ignore it, that's what we aspire to. Adrian Roe: 09:51 So, we came across a technology called Erlang, which takes a whole approach to how you build systems. It's different to the traditional. As I say, in itself is not a new technology and that's one of the things we like about it, but basically it says the problems that Erlang was trying to solve when it was created back in the '80s was specifically for things like mobile phones, which is where you would have a mobile phone switch, would be a whole bunch of proprietary boards, each of which could handle maybe, I don't know, five or 10 calls or something, and they'd be stuck together and a dish great big rack with some kind of backplane joining them altogether. And, the boards themselves were not very reliable, and in order for the mobile or for the Telcos to be able to deliver a reliable service using this kind of infrastructure, if any one particular board blew up, the service itself had to continue and other calls, it was really important to those other calls weren't impacted and so on and so forth. Adrian Roe: 10:48 So, this language Erlang was invented specifically to try and solve that class of problem. Now, what was interesting is if you then wind the clock forward 20, 30 years from that particular point and you consider something like the cloud, the cloud is lots and lots of individual computers that on their own aren't particularly powerful and on their own aren't particularly reliable, but they're probably connected together with some kind of LAN or WAN that actually is in pretty good shape. Adrian Roe: 11:17 And, the challenges that back then were really customed to the mobile and network space suddenly become incredibly good patterns of behavior for how you can build high scale cloud systems and super reliable cloud systems. And so, this as is always the case, these new shiny technologies, Erlang, for example, had its moment in the sun about a year or so back when WhatsApp was bought by Facebook, because when WhatsApp were bought by Facebook for $18,000,000,000 or whatever it was, I believe that WhatsApp had a total of 30 technical staff of which only 10 were developers, and they build all of their systems on top of Erlang and got some major advantage from that. Adrian Roe: 11:57 And so, when we came into the whole media space, we thought that there were some very interesting opportunities that would be presented by adopting those kinds of strategies. And now, what's nice then about what a Microservices come into that, so in Erlang or the way we build systems, you have lots of single responsibility, small bits of function, and you gather those bits of function together to make bigger, more complex bits of function and then you gather those together to make progressively more larger scale and more complex workflows. And, what's really nice about that as a strategy so people are increasingly comfortable with using Microservices where I'll have this to do my packaging and this to do my encoding, and then I'll plug these together and so on and so forth. Adrian Roe: 12:46 But, when your language itself is built in those kinds of terms, it gives you a very consistent way of describing about the user experience all the way through your stack. And, the sorts of strategies you have for dealing with challenges or problems that are very low level are exactly the same as the strategies you have for dealing with server outages, and so on and so forth. So, it gives you a very consistent way that you can think about the kind of problems you're trying to solve and how to go about them. Dror Gill: 13:10 Yeah, that's really fascinating. So basically, we're talking about building a very reliable system out of components where not all of these components are reliable all the time, and inside those components are made out of further sub components, which may fail. Adrian Roe: 13:28 Correct, yeah. Dror Gill: 13:29 And then, when you employ a strategy of handling those failures and failing over to different components, you can apply that strategy at all levels of your system from the very small components to the large servers that do large chunks of work. Adrian Roe: 13:45 I could not have put it better myself, that is exactly right. And, you get some secondary benefits, so one is I am strongly of the opinion that when you have systems as large and as complex as the media workflows that we all deal in, there will be issues. Things will go wrong either because of physical infrastructure role, just because of the straight complexity of the kinds of challenges you're looking to meet. So, Erlang would take an approach that says let's treat errors as a first class citizen, let's not try and pretend they're never going to happen, but let's instead have a very, very clear pattern of behavior about how you go about dealing with them, so you can deal with them in a very systematic way. And, if those errors that are very, very micro level, then the system will probably replace the things that's gone bad, and do so in a few well under a fractions of a millisecond. So, you literally don't notice. Adrian Roe: 14:41 We had one particular customer where they had a component that allowed them to patch audio into a live media workflow, and they upgraded their end of that particular system without telling us or going through a test cycle or something which was kind of disappointing. And, a week or so after their upgrade, we were looking at just some logs from an event somewhere, and they seemed a bit noisier than usual. We couldn't work out why and the event had been perfect, nothing had gone wrong, and we discovered that they started to send us messages, one part of the protocol, so they were just incorrectly sending us messages as part of this audio integration that they'd done and they were just sending us junk. Adrian Roe: 15:24 And, the handler forwarded our end was doing what it ought to do in those particular cases that was crashing and getting itself replaced. But, because we designed the system really well, the handler and the logic for it got replaced. The actual underlying TCP connection, for example, stayed up and there wasn't a problem. And, actually we're having to restart the handler several times a second on a live two way audio connection and you literally couldn't hear that it was happening. Mark Donnigan: 15:49 Wow. Adrian Roe: 15:49 Yeah. So yeah, you can get ... But, what's nice is exactly the same strategy in the way of thinking about things and works. Yeah, right at the other level where I've gone seven data centers, and 1000, or 1500 servers running and so on and so forth, and it gives you a camera and a consistent strategy for how you reason about how you're going to behave in order to deliver a service that just keeps on running and running and running even when things go bad. I will give one example, then I'll probably let Dom share some of his views for a second, which was there was a reasonably famous incident a few years back when Amazon in US East just disappeared off the map for about four days and a number of very large companies had some really big challenges with that, and frankly we were just offline for four days. Adrian Roe: 16:36 We had 168 servers running in US East at the time for Nasdaq, one of our customers, we did not get a support call. And so, all of the events that were running on there failed over to other servers that we're running in US West typically. About five minutes later we were back in a fully resilient setup, because we'd created infrastructure in Tokyo and Dublin and various other data center, so that had US West disappeared off the face of the earth as well. Again, we might've got a support call the second time around, but we literally read about it in the papers the next day. Mark Donnigan: 17:06 That's pretty incredible. Are there any other video systems platforms that are architected on Erlang, or are you guys the only ones? Adrian Roe: 17:15 The only other one I am aware of out of the box is a company that specializes more in the CDN and final content delivery side of things, so we're not quite unique, but we are certainly highly, highly unusual. Mark Donnigan: 17:28 Yeah. Yeah. I didn't want to go to Dom, and Dom with your experience in the industry, I'm curious what you're seeing in terms of how companies are architecting their workflows. Are you getting involved in, I guess evolutionary projects, that is you're extending existing platforms and you're in some cases probably shoe honing, legacy approaches, solutions, technologies, et cetera, to try and maybe bring them to the cloud or provide some sort of scale or redundancy that they need? Or, are people just re architecting and building from the ground up? What are people doing out there and what are specifically your clients doing in terms of- Dom Robinson: 18:20 So, it's interesting, I was talking, I did a big review of the Microservices space for Streaming Media Magazine, which came out I think in the October edition this year, and that generated quite a lot of conversations and panel sessions and so on. When we've been approached by broadcasters who have established working workflows, and they're sometimes quite testy because they've spent a lot of time and then they're emotionally quite invested in what they might have spent a decade building and so on. So, they often come with quite testy challenges, what advantages would this bring me? And quite often, there's very little advantage in just making the change for the sake of making the change. The value really comes when you're trying to scale up or take benefit from scaling down. So, with a lot of our financial needs clients the cycle of webcasts, if you'd like a strongly quarterly though, it's all about financial reporting at the end of financial quarters. So, they often want to scale down their infrastructure while during the quiet weeks or quiet months because it saves them costs. Dom Robinson: 19:25 Now, if you're doing 24/7 linear broadcasting, the opportunity to scale down may simply never present itself, you just don't have the opportunity to scale down. Scaling up is a different question, but scaling down, if it's 24/7, there's no real advantage to scaling down, and this is true of cloud as much as it is of Microservices specifically. But, when people come to us and say, right, we've really want to make that migration, they sometimes start with the premise that they'd like to take tiny little pieces of the workflow, and just migrate those little tiny incremental steps. In some cases we may do that, but we tend to try to convince them to actually build a Microservice architecture or virtualized architecture to run in parallel. So, quite often we might start with the client by proposing that they look at their virtualized strategies disaster recovery strategy in the first instance. And then, what happens is after the first disaster, they never go back to their old infrastructure. Mark Donnigan: 20:21 I'm sure, yeah. Dom Robinson: 20:22 And after that, they suddenly see they have all the benefits and it is reliable and despite the fact that they have no idea where on earth this physically is happening, it's working and it works really reliably. And, when it goes wrong, they can conjure up another one in a matter of seconds or minutes. These are not apparent until the broadcaster actually puts them into use. I spent 20 years trying to convince the broadcast industry that IP was going to be a thing, and then overnight they suddenly embraced it fully, and these things people do have epiphany's and they suddenly understand the value. Dom Robinson: 20:56 Disaster recovery has been a nice way to make people feel comfortable because it's not a suggestion of one day we're going to turn off your trusted familiar, nailed down tin and move it all into something you have no idea where it is, what it's running on, how it's running and so on. People are risk averse naturally in taking that type of leap of faith, but once they've done it, they almost invariably see the benefits and so on. So, it's about waiting for the culture in the larger broadcasters to actually place that confidence in the, if you like, the internet era, which generally means as people who are being cynical. I used to make testy comments on panel sessions about the over '50s, '60s, I don't know where you want to put your peg in there. Once those guys finally let internet natives take control, that's when the migration happens. Mark Donnigan: 21:48 Yeah, that's interesting. I can remember going back, oh, 10 years or more and sitting in the cable show which no longer exists, but certain sessions there and Cisco was presenting virtualized network function. And, when the room would always be packed and you'd have a sense if you're sitting in these sessions like this is really happening. This is, wow, this is really happening in all the biggest MSLs were there, all the people were there, right? And then, you'd come back the next year, it'd be the same talk the same people in the room, then come back the next year after that and nobody was [crosstalk 00:22:25], because it's the future. Dom Robinson: 22:23 Yeah, absolutely. Dror Gill: 22:28 It was always the future I was making fun of. Mark Donnigan: 22:30 Now, the switch has absolutely flipped and we're seeing that even on the codecs side, because there was a time where unless you were internet native as you said, you needed a full solution, a black box. It had to go on a rack, it had to ... That's what they bought. And so, selling a codec alone was a little bit of a challenge, but now they can't use black boxes, and they're ... So. Dom Robinson: 22:58 Sometimes I liken it to the era of HI-FI as digital audio and MP3 started to arrive, I was quite involved in MP3 as it emerged in the mid '90s. And, I have over the last two decades flip flop from being the musicians, worst enemy to best friend to worst enemy to best friend, and everybody just depends on the mood of the day. I was reflecting, and this is a bit of a tangent, but I was reflecting when you guys were talking about watching for artifacts in videos. I've spent so long watching 56K blocky video that Adrian, Nick and Steven, the rest of the team never ever let me give any opinion on the quality of video, because I'm quite happy watching a 56K video projected on my wall three meters wide and it doesn't bother me, but I'm sure Dror would be banging his head against the wall if he [inaudible 00:23:47] videos. Dror Gill: 23:49 No, I also started with 56K video and real video, and all of those the players and still in the '90s, but I managed to upgrade myself to SD and then to HD, and now if it's not HDR, it's difficult to view. But in any case, if we look at this transition that is happening, there are several levels to this transition. I mean, first of all, you make the transition from hardware to software then from the software to the cloud, and then from regular software running in the cloud and VMs to this kind of Microservices architecture with Dockers. And, when I talk to customers they say, yeah, we need it as a Docker, we're going to do everything as a Docker. But then, as Mark said, you're not always sure if they're talking about the far future, the new future, the present, and of course it changes if you're talking to the R&D department or you're talking with the people who are actually doing the day to day production. Adrian Roe: 24:51 There were some interesting ... And, I think Docker, this maybe a slightly unpopular thing to say, but yeah, so I think Docker is fantastic and yeah, we use it on a daily basis and development and it's a great on my laptop, I can simulate a cluster of eight servers or doing stuff and failing over between them and so on and so forth and it's fantastic. And, and we've had Docker based solutions in production for four years, five years, certainly a long time, and actually we were starting to move away from Docker as a delivery platform. Dror Gill: 25:22 Really? That's interesting. So, you were in the post Docker era? Adrian Roe: 25:26 Yes, I think just as other people are getting very excited that their software can run on Docker, which I always get confused with announcements like that, because Docker is essentially another layer of virtualization, and strangely enough people first all got excited because their software would run not on a machine but on a virtual machine and it takes quite a strange software requirement before the software can really even tell the difference between those. And then, you move from a virtual machine to a Docker type environment. Adrian Roe: 25:52 Yeah. Docker of course being conceptually nothing new and yeah, it's a wrapper around something the Linux kernel has been able to do for 10 years or so. Yeah. And, it gives you certain guarantees about kerniless and that the sandbox isn't going to interfere with the sandbox and so on and so forth. And, if those things are useful to you, then absolutely use Docker to solve those business problems. Adrian Roe: 26:13 And another thing that Docker can do that again solves a business problem for me when I'm developing is I can spin up a machine, I can instantiate a whole bunch of stuff, I can create virtual networks between them, and then when I rip it all down my laptop's back in pretty much the same state as it was before I started, and I have some guarantees around that. But especially in a cloud environment where I've got a premium job coming in of some sort, I'll spin up a server to do that and probably two servers in different locations to be able to do that. And, they'll do whatever they need to do and yeah, there'll be some complex network flows and so on and so on and so forth to deliver that. Adrian Roe: 26:48 And then, when that event's finished, what I do is I throw that server in the bin. And so, actually Docker there typically is just adding an extra abstraction layer, and that abstraction layer comes at a cost in particular incidence of disk I/O and network I/O that for high quality video workflows you want to go into with your eyes open. And so, when it's solving a business problem for you, I think Docker is a fantastic technology, and some very clever people are involved and so on and so forth. I think there's a massive amount of koolaid been drunk just to see if Docker where it's actually adding complexity and essentially no value. Dror Gill: 27:25 So, I would say that if you have, as I said, if you have a business problem, for example, you have Linux and Windows servers, it's a given you can't change that infrastructure and then you want to deploy a certain task with certain servers, and you wanted to work across them seamlessly with those standard interfaces that you mentioned, then Docker could be a good solution. On the other hand, what you're saying is that if I know that my cluster is fully Linux, certain version of Ubuntu, whatever, and because that's how I set it up, there's no advantage in using the Dockers because I can plan the workflow or the workload on each one of those servers, and at the level of cloud instances launch and terminate them, and then I don't need the Docker. And the issue of overhead, we haven't seen a very large overhead for Docker, we always compare it to running natively. However, we did find that if your software is structured in a certain way, it can increase the overhead of Docker beyond the average. Dom Robinson: 28:31 Something important that came up in some of the panels, Streaming Media West and content delivery world recently on this topic, at the moment people talk synonymously about Microservices and Docker, and that's not true. Just because something's running in Docker does not mean you're running a Microservices architecture. In fact if you dig under the ... All too often- Dror Gill: 28:50 Right, it could be a huge one of the thick servers. Servers that are just running on Docker. Dom Robinson: 28:54 Exactly. All too often people have just simply drop their monolith into a Docker container and called it a Microservice, and that's a ... Well, I won't say it on your Podcast, but that's not true. And, I think that's very important, hence we very much describe our own Erlang based architecture as a Microservices architecture. Docker is as Adrian was explaining, it's nice to have in certain circumstances, it's an essential, but in other circumstances it's just not relevant to us. So, it is important that Docker is a type of virtualization and is nothing to do with Microservices architecture, and it's a very different thing. So, well Adrian might kick me under the virtual table. Adrian Roe: 29:27 No, no, that's all ... Yeah, there's a lot of people who will say if you take an application and you turn it into ... You take a monolithic application and Microservicize it what you have is a monolithic application that's now distributed. So, you've taken a hard problem and made it slightly harder. Dom Robinson: 29:44 Exactly. Adrian Roe: 29:45 So, what's probably more important is that you have good tools and skills and understanding to deal with the kinds of challenges you get in distributed environments. And, actually understanding your own limitations is interesting there. I think if you look at how one coordinate stuff within a particular OS application, then Microservices are a great way of structuring individual applications, and they can cooperate, and they're all in the same space, and you can replace bits of them and that's cool. And then, if you look at one particular server, again, you're Microservices architecture there might go, okay, this component is an an unhealthy state, I'm going to replace it with a clean version and yeah, you can do that in very, very quick time and that's all fantastic. Adrian Roe: 30:33 And then, maybe even if I'm running in some kind of local cluster, I can make similar decisions, but as soon as I'm running in some kind of local cluster, you have to ask the question, what happens if the network fails? What's the probability of the network failing? And if it does, what impact is that going to have on my service? Because yeah, it's just as bad typically to have two servers trying to deliver the same instance of the same live services as it is to have none, because there'll probably be a closed network floods and all sorts of bad things can happen as a result, so. Adrian Roe: 31:08 And then, if you look at a system that's distributed over more than one day center that absolutely just going, oh, I can't see that other service. Yeah, so Microservice is part of my overall delivery. Making decisions based on that is is something you need to do extremely carefully and there's an awful lot of academic work done around consensus algorithms in the presence of network splits and so on and so forth, and it's not until you understand the problem quite well that you actually understand how damned hard the problem is. You're just naive understanding of it is, oh, how hard can it be just to have three servers agree on which of them should currently be doing x, y, z job? Turns out it's really, really, really hard, and that you stand on the shoulders of giants because there's some amazing work done by the academic community over the last few decades, go and leverage the kind of solutions that they've put together to help facilitate that. Dom Robinson: 31:59 I think one of the upsides of Docker though is it has subtly changed how dev teams are thinking, and I think it's because it represents the ability to build these isolated processes and think about passing data between processes rather than just sharing data in a way a monolith might have done. I think that started people to architect in a Microservices architecture. I think people think that that's a Docker thing, but it's not. Docker is more of a catalyst to it than actually bringing about the Microservices architecture. Mark Donnigan: 32:33 That's interesting Dom. I was literally just about to make the point or ask the question even. I wonder if Docker is the first step towards truly Microservices architecture for a lot of these organizations, and I think Adrian did a great job of breaking down the fact that a lot of maybe what is getting sold or assumed to be Microservices really isn't, but in reality it's kind of that next step towards a Microservices architecture. And, it sounds like you agree with that. Dom Robinson: 33:09 Yeah, yeah, yeah. I think it's part of the path, but it's a- Mark Donnigan: 33:12 That's right. Dom Robinson: 33:13 Going back to my original statement Doc- Adrian Roe: 33:13 I am not even sure that strongly it's an available tool in this space. Mark Donnigan: 33:18 It's an available tool, yeah. Adrian Roe: 33:18 You can absolutely build Microservices at dentonville Docker anywhere. Yeah. Mark Donnigan: 33:24 Sure. Absolutely. Yeah. I wasn't saying that Docker's a part of that, but I'm saying if you come from this completely black box environment where everything's in a rack, it's in a physical location, the leap to a truly Microservices architecture is massive. I mean, it's disruptive on every level. Adrian Roe: 33:46 And, it's a great tool, it's part of that journey. I completely do agree with that. Mark Donnigan: 33:48 Yeah, exactly. Exactly. Well, this leads into a conversation or a topic that's really hot in the industry right now, and that's a low latency. I was chuckling, I was walking around Streaming Media West just couple of weeks ago, and I don't think there was one booth, maybe there was one, I just didn't see it. Maybe the Panasonic camera booth, they didn't have low latency plastered all over it, but every booth, low latency, low latency, Adrian Roe: 34:16 There's some interesting stuff around low latency because there's a beautiful reinvention of the wheel happening because, [crosstalk 00:34:28]. Mark Donnigan: 34:29 Well, let's talk about this because maybe we can pull back a little bit of the, I don't know the myths that are out there right now. And also, I'd like to have a brief real honest conversation about what low latency actually means. I think that's one of the things that, again, everybody's head nods, low latency. Oh yeah, yeah, yeah, yeah. We want that too, but then you're like what does it mean? Dror Gill: 34:57 Yeah, everybody wants it. Why do they want it, is an interesting question. And, I heard a very interesting theory today because all the time you hear about this effect of if you're watching a soccer game and you have a lot of latency because you're viewing it over the internet and somebody else has cable or satellite and they view it before you, then you hear all those roars of the goal from around the neighborhood and this annoys the viewer. Dror Gill: 35:25 So, today I heard another theory that that's not the problem of low latency because to block those roars you can just isolate your house and put on headphones or whatever. The real problem that I heard today is that, if there's a large latency between when the game actually happens and when you see it, then you cannot affect the result of the game. Okay? So, the theory goes like this, you're sitting at home, you're wearing your shirt and your fan, and you're sitting in a position that is a lucky position that will help your team. So, if the latency is high then anything you do cannot affect the game because it's too late, because the latency is low you'll have some effect over the result of the game. Adrian Roe: 36:13 When TiVo was brand new and there was the first personal video digital video recorders were a thing. They had this fantastic advert where somebody was watching an american football game, and as they're in sudden death overtime and the kick is just about to do a 45 yard kick. Yeah, and if it goes over, they win the game and if it doesn't, they lose the game. Kickers just running up towards it and he hits pools on the live stream, runs off to the church, prays for half an hour, comes back, and it's really good. Dror Gill: 36:47 Oh, so that's the reason for having a high latency. Adrian Roe: 36:55 It's interesting, the primary businesses in broadcast distribution as In over the air type distribution, but we do a bunch of the hybrid TV services, and as part of that we actually have to do the direct hand off to a bunch of the TVs and set top boxes and so on and so forth. Principally because, the TVs and sets of boxes are so appallingly behaved in terms of the extent to which they deal with then follow standards and so on. So, in order to deliver the streams to a free view plus HDTV in the UK, we just deliver them a broadcast quality transport stream as a progressive download, and entirely so this has been live in the field for, I don't, seven years or something. And entirely without trying to, we have an end to end latency of around two seconds from when the viewer in the home sees it on the TV, as opposed to the original signal coming off the satellite. And nowadays, that would be called super low latency and actually clever and remarkable and so on and so forth. And actually, it's primarily created by the lack of segmentation. Mark Donnigan: 38:01 That's right. Adrian Roe: 38:03 What's happened that suddenly made you have an RTMP streams. It's depended a little bit on how much buffering you had in the player and so on, but they typically have an end to end latency in a video workflow based around RTMP, five, six seconds, that was normal and they would really comment on it. And now, suddenly that you have segment oriented distribution mechanisms like HLS and Dash and all these kinds of things, people talk about low latency and suddenly they mean five to 10 seconds and so on and so forth. And, that's actually all been driven by the fact that I think by and large CDNS hate media, and they want to pretend that all media or assets are in fact JPEGS or JavaScript files and so and so forth. Dror Gill: 38:48 Or webpages. Adrian Roe: 38:49 Exactly. Dror Gill: 38:50 Yeah, like small chunks of data that's what they know how to handle best. Adrian Roe: 38:52 Exactly. And so, the people distributing the content like to treat them as static assets, and they all have their infrastructures built around the very, very efficient delivery of static assets, and that creates high high latency. So, you then get technologies like WebRTC which is emerging, which we use heavily in production for ... So, one of our customers is a sports broadcaster, their customers can deliver their own live commentary on a system over WebRTC, and it basically doesn't add any latency to the process because while we'll hand off a low latency encoder of the feed over WebRTC to wherever the commentator is, the commentator will view the stream and commentate. Adrian Roe: 39:34 In the meantime, we're going to a really high quality encode. In fact, this might be a mutual customer, but I probably won't say their name on air. We're going to do a really high quality encoder that same content in the meantime, and by the time we get the audio back from the commentator, we just mix that in with the crowd noise, add it to the video that we've already encoded at that point and away you go. And, you're pretty much getting live commentary on a system for free in terms of end to end latency. Yeah, and then sports, so we should be using WebRTC, we should be in this ... Adrian Roe: 40:05 The problem, CDNS don't like WebRTC not at least because it's a connection oriented protocol. You can't just do the same for everybody. You've got to have separate encryption keys and it's all peer to peer and so on and so forth. And so, it doesn't scale using their standard models. And so, most of the discussion around low latency as far as I can tell is the extent to which you can pretend that your segmented assets are in fact live streams, and so Akamai has this thing where they'll start playing a segment before it's finished and so on and so forth. Well actually, it starts to look an awful lot like a progressive download at that point. Mark Donnigan: 40:41 That's a great point. That's absolutely. Absolutely. And, what I find as I've walked around, like I said, walking around Streaming Media West, and looking at websites, reading marketing material, of everybody who has a low latency solution with a few exceptions, nobody's addressing the end to end factor of it. So, it cracks me up when I see an encoding vendor really touting low latency, low latency and I'm sitting here thinking, I mean Dror, what are we like 20 milliseconds? How much more low latency can you get than that? Dror Gill: 41:19 Yeah, at the Kodak level it is very low. Mark Donnigan: 41:21 Yeah, at the Kodak level. And then, when you begin to abstract out and of course the process adds time, right? But still, I mean the point is, is like it's ... I don't know, I guess part of what am reacting to and what I'm looking for, even in your response is that end to end, yes, but addressing latency end to end is really complicated because now just as you said, Adrian, now you have to look at the CDN, and you have to look at what you're doing on packaging, and you have to look at even your player architecture like progressive download. Some players can deal with that, great, other players can't. So, what do you do? Dom Robinson: 42:04 So, one of the things that I think just stepping back and having a reasonably long game view of the evolution of the industry over here in, in the UK, particularly in Europe general, low latency has been a thing for 15, 20 years. And, the big thing that's changed and why low latencies all over the global US driven press is the deregulation of the gambling market, and that's why everyone's interested in low latency. Over here in the UK, we've had gambling online for live sports for 15, 20 years. And, for everyone ... I used to run a CDN from 2001 to end of the 2000s, and all the clients were interested in was fast start for advertising for VOD assets and low latency for betting delivery. And obviously, low latency is important because the lower the latency, the later you can shut your betting gates. And, if you've got a ten second segment or 30 seconds to an hour, three segments to wait, you've got to shut your betting maybe a minute, half a minute before the race finishes or before the race starts, whichever way you're doing the betting. Dom Robinson: 43:14 And, that was very important over here. You didn't have a gambling market in the states online until last year I believe. And so, low latency just really wasn't very interesting. People were really only interested in can actually deliver reliably a big audience rather than can I deliver this to even small audiences, but with low latency, because I've got a betting market going on. And, as that betting deregulations come in, suddenly all the US centric companies have become really fascinated in whether they can shorten that low latency and so on and so forth. And, that's why companies 15, 20 years ago, over here, some of the big sports broadcast and so on, they were using RTMP extensively so that they could run their betting gates until the last second, and it really ramps up the amount of betting in those few seconds before the race starts. Dom Robinson: 44:03 So, that's why it's important. It's not for any other reason. In fact, I sometimes rather sourly ask audiences if they really ever heard their neighbors cheering to a football game before they've seen it because being caught on a sweeney of socially gathering around the TV, and it's an important game like that where your neighbors might have have their TV on loud enough, you frankly got a TV and it's on as well. Dom Robinson: 44:28 The real benchmark of the whole thing is can you beat the tweet, that's the measurable thing, and there's absurd little data in a tweet and a lot of tweets are machine generated, a goal is scored and it doesn't even take a fan in the stadium to type it, and send it to his friends, it's just instantly updated trying to beat a few packets of data across the world compared to trying to compress video, get it buffered, get it distributed across probably two or three stages of workflow decoded in the player and rendered. You're never going to be to tweet at that level. So, really the excitement is about betting, the deregulation of the betting market and gambling market. Dror Gill: 45:06 So, that's interesting. Today you don't measure the latency between and over the air broadcast and the top over the internet broadcasts, but you want to beat another over the internet broadcast, which is a very small packets of the tweet. So. Adrian Roe: 45:22 Exactly right. Dror Gill: 45:23 Actually, competing with the social networks and other broadcast networks. Dom Robinson: 45:26 Exactly. Adrian Roe: 45:28 I can remember, there were tongue in cheek when WhatsApp were bought, they were boasting about the number of messages that they dealt with a day, and yeah it was very large number, billions of messages a day. And, I remembered a little back of an envelope calculation that if you ... Based on the adage that a picture was worth a thousand words, and across all the various different events and channels and live sports and stuff like that we cover, if you counted a thousand words for every frame of video that we delivered, we were two orders of magnitude higher than WhatsApp. Dror Gill: 46:07 So, yeah. So, you had more traffic in your small company, you had more traffic than WhatsApp. Adrian Roe: 46:11 Yeah. Dror Gill: 46:13 A picture is worth a thousand words, and then you have 25 or 50 pictures every second. And, this is across all of your channels. So, yeah [crosstalk 00:46:22]. Mark Donnigan: 46:21 That's a lot of words. It maybe chuckle up. Well, this is- Dror Gill: 46:27 We always say video is complicated and now we know why. Mark Donnigan: 46:32 Exactly. Well, this has been an amazing discussion, and I think we should bring into a close with, I'd really like your perspective, Adrian and Dom, you're working with broadcasters and presumably sitting right in the middle of this OTT transition. Dom, I know you mentioned that for 20 years you'd been evangelizing IP, and now finally it's a thing, everybody gets it. But, just curious, maybe you can share with the listeners some trends that you're seeing, how is a traditional broadcast or someone who's operating a little more of your traditional infrastructure, et cetera, how are they adopting OTT into their workflows? Are they building parallel workflows? Are some fork lifting and making the full IP transition. I think this is a great conversation to end with. Adrian Roe: 47:25 I think we're right at the cusp of exactly that. So, none of our customers are doing it side by side if they are full blown traditional broadcasters. I think increasingly a lot of our customers who may be deliver exclusively over the internet would also consider themselves broadcasters, and so I think the parlance is perhaps slightly out of date, but that's one of the things that I think is really interesting is some of the cultural challenges that come out of this. So, one of our customers who is a full blown traditional broadcaster, when you're dealing with fault tolerant large scale systems of the sort, that idea is built, then one of the things that's a given is that it's going to be a computer that decides which server is going to be responsible for which particular, this is BBC one's encoder, this is ... Yeah, whatever ITVs encoder or whatever. It's going to be a computer that makes those decisions because a computer can react in milliseconds if one of those services is no longer available and reroute it somewhere else. Adrian Roe: 48:28 And, this wasn't a public cloud implementation it was a private cloud implementation that they had couple of racks of servers and data management infrastructure on top that was doing all of the dynamic allocation and tolerance and all this clever stuff. And they said, so when we're showing our customers around, if channel four comes around, how can we tell then which is their encoder? And we said, you count. There isn't a channel four encoder there's an encoder that might be doing the job. Adrian Roe: 48:55 And, one of the features we had to add to the product as just to get over the cultural hurdle with them was the concept of a preferred encoder. So, if everything was in its normal happy state, then yeah, this particular encoder, halfway down on the right hand side of rack three, was going to be the one doing channel four, and just those simple things where they think people do still think in terms of appliances and raw rian and so on and so forth, and some of the challenges to move away from that into cloud thinking bit actually on the cloud or not, cloud thinking still applies it. It's funny where people trip up. Dom Robinson: 49:36 One of my bugbears in the industry, I'm a bit of a pedant with some of the terminology that gets used and so on. One of my bugbears is the term OTT. So, having spent a good long while playing with video and audio distribution over IP networks and so on, I struggle to think of any broadcast technology, which doesn't use IP at some point in this either production or distribution workflow, there just isn't any now. And so, if you're watching live news, the contribution visa coming over cell phones which are contribution is some sort of streaming protocol or a film or TV program production people are emailing files or they're dropboxing files, or they're sending them through digital asset management systems or however it may be. Dom Robinson: 50:20 But, the programs are being created using IP and have been for quite a while and increasingly nobody replaces technology with some sort of proprietary non IP based tool these days at any level in the broadcast industry. I rather store everything I can to try to avoid using the word OTT. And being a pedant about it, OTT simply means the paywall is outside of this last mile access network. That's all it means. It has nothing whatsoever to do with video distribution or streaming or anything like that. It's simply to do with where you take your payment from somebody. Dom Robinson: 50:57 So, Netflix has a hybridized side, but Netflix, you generally access through an ISP and when you make your payment, you pay Netflix directly. You don't pay through your ISP, that is an OTT service. Skype is an OTT service. Again, you connect through your phone service, your cable service, whatever it may be, but you actually subscribe directly with Skype, that is a true OTT service, and that's what OTT means. It's become in the last eight years synonymous with streaming ,and I can't think of a broadcast network which doesn't at some point use IP either streaming or file transfer based technologies to compose the program. Dom Robinson: 51:37 So, broadcast is streaming, streaming is broadcast. They have been synonymous for over a decade. It is how you connect the payment, which defines something as OTT, and it may well be that you can receive a video stream outside of one particular ISPs network, but that doesn't really mean anything. So, this battle between broadcast and OTT, it's a meaningless decision of where you're collecting payments for me. It really doesn't have any bearing on the technologies that we all work with which are video compression and distribution and so on. So. Mark Donnigan: 52:11 That's brilliant. That is really, really a smart observation and analysis there Dom. Well, I think we should wrap it up here. We definitely need to do a part two. I think we will have you guys back, there's so much more we could be talking about, but I want to thank our amazing audience, without you the Video Insiders Podcast would just be Dror and me talking to ourselves. Dror Gill: 52:38 Buzzing to ourselves some buzzy words. Mark Donnigan: 52:40 Buzzy words, buzzing, buzzing, taking up bits on a server somewhere and this has been a production of Beamer Imaging Limited, you can subscribe at thevideoinsiders.com where you can listen to us on Spotify, on iTunes, on Google Play, and more platforms coming soon. And, if you'd like to try out Beamer Codecs in your lab or production environment, we're actually giving away up to 100 hours of HEVC and H.264 encoding every month. Just go to beamer.com/free, that's F-R-E-E to get started. And until next time, thank you and have an awesome day encoding video. Speaker 1: 53:30 Thank you for listening to the Video Insiders Podcast, a production of Beamr Limited. To begin using Beamrs' Codecs today, go to https://beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 trans coding every month.

In this episode, we catch up with the President & CEO of the company who won the 2018 Frost & Sullivan Global Content Protection Entrepreneurial Company of the Year Award, where we talk about DRM and its ever-changing role in the video industry. Mark Donnigan: 00:00 This episode on DRM was so meaty, that we decided to jump right into a nine minute segment of Dror talking with Christopher Levy, who is President and CEO of BuyDRM, about how DRM technology fragmentation came to be, and the strategies behind DRM as a platform lock in. After this extended clip, we resumed the rest of the interview. You will definitely want to keep listening. Here's Dror and Christopher Levy. Dror Gill: 00:35 This is really an interesting trend you're talking about. On one hand, you have these silos, and the silos include the software platforms, the hardware devices, the content, and the DRM mechanism, which is made by a certain, by a specific company. Now, some of these companies have interest only in parts of this type of ecosystem. For example, Samsung have devices, they have a software platform, they don't have their own DRM, and they don't have much content of their own. So, now this collaboration with Apple is bringing more content, a lot more content, to Samsung devices, and bringing a lot more devices to Apple's content. We all know, all of you know, the rumors about Apple expanding their content service to be much wider than it is today, so it really makes sense. Dror Gill: 01:37 The topic you raise of which DRM will be used to enable this collaboration or cross-streaming of content between platform is really a very interesting issue. Another point you mentioned, which, you know, I can really resonate with, is the fact that standardization has happened across the video ecosystem in things, in components such as codex, packaging, controller mechanisms, manifests, things like that. And, DRM, although there have been attempts to standardize DRM, there has always been some internal component of that DRM that remained proprietary. That remained part of a closed, in siloed ecosystem such as PlayReady and Widevine, and this always struck me as kind of odd that everything else is standardized, and even the you know, mechanisms of exchanging keys in DRM's are defining DRM protocols. Dror Gill: 03:08 Everything is standardized but finally, the key. Those very large companies do not want to give up the key. The key is what they control, and it is the key of opening the content, but also the key to the whole ecosystem, and platform which enables their own platforms to grow. Dror Gill: 03:31 My question is, and referring to the fact that you also said that more and more layers or components of DRM are being standardized. Do you see somewhere in our near future that finally this content protection component will also be fully standardized, and in the same way that we're now having the harmonization of HLS and DASH with CMF, have harmonization of different DRM systems, and no single company would control those, this key to the industry? Christopher Levy: 04:10 You make a really good point that, you know, in essence DRM and Codec have had similar kind of evolutions over time. If you look specifically at the DRM industry, and not to make a short story long, but to kind of paint a picture of why we're at, where we're at, you've got an odd mix of singularities that it would seem would leave almost no possibility that there would be a marketplace for DRM where their companies would have to pay for it, or that companies would continue to invest in it. Christopher Levy: 04:46 I mean, if you fall way back to the beginning of the invention of DRM per se, as we know it, you fall way back to a meeting between Intertrust and Microsoft in, I think late 1999, where they agreed they were going to collaborate on some stuff. But then, at some point when Reciprocal launched, and decided that they were gonna partner close with Microsoft, Intertrust made an offer to Microsoft." Hey, give us two hundred and fifty million dollars, and license our technology," and a certain gentleman at Microsoft made the decision with his team to say, no. Only to later than lose a multi-billion dollar lawsuit to Intertrust, and Bill Gates wrote them a check that later allowed them then to go pursue every single company in the world that uses DRM. And so now, you've got Intertrust, who has a DRM, Marlin, that nobody uses in the U.S., only uses it in China, but Intertrust doesn't have a browser or an operating system. But, they own all the intellectual property around DRM, and so Apple, Microsoft, Google, Samsung, Sony, anyone in the world who touches DRM has had to take a license from Intertrust. Christopher Levy: 06:00 But, then Intertrust, wasn't able to be successful with their own DRM technology, because, as I mentioned, they're locked out when it comes to having a browser or an operating system. So, they actually have somewhat abandoned Marlin, and moved to support Google, Apple, and Microsoft's DRM's. But then, you look at them and you say, "Okay, what would drive these companies to integrate such, so they an be interoperable?" Because that's kind of what we're talking about here, is how are Samsung and Apple gonna interop, but how is that gonna help everyone? Including HEVC, and what you find out is, that you know, DRM was clearly created. When I say created, when it was commercialized by Apple, Google, and Microsoft, it was obviously done on two kinda bifurcated paths. Christopher Levy: 06:46 One, to satisfy potential looming lawsuits related to record labels, and studios, and artists, and creators, and content owners, pointing a finger at these large companies, saying your technology platforms are massive piracy platforms. Secondly, it was done as a platform play, to get you to use the platform. I mean if we look back at PlayReady. PlayReady was a technology that was completely driven to lock you in to using Windows based technologies, and Microsoft based technologies. Christopher Levy: 07:16 Now, if you pull that out, if you pull Intertrust, and Microsoft completely out of the DRM discussion, and you just look at Apple and Google, who really are driving the entire industry now. They both have been using DRM to date, and on both those paths. To satisfy the lawyers, and to satisfy the lock in, and that is just where we're at, but, now the market has gotten so saturated. Christopher Levy: 07:42 Google has not been successful selling devices. The Google Chromebook is a disaster. The Google Pixel phones are not selling as well as Google would expect they would sell, as the inventor, and owner of Android. So, now you get down to, okay, DRM previously was a legal thing, it was a lock in thing, but now, what is it? And I think what we're starting to see come to light is, that with the movement of common encryption, by you know, different various parties, the movement towards CMAF, the movement away ASCTR encryption, that was designed in PlayReady, into CBC encryption, we're really close to having a CMF, CMAF file, that using common encryption would have decryptors for Fairplay, PlayReady, and Widevine. Christopher Levy: 08:37 So, we're getting very close to that. A deal like this, that Apple and Amazon have struck. It really could be the gas to the match. I sense that there's gonna be a push through here, the technology, Apple's Fairplay has gotten a lot of deployment experience now, so there's a big community contributing back to Apple. Christopher Levy: 08:57 Apple has a very small team, if you knew the number of people working a DRM in Google and Apple, you would be shocked, and yet, they're converging. And, I think the reason they're converging, is that, you know, the consumer in the end, is dictating what they want, and consumers have made it very clear they want, you know, Samsung smart TV's. They want Apple TV's. They want Android tablets. They want Apple IPhones. Christopher Levy: 09:24 I think both of them now, are gonna take a little play out of Steve Jobs DRM playbook, and probably find a way to cross pollinate their businesses, because Apple's not in the search business, you know. They try and interact in the home marketplace, but Google already owns the home, outside of Alexa. So, it's interesting, you know, to just clearly take one stab at it. I would say that we are headed towards complete inter op ability and that has a lot of benefits. Christopher Levy: 09:57 It benefits operators, in cost reductions. It benefits consumers, in less confusion and playback stops. But mostly, it's gonna give Google a shot at, you know, exposing their offering to Apple's audience and vice versa. Announcer: 10:15 The Video Insiders is the show that makes sense of all that is happening in the world of online video as seen through the eyes of a second generation Codec nerd, and a marketing guy who knows what Iframes and macro blocks are. Here are your hosts, Mark Donnigan, and Dror Gill. Mark Donnigan: 10:36 Let's rejoin the interview with Christopher Levy from BuyDRM. Christopher Levy: 10:41 To kind of just give a quick summary, the company is one of the dark horses of the content protection, and DRM business. We have a pretty well known brand as a company. We have extended our platform out pretty widely in the business. So we have a Multi-DRM platform called KeyOS, and we have a couple of different components of it. Christopher Levy: 11:03 We have the encryption tools, we have the licensing tools, and we have the player tools, and we're integrated with about fifty different encoder server player companies in the marketplace. We service some of the major brands that you might be familiar with, like BBC iplayer BBC sounds, Sony Crackle, Showtime OTT, Blizzard, Warner Brothers, and we do a lot of work that we're not really at the liberty to discuss. Christopher Levy: 11:30 But we do a lot of pre-release work as well. So, a lot of the focus in the business is on consumer media, but we also have a pretty significant business that's, you know, pre-release. So, Digital Daily, Screeners, Academy voters. We are very active in the Academy voter space. We currently host Apple Fairplay certificates for the five largest media companies in the world today. Some of which you're familiar with, I'm sure. Christopher Levy: 11:57 To kind of fast forward, the company is privately owned. We are profitable. We own the company, myself, and the Chairman, Ron Baker, is my partner in the business, and we have different development teams based around the world. We've got our core team in Riga, Latvia. We have a team in Moscow, and a couple of people in St. Petersburg, and then we also have some people in Paris that work on our Android and IOS SDK's and our CTO is in Vancouver, and the company and myself, and the sales marketing management teams are all based in Austin, Texas, and yeah. Christopher Levy: 12:34 Just to fast forward, we, late last year, for the first time ever, in the Frost and Sullivan Global Content Protection Report. This report is, you know, it's kind of a bigger picture report. It's kind of what they call content protection includes casts and DRM, so we are listed in the report with some of the heavy weights like you know NAGRA or Detto. But we were included in that report, and we ultimately were selected as the entrepreneurial company of the year for our variety of different business models. Christopher Levy: 13:07 You know, we pride ourselves on having a very strong core DRM platform. But, we also now license our technology, so we've expanded into India, and all over Europe. We have several large major gaming companies, media companies that now run our software in their own data center, in their own cloud. So, that kind of vision shift in the company, I think is what got us over the goal line with the award. Christopher Levy: 13:30 But, we're just you know, wrapping up one of our best years ever, if not our best year ever, last year. And, we're just kind of waiting to see all the different crazy announcements that come out of CES, you mentioned our team is there on site. But, I'm closely watching the announcements that Apple made about partnering with Samsung in LG, because it creates some very interesting possible synergies that all of us can benefit from. Mark Donnigan: 13:56 Definitely. We're tracking that very closely as well. I mean, let's start there. Christopher Levy: 14:03 Well, you know, the DRM industry at large is very interesting, because it has become a bit of the political third rail of digital media, as I'm sure you all know. At this point, each DRM technology is siloed into a global technology company. So, if you start left to right based on the kind of market, you know, availability of the product, you had Microsoft with PlayReady. PlayReady runs in IE and EDGE, and on Windows natively. You've got Google with Widevine that runs in Chrome, primarily on Windows and Android, but also runs in IOS. It's the one technology that runs on all three platforms, and you've got Apple's Apple Fairplay DRM, which really only works in Safari on MacOS, and Safari on IOS, and it works for tvOS. It will also work possibly on other products, we may find out here soon. Christopher Levy: 15:11 I have to be careful what I say, but to kind of track what's going on, you know this announcement that Apple made about being able to move their business offering over to other platforms, I think, was largely driven by the tipping point of the iPhone sales over the past couple of years. It's no secret that Apple's last couple of iPhone product lines have not sold that well, so that's created kind of a tipping point in the company where now, they're trying to figure out, okay, where do we go next? And clearly Apple has a massive media empire. Christopher Levy: 15:45 They're one of the first companies to ever have a license to just about every song, and movie, and TV show that consumers in America are familiar with. And, they obviously have a globally strong brand. But, because DRM has been a political silo, today, you know, iTunes doesn't appear on Android. It doesn't appear on Tizen. It's not on Ruku. It's not on Smart TV's. But, that is going to change, and the question is, how will it change? Christopher Levy: 16:13 And to kind of give an example, if you take a look at Roku, who has gone through a similar transition where they were a streaming puck company, they were a streaming stick company. Then, Amazon entered the streaming stick company, and entered with Amazon prime, and Roku then suddenly decides, now, it's a content company. But, it also wants to get eyeballs and users onto it's platform, regardless of the direction it's going, and so Roku had to go. To support YouTube, they had to work with Google to implement Widevine DRM on the Roku platform, which previously was a PlayReady, and Verimatrix platform, natively and solely, and so that model where Roku kind of stepped over the fence and implemented Google's DRM to get YouTube is an interesting example of maybe what's going on with Samsung. We don't totally know yet. What Samsung and LG are doing, and we have our feelers out, and of course, we've talked to Apple pretty extensively about it, because we have a very close relationship with Apple as one of their frontline partners in the industry. Christopher Levy: 17:18 But, I think it plays out one of two ways, and it is somewhat DRM dependent, and Codec dependent, because of the fact that Apple is either going to allow Samsung to distribute iTunes on their platform, or really Apple is gonna distribute, I say, because it's an open app marketplace. But, Apple has a decision to make, and it's, do they deploy it using WideVine, and reformat their application platform to use Widevine DRM instead of Fairplay, or does Samsung jump the shark and implement Fairplay? Christopher Levy: 17:54 Because at the core of all these DRM's, the encryption decryption components are almost identical. At this point, all three DRM's us AES one twenty in encryption. There are some various different tweaks there, with regards to, the encryption mode CBC, verses CBR, but we're starting to see some standardization. I'm sure you're familiar with, with around formats. I personally believe it could go either way, or it could go both ways, because if Samsung were to implement Fairplay on their newer platforms, that would create a whole new synergy between Samsung and Apple that, oddly enough, hasn't been destroyed by the multi-billion dollar IP lawsuits that have gone back and forth between the two of them as vendors and competitors. Christopher Levy: 18:40 But on the other hand, I could see, you know, Apple just wanting to push it out through Widevine, because if they got iTunes to work with Widevine, and I mean iTunes video is what I'm focused on, then the majority of the relatively, recently shipped Samsung TV's, more than likely, can all support iTunes. Which would be kind of cosmic shift in these siloed offerings that all fall back to DRM. Am I right? I mean, Apple's got iTunes on Fairplay. Google's got Google Play on Widevine. So, it's an interesting thing that's gonna happen. I am very curious myself. Mark Donnigan: 19:21 It does sound like really good news ultimately. It's interesting your observation about, you know, the platform lock in. I'm thinking back to when I was active in the DECE, which became the ultraviolet, you know, which, was really revolutionary at the time. Because, you know, back then, you consume content from a particular store, if that was Vudu, for example. You were locked into Vudu, right. You know if Vudu wasn't on a particular device, then I was also locked in to the devices I could watch it on. Mark Donnigan: 20:00 So,the consumer now is going to enjoy the benefit of this truly, any content, anywhere, on any device, at any time. You know, so, that's all very good things. You know Christopher, I was reading your blog and by the way, listeners should definitely go to the blog, why don't you tell them again, I just I don't recall the actual url. Tell them the address of your blog. Christopher Levy: 20:29 Yeah, it's really simple to remember it's: thedrmblog.com. Mark Donnigan: 20:35 That's it. Thedrmblog.com that's awesome. Yeah, kinda like thevideoinsiders.com, that's right. No, Christopher, I want to get your comment on, I think it's your latest post, where you're talking about HTML5, kind of the App-less approach, and you know, I appreciated the article. Mark Donnigan: 21:01 It was presenting a little bit of the pro's and the con's of, and I think you were doing it in the context of inflight entertainment. And, I know that people, if you're running a video service, if you're Amazon, if you're Netflix, you know, even if you're Vudu, Hulu, whatever, you know, they have to maintain up, hundred, you know, multiple hundreds of different player SDK's. You know, it's incredibly complex. So, the idea that you could perhaps, just scale that way back, and perhaps just go to an HTML5 app, is interesting. So, maybe you can share with the listeners, both, your thoughts, and the pro's and the con's, and give kind of a recap of that blog post. Christopher Levy: 21:48 You bet. And, I mean clearly, that obviously, is also effected by the evolution of Codec, and HEVC and others, but there's this trend, and the in-flight entertainment space is an interesting creature. I've spent the past two years researching this space because previously BuyDRM had a bunch of clients in the space, but they were through third parties. So, you know, we had a business with Lufthansa, Technology Solutions, where they were deploying our technology in Virgin Airlines, LL Airlines, Lufthansa Airlines. Christopher Levy: 22:24 They put the technology on Greyhound buses. Post Bus, which is the largest bus company in Germany, and we also have a little bit of business with companies like Global Eagle, and some others, and we started to look at, you know what's the opportunity for us to enter the space directly. Christopher Levy: 22:41 So, we started going, attending shows, and doing research, talking to people. So, the way that in-flight entertainment systems make it on airplanes is different than you might expect. The airline industry has about four conglomerates that all, kind of, control what you call, you know, in-flight experience. Now, the in- flight experience, you know, the video piece is what we're focused on, but it includes interiors. It includes catering. It includes environment. It includes wifi. It includes being green. Entertainment's one component of it, but it's locked in with all these other kind of aspects of the business, and so therefore, it's treated in a very, what I would say, in a very institutional manner. Christopher Levy: 23:19 To date, in-flight systems have been wired, and they're in your head rest, or it's a fold up screen if you're a business, and you're first class, it extends out a little booth you're in, and you're limited to watch videos that are in a dedicated platform that's hardwired on to the plane, and that was the experience. Christopher Levy: 23:38 Then, along came satellite. Then, along came in-flight wifi. And, IFEC, you know, in-flight entertainment. The connected version with wires, suddenly pivoted to in- flight entertainment overnight. Which means wireless, and then DRM became a big topic. But, what you started to see DRM really drive, was the issue of, do airlines want to maintain premium content apps for their clients so they can watch content? Or do they just want them to open their browser, and get on the wifi network, sign in, and then have access to all the content through a browser? Christopher Levy: 24:12 There's this trend in the business where a lot of companies have gone the direction of the browser, so like, if you get on a Southwest Airlines flight, you want to watch Dish TV live, you know, the implementation is there, on the plane. There's a dish receiver on the top of the plane that's got multiple different LMB's. Each channel is switchable. They got an encoder on the plane that takes the MPEG transport stream coming down over the dish, converts it, encrypts it, shoots it out of a server, on the plane to your browser. And that's easy, and it's fun. And it works, and it's especially effective for live TV. Christopher Levy: 24:47 Stepping away from that, when you start to talk about doing things that are more efficient, and I think where consumers are headed, which is downloads, offline playback, bring your own device, the browser kind of starts to die because it doesn't work offline well. It doesn't do downloads well, because each browser has a protected limited amount of storage on the device it's running for security reasons. And browsers, the implementation, most players in them are not that efficient, and so what you find is that the browser is quicker, it's faster, it's dirtier, it's cheaper, but it opens up the door for a bunch of fails on the consumer side. Which is, decreased battery life, forced to use streaming, which uses the wifi radio which is decreased battery life, increased overhead on the aircraft. Christopher Levy: 25:43 You don't get offline playback or download, so you can't download a stream and play it in a browser effectively offline. And lastly, consumers are very comfortable with their devices. Like, if I'm given the option of watching my ten point seven iPad pro with my bose qc thirty-five-two headphones, I'm gonna pick that every time over plugging some crappy, hand wiped headphones that hardly fit, that sound terrible, into a jack that's crackling, so I can watch a screen that has a four inch thick screen protector on it. Christopher Levy: 26:20 The airline industries are trying to figure out, okay, well what do we do, because we're not OTT operators, but how do we make clients happy? And so, they're caught in a dilemma right now. Now, you know, I see it going two ways. I frankly think the live TV will continue to be in the browser. Remember, DRM adds some overhead cause you gotta decrypt the content and add some CPU overhead therefore decrease battery life. Christopher Levy: 26:44 When you move to an app, I think apps are gonna be a lot more prevalent for VOD content and shifted viewing, and TV viewing. The last thing that's going on, that the airlines don't totally understand, and I've spent a lot of time trying to educate them about, and this is kind of a tangential issue, but I'm sure you can understand, is that every single passenger that's on an airplane, more than likely, and I said within a ninety percent or higher realm of operation, especially on International and business flights, have a Netflix, Itunes ,Google Play, Hulu account. And now, with DRM they can download all the movies and TV shows they want to their device, and just go on the plane, and have every blockbuster, every TV show, every highlight, every documentary, every podcast, that they want to. Access on their own device, and use it in their own way in their own time, in their own comfort. Christopher Levy: 27:42 So, that's kind of the big divide right now, is companies are trying to figure out, well, we can save time and money on not having to build IFE apps, and just go to the browser, but we lose a bunch of things that consumers might want. There's a couple of other things which are also driving that, and those are accessibility issues which I think, will drive a lot of companies be forced to maintain apps, and those accessibility issues. Christopher Levy: 28:03 Accessibility use on devices, you know, iPhone and Android have different functions for people that have disabilities or motor challenged, and aren't, you know, able to use the device the same way they would use an IFE platform where they gotta touch the screen in front of them, you know, reach up, and et cetera, et cetera. Christopher Levy: 28:24 Secondly is multi-track audio. Thirdly, is multi-language caption support. I think those are the three issues, are more gracefully handled within applications. Christopher Levy: 28:34 Lastly, I think applications are more likely to support advanced codecs, like HEVC, sooner. Because the applications are running on devices that are being modernized, updated, purchased more widely across a wider range of markets. And so, the people that design the player SDK's and apps, and the operating systems in the devices, are much more likely to embrace newer codecs like HEVC, then browser operators are. Because browsers update at a crawl. Christopher Levy: 29:09 So, I mean, Google is the fastest browser updater in the business. But then, if you look at Safari, and IE and Edge, it's like, you know, waiting for your Grandmother to mail your birthday present. You get it like, four months later. But you're happy you got it, so I think that's the last kind of hidden thing, is that you know, within premium apps on devices, in a bring your own device model there's a greater chance that you're gonna get higher quality content sooner with DRM than you would in a set top box, or in a seat back implementation. Mark Donnigan: 29:40 Yeah this is a really important discussion, I think for any of our listeners who are planning video services, and maybe, sort of haven't been able to do that next level of research and are thinking, "hey, you know, I can just appoint HTML five, it will reduce complexity, it will get me to market faster." Those are all true, but you have to know what you're also not gonna be able to deliver to your customer. Mark Donnigan: 30:07 One of the other things, that I didn't hear you say, maybe I missed it, but I know one hundred percent, you know to be true, is that content licensing in some cases prohibits for example, HD in a browser, or certain browsers or in certain configurations. So yeah, you may be able to deliver in to that browser, but you're limited to SD, you know? 480p or maybe 720p, but not 1080p, so you're not able to deliver even the full quality. Mark Donnigan: 30:41 Now, in-flight entertainment, the bandwidths are so low that you know, I think 1080p is not very common anyway, but the point is, is that those are even things that you have to think about. Christopher Levy: 30:53 Well, a researcher David McCannon, he's pretty famous, he's a young guy over in the UK who previously, was responsible for a pretty significant kind of white hat hack that started to turn ugly. He's a pretty brilliant guy. He published some stuff on Monday of last week that indicated that he had breached Widevine's level three DRM. Which is the lowest level of DRM, mostly used in the Chrome browser, now it appears that what he breached, wasn't exactly Google's technology, but a third party[inaudible 00:31:30] technology that Google was using to wrap up their content decryption module that sits inside Chrome. Christopher Levy: 31:37 But, it's a good example of where, devices, especially Android devices, you know, they have hardware in them that allows hardware assisted key management. So, they have a hardware manage black box that sits on the device that is basically impenetrable. And so, that's another benefit of using devices. Christopher Levy: 32:02 Apple has the same thing, so Fairplay on IOS, taps into a trusted computing module that's on the chip that's in the iPhone or iPad. Same thing with Android with Google's DRM, you can get level one Widevine playback for HD, and 4K content on the device and then you can cast that out to a much bigger screen if you want over Chromecast, or over Airplay for example. So that's, that's another example where, you know, apps are much more secure than play back in the browser. Christopher Levy: 32:34 So, what has to happen now is Google's gotta go modify, and what they're in the process of, from what I understand, of updating their content decryption module for Widevine and Chrome, so that their level three use, which is what most of the operators use, is safe. Christopher Levy: 32:51 But again, they're operating on a non-native platform to them. Windows, in the most cases. Widevine also runs in Chrome on, on, on MacOS, but in those browser models, browsers are sitting on top of operating systems that the operator doesn't always own, and so that's again another benefit to using Premium apps. Mark Donnigan: 33:15 This is an awesome lead in to a discussion about AV1, and DRM support. I don't know, have you had the chance to do some research around you know DRM support for AV1? Christopher Levy: 33:30 Yeah, I mean, we've been following it pretty closely. We are really closely aligned with some companies that are working pretty seriously on it, I mean. We're very aligned with Google, and Bitmovin, and Amazon and Intel, and some of the other people that are involved in it. Christopher Levy: 33:47 But again, the big question is, at what point does AV1 start to appear in content in browsers with DRM's? And I guess, the problem that we kinda have right now, is that that hasn't really happened, and they've done some kind of stuff playing around with Firefox, to play AV1 content. But really, it's gonna be up to, again, it's gonna be up to Apple, Google, and Microsoft. Right? Because they are the ones that own the DRM, and the browser, and so again, you've got a weird. It's not a simple economy of supply and demand, it's, there's this third, you know, Robby Botter's hidden hand that's influencing who is gonna implement what. Christopher Levy: 34:43 You've got HEVC, which is widely deployed, heavily proven in the marketplace. It's gone through some royalty and licensing politics that are pretty consistent that all codecs go through. I kinda wish sometime, the encoding business had the same oligarchy god that DRM has, where Intertrust can just licensed everybody and be done with it. Christopher Levy: 35:07 But, HEVC in comparison to AV1, in HEVC there are tons of documents on you know, Apple's developer page, Google's developer portal, Microsoft's developer portal. Showing how to use their DRM with HEVC on different platforms, and there are numerous, numerous gibset manufacturers as you well know, and which we provided you a list of, that support it. And also, its supported in a lot of the browsers already, if not all of them. Christopher Levy: 35:36 AV1, on the other hand, is kind of nowhere with any of that yet, but it seems to be, you know, a little less encumbered with the intellectual property issues. But frankly, I feel like as it gets closer to being deployed, and people start to really get their hooks in to it, we're probably going to see the same thing happen to AV1 that happened to HEVC. But, I think it's gonna happen before it ever gets widely deployed, in my opinion. Dror Gill: 36:06 You think, when you say the thing that's gonna happen are you referring to patent accusations or patent infringement? Christopher Levy: 36:21 Yeah, I try not to pick a side, because you know let's face it. If you picked our entire industry, the two most researched dollar intensive things are codecs and DRM. You could build a Codec, and at the end of spending millions, throw it in the trash, because it didn't scale. You could build a DRM, and in the end, because you weren't doing a freedom to operate analysis ahead of time, find out that you built a great technology, but it's never gonna see the light of day in the market because you are infringing on someone else's IP. Christopher Levy: 36:55 I think what's going on with HEVC is kind of normal, right? Like, all these companies invested in it. And clearly, they intend to see their return on the investment, and they're looking at what happened with H- two-sixty-four, the patent pull stuff, all the, the kind of facts that we all know that there's quite a few companies in the business that aren't reporting royalties properly, and have kind of jumped the shark there. Christopher Levy: 37:20 So, I think HEVC has a better chance than AV1, if I were to weigh the two. Just because, it's, you know, all the points I've mentioned; much further widely deployed, chip support, browser support, DRM support. AV1 doesn't have any of that, and it doesn't have the encumbrances of potential legal battles, yet. But, I don't know. What do you guys think is gonna happen, when it comes time to walk the aisle with AV1? Dror Gill: 37:46 Indeed, nobody is giving you identification against any patent lawsuits for AV1. The companies involved in developing the codec itself, have signed agreements that they will not switch other, or the users of AV1, but this doesn't mean that somebody else will not claim any IP rights on algorithms used in AV1. Dror Gill: 38:18 And, on the other hand, the conclusion that we reached is that, the fact, it is well know that AV1, right now, is much more computationally complex then HEVC. Right now, it's like a hundred times more complex, and even the people involved in AV1 development have told us that in the end, when everything is optimized, it will still be five to ten time more complex than HEVC. Dror Gill: 38:49 And, we think that one of the reasons for that is all of that side stepping of patents. All of these techniques, which to be efficient in terms of bitrate consumption, as HEVC, but cannot use the same tools, and therefore I have to go in very weird ways around those protected methods in order to achieve the same result. And this is part of the problem, and why it is so computationally complex. Dror Gill: 39:26 Recently, I've come up with yet another conspiracy theory after hearing that a lot of the decisions made, somebody wrote this in a blog post. A lot of the decisions that were made during the development of AV1 were driven by the hardware companies were members of the AOM. Christopher Levy: 39:47 I was just gonna say that, Dror, is that A. There's not free lunch whether it's physics, mathematics, which is you know, part of physics in technology, in relationships, in religion, and that doesn't surprise me. Christopher Levy: 40:02 But, what I was gonna point out, was Occam's razor says, "the simplest answer is more than likely, the answer," is correct. I would say, that is what's driving it, because let's face it, I mean, there's not a person working on it that doesn't benefit from that. I'm pretty sure that Google, Amazon, Microsoft, Apple, all the other companies sell computing software, and technology and silicon, and intel, so I can't imagine why that wouldn't be the case. Christopher Levy: 40:32 But, you make a good point, that regardless of the fact that their trying to ignore the three laws of thermodynamics. I imagine they are, have a strategy for how they're going to sort that out, but the question is, will it really work? And, the only thing too, is if they don't adopt DRM into their message share pretty soon, and start showing examples of AV1 content with DRM, it's just gonna be another Ultraviolet. It's gonna be shiny. It's gonna sparkle. It's gonna have all the right looks and feels. It's got a cool logo. The stuff on the side is really cool, but will people use it, or is it just gonna be another augmented reality, virtual reality three-d, a year from now? Mark Donnigan: 41:14 You know, I sometimes find myself feeling a little agitated or sitting in a conference, and I'm listening to a panel, and I'm hearing either a panelist or even Mozilla, you know, saying, you know," it's coming, player support is coming. It's just months away. It's gonna be in the browser." And then they start, and I'm going, so, really? So, Sony pictures, and Warner Brothers is gonna allow you to play their movies inside a browser without DRM? Yeah. Let's see how that works. You know? Like? Then you've got up on the stage usually, or you hear speakers, and they're throwing off big service names, and Netflix is heavily behind AV1, so I am not naïve that Netflix is having these discussions, I'm sure. Mark Donnigan: 42:04 But the point is, that DRM is DRM. It has to be implemented. It has to, to work with the standards the content owners accept. But the fact that you don't hear DRM, it's sort of just... it's almost like, oh yeah, yeah. It's gonna be in the browser. It's gonna be supported. I'm like, that's just not how it works. It will come later. It's coming, don't worry about it.[inaudible 00:42:29] Christopher Levy: 42:28 I mean, nevermind the battle that was fought at the W3C by all the media companies just named, and a hundred more, along with Google, and Apple, and Microsoft to implement DRM in the browser, because they know that's where people want to view content on their computers, whether it be desktop or laptop. Christopher Levy: 42:46 But, they didn't do all the work, and engineering to get MSC and CDM's working to just all of the sudden, say, "see we're gonna throw it out the window because there's this new Codec in town." Mark Donnigan: 43:00 Yeah, exactly. Exactly. So, wow. Well, I'm looking at our time here. This has been an amazing discussion Christopher, and we absolutely, need to have you back because we didn't get to talk about players, and I know you guys are active, and I know also in the player development. So, I think, Dror, what do you think? I think a part two should be players. Dror Gill: 43:24 You know, Mark, Christopher did make this analogy between Codecs and DRM in one of the first episodes we told. Like the story of the Codecs, how they've been developed and DRM is also really a fascinating story, and even more because it's beyond standards that spam dozens of companies. It's really a few companies holding the power, holding the key, and that's also the DRM key. In the whole industry, and how it's gonna develop in the future I think would be really interesting to see whether we are going for true standards, finally, and a much easier life for consumers to play their content anywhere, or do we still have few years of struggling? So, really, thank you very much, Christopher. Mark Donnigan: 44:16 Christopher, your website is Buydrm.com, correct? Christopher Levy: 44:23 That's correct, and the blog is thedrmblog.com and once you guys get this podcast up and done, we'll go ahead and feature it on the blog, and I just wanted to quickly mention that in the next couple of days, we're gonna have a new blog post come out about deploying secure SDK's. And, we tackle a lot of the issues we talked about here in a generalist way. We do talk about our own SDK players, but I'll notify you when that blog is up. I think your readers will find it interesting. Christopher Levy: 44:53 We're also have an HEVC update on our blog, but after today once you post the final edited blog, then we'll go ahead and roll out our update that I provided you with regards to kinda where the market's at as well. Mark Donnigan: 45:09 Awesome. Awesome. Sounds good. Okay, well, we want to thank you again for listening to this incredibly engaging episode of the video insiders and until next time, what do we say Dror? Encode on? Is that our new..? Dror Gill: 45:29 Encode on! Encode happily! Mark Donnigan: 45:30 Encode happily, we've got to come up with something. Dror Gill: 45:32 Yeah, we need to invent something like, you can never compress too much. Mark Donnigan: 45:36 That's right, you can never compress too much, but you must preserve all the original quality. Alright, have a great day everyone. Thank you for listening. Christopher Levy: 45:45 Thank you Announcer: 45:47 Thank you for listening to the Video Insiders podcast. A production of Beamer limited. To begin using Beamer's Codecs today, go to Beamer.com/free to receive up to one hundred hours of no cost HEVC and H.264 transcoding every month.

Open source codec pioneer, Tom Vaughan, talks about the advantages & disadvantages of proprietary & open source technology. What he says may surprise you – despite which side of the fence you are on. The following blog post first appeared on the Beamr blog at: https://blog.beamr.com/2019/01/24/in-the-battle-between-open-source-proprietary-technology-does-video-win-podcast/ Video engineers dedicated to engineering encoding technologies are highly skilled and hyper-focused on developing the foundation for future online media content. Such a limited pool of experts in this field creates a lot of opportunity for growth and development, it also means there must be a level of camaraderie and cooperation between different methodologies. In past episodes, you've seen The Video Insiders compare codecs head-to-head and debate over their strengths and weaknesses. Today, they are tackling a deeper debate between encoding experts: the advantages and disadvantages of proprietary technology vs. community-driven open source. In Episode 05, Tom Vaughan surprises The Video Insiders as he talks through his take on open source vs. proprietary technology. Press play to hear a snippet from Episode 05, or click here for the full episode. Want to join the conversation? Reach out to TheVideoInsiders@beamr.com TRANSCRIPTION (lightly edited to improve readability only) Mark Donnigan: 00:00 In this episode, we talk with a video pioneer who drove a popular open source codec project before joining a commercial codec company. Trust me, you want to hear what he told us about proprietary technology, open source, IP licensing, and royalties. Announcer: 00:18 The Video Insiders is the show that makes sense of all that is happening in the world of online video, as seen through the eyes of a second generation codec nerd and a marketing guy who knows what iframes and macroblocks are. Here are your hosts, Mark Donnigan and Dror Gill. Mark Donnigan: 00:35 Okay. Mark Donnigan: 00:35 Well, welcome back everyone to this very special edition. Every edition is special, isn't it, Dror? Dror Gill: 00:43 That's right. Especially the first editions where everybody's so excited to see what's going to happen and how it would evolve. Mark Donnigan: 00:49 You know what's amazing, Dror, we had in the first 48 hours, more than 180 download. Dror Gill: 00:55 Wow. Mark Donnigan: 00:56 You know, we're like encoding geeks. I mean, are there even 180 of us in the world? Dror Gill: 01:01 I don't know. I think you should count the number of people who come to Ben Wagoner's compressionist breakfast at NAB, that's about the whole industry, right? Mark Donnigan: 01:09 Yeah. That's the whole industry. Mark Donnigan: 01:11 Hey, we want to thank, seriously in all seriousness, all the listeners who have been supporting us and we just really appreciate it. We have an amazing guest lined up for today. This is a little personal for me. It was IBC 2017, I had said something about a product that he was representing, driving, developing at the time. In fact, it was factually true. He didn't like it so much and we exchanged some words. Here's the ironic thing, this guy now works for us. Isn't that amazing, Dror? Click to view x265 vs. Beamr 5 speed and performance test. Dror Gill: 01:49 Yeah, isn't that amazing? Mark Donnigan: 01:52 You know what, and we love each other. The story ended well, talk about a good Hollywood ending. Mark Donnigan: 01:58 Well, we are talking today with Tom Vaughn. I'm going to let you introduce yourself. Tell the listeners about yourself. Tom Vaughn: 02:10 Hey Mark, hey Dror. Good to be here. Tom Vaughn: 02:12 As Mark mentioned, I'm Beamr's VP of strategy. Joined Beamr in January this year. Before that I was Beamr's, probably, primary competitor, the person who started and led the x265 project at MulticoreWare. We were fierce competitors, but we were always friendly and always friends. Got to know the Beamr team when Beamr first brought their image compression science from the photo industry to the video industry, which was three or four years ago. Really enjoyed collaborating with them and brainstorming and working with them, and we've always been allies in the fight to make new formats successful and deal with some of the structural issues in the industry. Dror Gill: 03:02 Let me translate. New formats, that means HEVC. Structural issues, that means patent royalties. Tom Vaughn: 03:08 Yes. Dror Gill: 03:09 Okay, you can continue. Tom Vaughn: 03:11 No need to be subtle here. Tom Vaughn: 03:13 Yeah, we had many discussions over the years about how to deal with the challenging macro environment in the codec space. I decided to join the winning team at Beamr this year, and it's been fantastic. Mark Donnigan: 03:28 Well, we're so happy to have you aboard, Tom. Mark Donnigan: 03:32 I'd like to just really jump in. You have a lot of expertise in the area of open source, and in the industry, there's a lot of discussion and debate, and some would even say there's religion, around open source versus proprietary technology, but you've been on both sides and I'd really like to jump into the conversation and have you give us a real quick primer as to what is open source. Tom Vaughn: 04:01 Well, open source is kind of basic what it says is that you can get the full source code to that software. Now, there isn't just one flavor of open source in terms of the software license that you get, there are many different open source licenses. Some have more restrictions and some have less restrictions on what you can do. There are some well known open source software programs and platforms, Linux is probably the most well known in the multimedia space, there's FFmpeg and Libav. There's VLC, the multimedia player. In the codec space, x264, x265, VP9, AV1, et cetera. Dror Gill: 04:50 I think the main attraction of open source, I think, the main feature is that people from all over the world join together, collaborate, each one contributes their own piece, then somehow this is managed together. Every bug that is discovered, anyone can fix it, because the source is open. This creates kind of a community and together a piece of software is created that is much larger and more robust than anything that a single developer could do on his own. Tom Vaughn: 05:23 Yeah, ideally the fact that the source code is open means that you have many sets of eyes, not only trying the program, but able to go through the source code and see exactly how it was written and therefore more code review can happen. On the collaboration side, you're looking for volunteers, and if you can find and energize many, many people worldwide to become enthusiastic and devote time or get their companies motivated to allocate developers full- or part-time to a particular open source project, you get that collaboration from many different types of people with different individual use cases and motivations. There are patches submitted from many different people, but someone has to decide, does that patch get committed or are there problems with that? Should it be changed? Tom Vaughn: 06:17 Designed by a committee isn't always the optimal, so someone or some small group has to decide what should be included, what should be left out. Dror Gill: 06:27 It's interesting to see, actually, the difference between x264 and x265 in this respect, because x264, the open source implementation of x264 was led by a group of developers, really independent developers, and no single company was owning or leading the development of that open source project. However, with x265, which is the open source implementation of HEVC, your previous company, MulticoreWare, has taken the lead and devoted, I assume, most of the development resources that have gone into the open source development, most of the contributions came from that company, but it is still an open source project. Tom Vaughn: 07:06 That's right. x264 was started by some students at a French university, and when they were graduating, leaving the university, they convinced the university to enable them to take the code with them, essentially under an open source license. It was very much grassroots open source beginnings and execution where developers may come and go, but it was a community collaboration. Tom Vaughn: 07:31 I started x265 at MulticoreWare with a couple of other individuals, and the way we started it was finding some commercial companies who expressed a strong interest in such a thing coming to life and who were early backers commercially. It was quite different. Then, because there's a small team of full-time developers on it working 40 hours plus a week, that team is moving very fast, it's organized, it's within a company. There was less of a need for a community. While we did everything we could to attract more external contributors, attracting contributors is always a challenge of open source projects. Mark Donnigan: 08:14 What I hear you saying, Tom, is it sounds like compared to the x264 project, the x265 project didn't have as large of a independent group of contributors. Is that …? Tom Vaughn: 08:29 Well, x264 was all independent contributors. Mark Donnigan: 08:32 That's right. Tom Vaughn: 08:33 And still is, essentially. There are many companies that fund x264 developers explicitly. Chip companies will fund individual developers to optimize popular open source software projects for their instruction set. AVX, AVX2, AVX512, essentially, things like that. Tom Vaughn: 08:58 HEVC is significantly more complex than AVC, and I think, if I recall correctly, x265 already has three times the number of commits than x264, even though it's only been in existence for one third of the life. Dror Gill: 09:12 So Tom, what's interesting to me is everybody's talking about open source software being almost synonymous with free software. Is open source really free? Is it the same? Tom Vaughn: 09:23 It can be at times. One part depends on the license and the other part depends on how you're using the software. For example, if it's a very open license like Apache, or BSD, or UIUC, that's an attribution only license, and you're pretty much free to create modifications, incorporate the software in your own works and distribute the resulting system. Tom Vaughn: 09:49 Software programs like x264 and x265 are licensed under the GNU GPL V2, that is an open source license that has a copyleft requirement. That means if you incorporate that in a larger work and distribute that larger work, you have to open source not only your modifications, but you have to open source the larger work. Most commercial companies don't want to incorporate some open source software in their commercial product, and then have to open source the commercial product. The owners of the copyright of the GPL V2 code, x264 LLC or MulticoreWare, also offer a commercial license, meaning you get access to that software, not under the GNU GPL V2, but under a separate, different license, in which case for you, it's not open source anymore. Your commercial license dictates what you can and can't do. Generally that commercial license doesn't include the copyleft requirement, so you can incorporate it in some commercial product and distribute that commercial product without open sourcing your commercial product. Dror Gill: 10:54 Then you're actually licensing that software as you would license it from a commercial company. Tom Vaughn: 10:59 Exactly. In that case it's not open source at all, it's a commercial license. Dror Gill: 11:04 It's interesting what you said about the GPL, the fact that anything that you compile with it, create derivatives of, incorporate into your software, you need to open source those components that you integrate with as well. I think this is what triggered Steve Ballmer to say in 2001, he said something like, “Open source is a cancer that spreads throughout your company and eats your IP.” That was very interesting. I think he meant mostly GPL because of that requirement, but the interesting thing is that he said that in 2001, and in 2016 in an interview, he said, “I was wrong and I really love Linux.” Today Microsoft itself open sources a lot of its own development. Mark Donnigan: 11:48 That's right. Yeah, that's right. Mark Donnigan: 11:50 Well Tom, let's … This has been an awesome discussion. Let's bring it to a conclusion. When is proprietary technology the right choice and when is open source maybe the correct choice? Can you give the listeners some guidelines? Tom Vaughn: 12:08 Sure, people are trying to solve problems. Engineers, companies are trying to build products and services, and they have to compete in their own business environment. Let's say you're a video service and you run a video business. The quality of that video and the efficiency that you can deliver that video matters a lot. We know what those advantages of open source are, and all things being equal, people gravitate towards open source a lot because engineers feel comfortable actually seeing the source code, being able to read through it, find bugs themselves if pushed to the limit. Tom Vaughn: 12:45 At the end of the day, if an open source project can't produce the winning implementation of something, you shouldn't necessarily use it just because it's open source. At the end of the day you have a business to run and what you want is the most performant libraries and platforms to build your business around. If you find that a proprietary implementation in the long run is more cost effective, more efficient, higher performance, and the company that is behind that proprietary implementation is solid and is going to be there for you and provide a contractual commitment to support you, there's no reason to not choose some proprietary code to incorporate into your product or service. Tom Vaughn: 13:32 When we're talking about codecs, there are particular qualities I'm looking for, performance, how fast does it run? How efficiently does it utilize compute resources? How many cores do I need in my server to run this in real time? And compression efficiency, what kind of video quality can I get at a given bit rate under a given set of conditions? I don't want the second best implementation, I want the best implementation of that standard, because at scale, I can save a lot of money if I have a more efficient implementation of that standard. Mark Donnigan: 14:01 Those are excellent pointers. It just really comes back to we're solving problems, right? It's easy to get sucked into religious debates about some of these things, but at the end of the day we all have an obligation to do what's right and what's best for our companies, which includes selecting the best technology, what is going to do the best job at solving the problems. Mark Donnigan: 14:24 Thank you again for joining us. Tom Vaughn: 14:25 My pleasure, thank you. Dror Gill: 14:26 I would also like to thank you for joining us, not only joining us on this podcast, but also joining Beamr. Mark Donnigan: 14:32 Absolutely. Mark Donnigan: 14:33 Well, we want to thank you the listener for, again, joining The Video Insiders. We hope you will subscribe. You can go to thevideoinsiders.com and you can stream from your browser, you can subscribe on iTunes. We're on Spotify. We are on Google Play. We're expanding every day. Announcer: 14:57 Thank you for listening to The Video Insiders podcast, a production of Beamr Limited. To begin using Beamr's codecs today, go to Beamr.com/free to receive up to 100 hours of no-cost HEVC and H.264 transcoding every month.

E04: In this episode, The Video Insider's catch up with industry expert, Tim Siglin, to discuss HEVC implementation trends that counter previous assumptions, notable 2018 streaming events, and what's coming in 2019. The following blog post first appeared on the Beamr blog at: https://blog.beamr.com/2019/01/01/2018-the-year-hevc-took-flight/ By now, most of us have seen the data and know that online video consumption is soaring at a rate that is historically unrivaled. It's no surprise that in the crux of the streaming era, so many companies are looking to innovate and figure out how to make their workflows or customers workflows better, less expensive, and faster. In Episode 4 of The Video Insiders, we caught up with streaming veteran Tim Siglin to discuss HEVC implementation trends that counter previous assumptions, notable 2018 streaming events, and what's coming in 2019. Tune in to hear The Video Insiders cover top-of-mind topics: HEVC for lower resolutions Streaming the World Cup Moving from digital broadcast to IP-based infrastructure What consumers aren't thinking about when it comes to 4K and HDR Looking forward into 2019 & beyond Tune in to Episode 04: 2018, the Year HEVC Took Flight or watch the video below. Want to join the conversation? Reach out to TheVideoInsiders@beamr.com TRANSCRIPTION (lightly edited to improve readability only) Mark Donnigan: 00:00 On today's episode, the Video Insiders sit down with an industry luminary who shares results of a codec implementation study, while discussing notable streaming events that took place in 2018 and what's on the horizon for 2019. Stay tuned. You don't want to miss receiving the inside scoop on all this and more. Announcer: 00:22 The Video Insiders is the show that makes sense of all that is happening in the world of online video, as seen through the eyes of a second generation Kodak nerd and a marketing guy who knows what I frames and macroblocks are. Here are your hosts, Mark Donnigan and Dror Gill. Mark Donnigan: 00:40 Welcome, everyone. I am Mark Donnigan, and I want to say how honored Dror and I are to have you with us. Before I introduce this very special guest and episode, I want to give a shout of thanks for all of the support that we're receiving. It's really been amazing. Dror Gill: 00:58 Yeah. Yeah, it's been awesome. Mark Donnigan: 00:59 In the first 48 hours, we received 180 downloads. It's pretty amazing. Dror Gill: 01:06 Yeah. Yeah, it is. The industry is not that large, and I think it's really an amazing number that they're already listening to the show from the start before the word of mouth starts coming out, and people spread the news and things like that. We really appreciate it. So, if it's you that is listening, thank you very much. Mark Donnigan: 01:29 We really do aim for this to be an agenda-free zone. I guess we can put it that way. Obviously, this show is sponsored by Beamr, and we have a certain point of view on things, but the point is, we observed there wasn't a good place to find out what's going on in the industry and sort of get unbiased, or maybe it's better to say unfiltered, information. That's what we aim to do in every episode. Mark Donnigan: 01:57 In this one, we're going to do just that. We have someone who you can definitely trust to know what's really happening in the streaming video space, and I know he has some juicy insights to share with us. So, without further ado, let's bring on Tim Siglin. Tim Siglin: 02:15 Hey, guys. Thank you for having me today and I will definitely try to be either as unfiltered or unbiased as possible. Mark Donnigan: 02:21 Why don't you give us a highlight reel, so to speak, of what you've done in the industry and, even more specifically, what are you working on today? Tim Siglin: 02:31 Sure. I have been in streaming now for a little over 20 years. In fact, when Eric Schumacher-Rasmussen came on as the editor at StreamingMedia.com, he said, “You seemed to be one of the few people who were there in the early days.” It's true. I actually had the honor of writing the 10-year anniversary of Streaming Media articles for the magazine, and then did the 20-year last year. Tim Siglin: 02:57 My background was Motion Picture production and then I got into video conferencing. As part of video conferencing, we were trying to figure out how to include hundreds of people in a video conference, but not need necessarily have them have two-way feedback. That's where streaming sort of caught my eye, because, ultimately, for video conferencing we maybe needed 10 subject matter experts who would talk back and forth, and together a hundred, then it went to thousands, and now hundreds of thousands. You can listen in and use something like chat or polling to provide feedback. Tim Siglin: 03:31 For me, the industry went from the early revolutionary days of “Hey, let's change everything. Let's get rid of TV. Let's do broadcast across IP.” That was the mantra in the early days. Now, of course, where we are is sort of, I would say, two-thirds of the way there, and we can talk a little bit about that later. The reality is that the old mediums are actually morphing to allow themselves to do heap, which is good, to compete with over the top. Tim Siglin: 04:01 Ultimately, what I think we'll find, especially when we get to pure IP broadcast with ATSC 3.0 and some other things for over-the-air, is that we will have more mediums to consume on rather than fewer. I remember the early format ways and of course we're going to talk some in this episode about some of the newer codec like HEVC. Ultimately, it seems like the industry goes through the cycles of player wars, format wars, browser wars, operating system wars, and we hit brief periods of stability which we've done with AVC or H.264 over the last probably eight years. Tim Siglin: 04:46 Then somebody wants to stir the pot, figure out how to either do it better, less expensively, faster. We go back into a cycle of trying to decide what the next big thing will be. In terms of what I'm working on now, because I've been in the industry for almost 21 years. Last year, I helped start a not-for-profit called Help Me Stream, which focuses on working with NGOs in emerging economies, trying to help them actually get into the streaming game to get their critical messages out. Tim Siglin: 05:18 That might be emerging economies like African economies, South America, and just the idea that we in the first world have streaming down cold, but there are a lot of messages that need to get out in emerging economies and emerging markets that they don't necessarily have the expertise to do. My work is to tie experts here with need there and figure out which technologies and services would be the most appropriate and most cost effective. Mark Donnigan: 05:46 That's fascinating, Tim. Tim Siglin: 05:48 The other thing I'm working on here, just briefly, is we're getting ready for the Streaming Media Sourcebook, the 2019 sourcebook. I'm having to step back for the next 15 days, take a really wide look at the industry and figure out what the state of affairs are. Dror Gill: 06:06 That's wonderful. I think because this is exactly the right point, is one you end and the other one begins, kind of to summarize where we've been in 2018, what is the state of the industry and the fact that you're doing that for the sourcebook, I think, ties in very nicely with our desire to hear from you an overview of what were the major milestones or advancements that were made in the streaming industry in 2018, and then looking into next year. Dror Gill: 06:39 Obviously, the move to IP, getting stronger and stronger, now the third phase after analog and digital, now we have broadcast over IP. It's interesting what you said about broadcasters not giving up the first with the pure OTT content providers. They have a huge business. They need to keep their subscribers and lower their churn and keep people from cutting the cord, so to speak. Dror Gill: 07:04 The telcos and the cable companies still need to provide the infrastructure for Internet on top of which the over-the-top providers and their content, but they still need to have more offering and television and VLD content in order to keep their subscribers. It's very interesting to hear how they're doing it and how they are upgrading themselves to the era of IP. Tim Siglin: 07:30 I think, Dror, you hit a really major point, which is we, the heavy lift … I just finished an article in ATSC 3.0 where I talk about using 2019 to prepare for 2020 when that will go live in the U.S. The heavy lift was the analog to digital conversion. The slightly easier lift is the conversion from digital to IP, but it still requires significant infrastructure upgrade and even transmission equipment to be able to do it correctly for the over-the-year broadcasters and cable. Dror Gill: 08:07 That's right. I think on the other hand, there is one big advantage to broadcast, even broadcast over-the-air. That is the ability to actually broadcast, the ability to reach millions, tens of millions, hundreds of millions of people over a single channel that everybody is receiving. Whereas, because of historic reasons and legacy reasons in IP, we are limited, still, when you broadcast to the end user to doing everything over unicast. When you do this, it creates a tremendous load on your network. You need to manage your CDNs. Dror Gill: 08:46 I think we've witnessed in 2018 on one hand very large events being streamed to our record audience. But, on the other hand, some of them really failed in terms of user experience. It wasn't what they expected because of the high volume of users, and because more and more people have discovered the ability to stream things over IP to their televisions and mobile devices. Can you share with us some of the experience that you have, some of the things that you're hearing about in terms of these big events where they had failures and what were the reasons for those failures? Tim Siglin: 09:30 I want to reiterate the point you made on the OTA broadcast. It's almost as if you have read the advanced copy of my article, which I know you haven't because it's only gone to the editor. Dror Gill: 09:42 I don't have any inside information. I have to say, even though we are the Video Insiders. Mark Donnigan: 09:47 We are the Video Insiders. That's right. Dror Gill: 09:49 We are the Video Insiders, but … Mark Donnigan: 09:49 But no inside information here. Dror Gill: 09:51 No inside information. I did not steal that copy. Tim Siglin: 09:55 What I point out in that article, Dror, I think which will come out in January shortly after CES is basically this. We have done a good job in the streaming industry, the OTT space of pushing the traditional mediums to upgrade themselves. One of the things as you say with OTA, that ability to do essentially a multicast from a tower wirelessly is a really, really good thing, because to get us to scale, and I think about things like the World Cup, the Olympics and even the presidential funeral that's happened here in December, there are large-scale events that we in the OTT space just can't handle, if you're talking about having to build the capacity. Tim Siglin: 10:39 The irony is, one good ATSC transmission tower could hit as many people as we could handle essentially globally with the unicast (OTT) model. If you look at things like that and then you look at things like EMBMS in the mobile world, where there is that attempt to do essentially a multicast, and it goes to points like the World Cup. I think one of the horror stories in the World Cup was in Australia. There was a mobile provider named Optus who won the rights to actually do half of the World Cup preliminary games. In the first several days, they were so overwhelmed by the number of users who wanted to watch and were watching, as you say, in a unicast model that they ended up having to go back to the company they had bid against who had the other half of the preliminaries and ask them to carry those on traditional television. Tim Siglin: 11:41 The CEO admitted that it was such a spectacular failure that it damaged the brand of the mobile provider. Instead of the name Optus being used, everybody was referring to it as “Floptus.” You don't want your brand being known as the butt of jokes for an event that only happens once every four years that you have a number of devotees in your market. And heaven forbid, it had been the World Cup for cricket, there would have been riots in the street in Sydney and Melbourne. Thank goodness it was Australia with soccer as opposed to Australia with cricket. Tim Siglin: 12:18 It brings home the point that we talk about scale, but it's really hard to get to scale in a unicast environment. The other event, this one happened, I believe, in late 2017, was the Mayweather fight that was a large pay-per-view event that was streamed. It turned out the problem there wasn't as much the streams as it was the authentication servers were overwhelmed in the first five minutes of the fight. So, with authentication gone, it took down the ability to actually watch the stream. Tim Siglin: 12:53 For us, it's not just about the video portion of it, it's actually about the total ecosystem and who you're delivering to, whether you're going to force caps into place because you know you can't go beyond a certain capacity, or whether you're going to have to partner up with traditional media like cable service providers or over-the-air broadcasters. Mark Donnigan: 13:14 It's a really good point, Tim. In the World Cup, the coverage that I saw, it was more of, I'd almost say or use the phrase, dashed expectations. Consumers, they were able to watch it. In most cases, I think it played smoothly. In other words, the video was there, but HDR signaling didn't work or didn't work right. Then it looked odd on some televisions or … Tim Siglin: 13:40 In high frame rate … Tim Siglin: 13:43 20 frames a second instead of 60 frames a second. Mark Donnigan: 13:48 Exactly. What's interesting to me is that, what I see is, the consumer, they're not of course walking around thinking as we are, like frame rate and color space and resolution. They are getting increasingly sensitive to where they can look at video now and say, “That's good video,” or “That doesn't look right to me.” I know we were talking before we started recording about this latest Tom Cruise public service announcement, which is just super fascinating, because it … Tim Siglin: 14:24 To hear him say motion interpolation. Mark Donnigan: 14:26 Yeah. Maybe we should tell the audience, for those, since it literally just came out I think today, even. But you want to tell the audience what Tom Cruise is saying? Tim Siglin: 14:38 Essentially, Tom Cruise was on the set of Top Gun, as they're shooting Top Gun. Another gentleman did a brief PSA for about a minute asking people to turn off motion interpolation on their televisions, which motion interpolation essentially takes a 24-frame per second and converts it to 30 frames per second by adding phantom frames in the middle. Because Mission Impossible: Fallout is just being released for streaming, Cruise was concerned and obviously others were concerned that some of the scenes would not look nearly as good with motion interpolation turned on. Tim Siglin: 15:17 I think, Mark, we ought to go to a PSA model, asking for very particular things like, “How do you turn HDR on? How do you …” Those types of things, because those get attention in a way that you and I or a video engineer can't get that attention. Dror Gill: 15:33 How do you know if what you're getting is actually 4K or interpolate HD, for example? Tim Siglin: 15:38 Especially in our part of the industry, because we will call something OTT 4K streaming. That may mean that it fits in a 4K frame, but it doesn't necessarily mean it's that number of pixels being delivered. Dror Gill: 15:52 It can also mean that the top layer in your adaptive bit rate stream is 4K, but then if you don't have enough bandwidth, you're actually getting the HD layer or even lower. Tim Siglin: 16:01 Exactly. Dror Gill: 16:02 Even though it is a 4K broadcast and it is 4K content. Sometimes, you can be disappointed by that fact as well. Mark Donnigan: 16:11 I have to give a very, very funny story directly related, and this happened probably, I don't know, maybe, at least 18 months ago, maybe two years ago. I'm sitting on an airplane next to this guy. It's the usual five-minute, get acquainted before we both turn on our computers. Anyway, when someone asks, “What do you do?” I generally just say, “I work for a video software company,” because how do you explain digital encoding? Most people just sort of stop at that, and don't really ask more. Mark Donnigan: 16:44 But this guy is like, “Oh, really?” He said, “So, I just bought a 4K TV and I love it.” He was raving about his new Samsung TV. Of course, he figured I'm a video guy. I would appreciate that. I said, “Hey.” “So, you must subscribe to Netflix.” “Yes. Yes, of course,” he says. I said, “What do you think of the Netflix quality? It looks great, doesn't it?” Mark Donnigan: 17:10 He sort of hem and hawed. He's like, “Well, it really … I mean, yeah. Yeah, it looks great, but it's not quite … I'm just not sure.” Then, I said, “I'm going to ask you two questions. First of all, are you subscribed to the 4K plan?” He was. Then I said, “How fast is your Internet at home.” He's like, “I just have the minimum. I don't know. I think it's the 20 megabit package,” or whatever it was. I don't remember the numbers. Mark Donnigan: 17:38 I said, “There's this thing.” And I gave him like a 30-second primer on adaptive bit rate, and I said, “It is possible, I have no idea of your situation, that you might be watching the HD version.” Anyway, he's like, “Hah, that's interesting.” I connect with the guy on LinkedIn. Three days later, I get this message. He says, “I just upgraded my Internet. I now have 4K on my TV. It looks awesome.” Mark Donnigan: 18:04 On one hand, the whole situation was not surprising and, yet, how many thousands, tens of thousands, maybe millions of people are in the exact same boat? They've got this beautiful TV. It could be because they're running some low-end router in the house. It could be they truly have a low end bandwidth package. There could be a lot of reasons why they're not getting the bandwidth. They're so excited about their 4K TV. They're paying Netflix to get the top layer, the best quality, and they're not even seeing it. It's such a pity. Tim Siglin: 18:37 I had a TSA agent asked me that same question, Mark, when I came through customs. I'm like, “Sure. I'll stand here and answer that question for you.” The router was actually what I suggested that he upgrade, because he said his router was like this (old unit). Mark Donnigan: 18:53 In a lot of homes, it's a router that's 15 years old and it just isn't (up to the task). Tim Siglin: 18:58 But it brings out the point that even as we're talking about newer codecs and better quality, even if we get a lower sweet spot in terms of 4K content (streaming bandwidth), or as we found in the survey that we worked on together, that using HEVC for 1080p or 720p, if the routers, if the software in the chain is not updated, the delivery quality will suffer in a way that people who have a tuned television and seen the consistent quality aren't certain what to do to fix when they use an over-the-top service. Tim Siglin: 19:34 I think this is a key for 2019. As we prepare for ATSC 3.0 on over-the-air broadcast where people will be able to see pristine 4K, it will actually force those of us in the OTT space to up our game to make sure that we're figuring out how to deliver across these multiple steps in a process that we don't break. Dror Gill: 19:54 You really see ATSC 3.0 as a game-changer in 2019? Tim Siglin: 19:59 What I see it as is the response from the broadcast industry to, A) say that they're still relevant, which I think is a good political move. And, B) it provides the scale you were talking about, Dror. See, I think what it does is it at least puts us in the OTT space on notice that there will be in certain first world countries a really decent quality delivery free of charge with commercials over the air. Tim Siglin: 20:31 It takes me back to the early days of video compression when, if you had a good class-one engineer and an analog NTSC transmission system, they could give you really good quality if your TV was tuned correctly. It only meant having to tune your TV. It didn't mean having to tune your router or having to tune your cable modem, having to tune your settings on your TV. I think that's where the game-changer may be, is that those tuner cards, which will send HDR signaling and things like that with the actual transmission, are going to make it much easier for the consumer to consume quality in a free scenario. I think that part of it is a potential game-changer. Mark Donnigan: 21:19 That's interesting. Tim, we worked together earlier this year on a survey, an industry survey that I think it would be really, really interesting to listeners to talk about. Shall we pivot into that? Maybe you can share some of the findings there. Tim Siglin: 21:38 Why don't you take the lead on why Beamr wanted to do that? Then I'll follow up with some of the points that we got out of it. Mark Donnigan: 21:46 Obviously, we are a codec developer. It's important for us to always be addressing the market the way that the market wants to be addressed, meaning that we're developing technologies and solutions and standards that's going to be adopted. Clearly, there has been, especially if we rewind a year ago or even 18 months ago, AV1 was just recently launched. There were still questions about VP9. Mark Donnigan: 22:19 Obviously, H264 AVC is the standard, used everywhere. We felt, “Let's go out to the industry. Let's really find out what the attitudes are, what the thinking is, what's going on ‘behind closed doors' and find out what are people doing.” Are they building workflows for these new advanced codecs? How are they going to build those workflows? That was the impetus, if you will, for it. Mark Donnigan: 22:49 We are very happy, Tim, to work with you on that and of course Streaming Media assisted us with promoting it. That was the reason we did it. I know there were some findings that were pretty predictable, shall we say, no surprises, but there were some things that I think were maybe a little more surprising. So, maybe if you like to share some of those. Tim Siglin: 23:12 Yeah. I'll hit the highlights on that. Let me say too that one of the things that I really like about this particular survey, there was another survey that had gone on right around that time that essentially was, “Are you going to adopt HEVC?” What we took the approach on with this survey was to say, “Okay. Those of you who've already adopted HEVC, what are the lessons that we can learn from that?” Tim Siglin: 23:36 We didn't exclude those who were looking at AV1 or some of the other codes, even VP9, but we wanted to know those people who used HEVC. Were they using it in pilot projects? Were they thinking about it? Were they using it in actual production? What we found in the survey is that AVC, or H.264, was still clearly dominant in the industry, but that the ramp-up to HEVC was moving along much faster than at least I … I believed. Mark, I told you when we started the survey question creation, which was about a year ago and then launched it in early 2018, I expected we wouldn't see a whole lot of people using HEVC in production. Tim Siglin: 24:23 I was pleasantly surprised to say that I was wrong. In fact, I think you mentioned in our recent Streaming Media West interview that there was a statistic you gave about the number of households that could consume HEVC. Was it north of 50%? Mark Donnigan: 24:40 Yeah, it's more than 50%. What's interesting about that number is that that actually came from a very large MSO. Of course, they have a very good understanding of what devices are on their network. They found that there was at least one device in at least 50% of their homes that could receive and decode, playback, HEVC. That's about as real world as you can get. Tim Siglin: 25:06 What was fascinating to me too in this study was, we asked open-ended questions, which is what I've done in the research projects for the last 25 years both the video conferencing and streaming. One of the questions we asked was, “Do you see HEVC as only a 4K solution or do you see it as an option for lower resolutions?” It turned out overwhelmingly, people said, “We not only see it for 4K. We see it for high-frame rate (HFR) 1080p, standard frame rate 1080p, with some HDR.” Tim Siglin: 25:40 Not a majority, but a large number of respondents said they would even see it as a benefit at 720p. What that tells me is, because we had a large number of engineers, video engineers, and we also have people in business development who answer these questions, what it tells me is that companies know as we scale because of the unicast problem that Dror pointed out in the beginning that scaling with a codec that consumes more bandwidth is a good way to lose money, kind of like the joke that the way a rich man can lose money really fast is to invest in an airline. Tim Siglin: 26:19 If indeed you get scale with AVC, you could find yourself with a really large bill. That look at HEVC is being not just for 4K, HDR, or high frame rate in the future, but also for 1080p with some HDR and high frame rate. It tells me that the codec itself or the promise of the codec itself was actually really good. What was even more fascinating to me was the number of companies that had AVC pipelines that were actually looking to integrate HEVC into those same production pipe. Tim Siglin: 26:55 It was much easier from a process standpoint to integrate HEVC into an AVC pipeline, so in other words, H265 into H264 pipeline than it was to go out of house and look at something like AV1 or VP9, because the work that was done on HEVC builds on the benefits that were already in place in AVC. Of course, you got Apple who has HLS, HTTP Live Streaming, and a huge ecosystem in terms of iPhones and iPads, laptops and desktops supporting HEVC not just as a standard for video delivery, but also with the HEIC or HEIF image format, now having all of their devices shoot images using HEVC instead of JPEG. That in and of itself drives forward adoption of HEVC. I think you told me since that survey came out, probably now seven months ago, you all have continued to see the model of all-in HEVC adoption. Dror Gill: 28:03 This is what we promote all the time. It's kind of a movement. Are you all in HEVC or are you doing it just for 4K, just where you have to do it? We really believe in all-in HEVC. Actually, this week, I had an interesting discussion with one of our customers who is using our optimization product for VOD content, to reduce bit-rate of H.264 (streams). He said, “I want to have a product. I want to have a solution for reducing bit-rates on our live channels.” Dror Gill: 28:32 So, I asked them, “Okay. Why don't you just switch your codec to HEVC?” He said, “No, I can't do that.” I said, “Why not?” He said, “You know compatibility and things like that.” I asked, “Okay. What are you using? What are you delivering to?” He said, “We have our own set-top boxes (STB), IP set-top boxes which we give out to our customers. Well, these are pretty new.” So, they support HEVC. I'm okay there. “Then we have an Apple TV app.” “Okay, Apple TV has a 4K version. So, it supports HEVC. All of the latest Apple TV devices have HEVC. That's fine.” “Then we have smartphone apps, smart TV apps for Android TV and for the LG platform.” Dror Gill: 29:15 Obviously, TV's support 4K. So, I'm okay there. With delivering to mobile devices, all the high-end devices already support HEVC. He was making this estimate that around 50 to 60% of his viewers are using devices that are HEVC capable. Suddenly, he's thinking, “Yeah, I can do that. I can go all in HEVC. I will continue, of course, to support H.264 for all of the devices that don't support HEVC. But if I can save 50% of the bandwidth to 50 to 60% of my customers, that's a very big savings.” Mark Donnigan: 29:48 What's interesting about this conversation, Dror, is first of all I'm pretty certain that the operator you're talking with is different than the operator that I shared, found the exact same thing. This is a consistent theme, is that pretty much in developed parts of the world, it really is true that 50% or more of the users can today receive HEVC. This number is only growing. It's not like it's static It is just growing. Next year, I don't know if that number will be 60% or 70%, but it's going to be even bigger. Mark Donnigan: 30:27 What's fascinating is that, again, we've said earlier, that the consumer is getting just more aware of quality, and they're getting more aware of when they're being underserved. For operators who are serving to lowest common denominator, which is to say, AVC works across all my devices, and it's true. AVC works on all the high-end devices equally well, but you're under-serving a large and growing number of your users. Mark Donnigan: 31:01 If your competitors are doing the same, then I guess you could say … well, “Who are they going to switch to?” But there are some fast-moving leaders in the space who are either planning or they're shortly going to be offering better quality. They're going to be extending HEVC into lower bit rates or lower resolutions, that is, and therefore lower bit rates, and the consumers are going to begin to see like, “Well, wait a second. This service over here that my friend has or we have another subscription in the household, how come the video looks better?” They just begin to migrate there. I think it's really important when we have these sorts of conversations to connect to this idea that don't underserve your consumer in an effort to be something to everybody. Tim Siglin: 31:57 I would add two other quick things to that, Mark. One is, we've always had this conversation in the industry about the three-legged stool of speed, quality and bandwidth in terms of the encoding. Mark Donnigan: 32:09 That's right. Tim Siglin: 32:09 Two of those are part of the consumer equation, which is quality and bandwidth. Then, oftentimes, we've had to make the decision between quality and bandwidth. If the argument is ostensibly that HEVC as it stands right now, had a couple years of optimization, can get us to about, let's say, 40%. Let's not even say 50%. For equivalent quality, it can get us to 40% bandwidth reduction. Why wouldn't you switch over to something like that? Tim Siglin: 32:39 Then the second part, and I have to put a plugin for what Eric Schumacher-Rasmussen and the Streaming Media team did at Streaming Media West by having Roger Pantos come and speak, Roger Pantos being of course the inventor of HLS, and I'm not a huge fan of HLS, just because of the latency issues. But he pointed out in his presentation, his tutorial around HLS that you can put two different codecs in a manifest file. There is absolutely no reason that an OTT provider could not provide both HEVC and AVC within the same manifest file and then allow the consumer device to choose. Tim Siglin: 33:22 When Dror mentioned the company who has the OTT boxes that they give away, they could easily set a flag in those boxes to say, “If you're presented with a manifest file that has AVC and HEVC, go with HEVC to lower the bandwidth, overall.” The beauty is it's a technical issue at this point and it's a technical implementation issue, not a ‘can we make it work?' Because we know that it works based around the HLS. Mark Donnigan: 33:54 This is excellent. Tim, let's wrap this up, as I knew it would be. It has just been an awesome conversation. Thank you for sharing all your years of collective experience to give some insight into what's happening in the industry. Let's look at 2019. I know we've been talking a little bit about … you've made references to ATSC 3.0. Some of our listeners will be going to CES. Maybe there's some things that they should be looking at or keeping their eyes opened for. What can you tell us about 2019? Tim Siglin: 34:35 Here's what I think 2019 is bringing. We have moved in the cloud computing space and you all are part of this conversation at Beamr. We've moved from having cloud-based solutions that were not at parity with on-premise solutions to actually in 2018 reaching parity between what you could do in an on-premise solution versus the cloud. Now, I think in 2019, what we're going to start seeing is a number of features in cloud-based services, whether it's machine learning, which the popular nomenclature is AI, but I really like machine learning as a much better descriptor, whether it's machine learning, whether it's real-time transcoding of live content, whether it's the ability to simultaneously spit out AVC and HEVC like we've been talking about here that the cloud-based solutions will move beyond parity with the on-premise solutions. Tim Siglin: 35:35 There always will be needs for the on-premise parts from a security standpoint in sort of the industries, but I don't think that will inhibit cloud-based in 2019. If people are going to CES, one of the things to look at there, for instance, is a big leap in power consumption savings for mobile devices. I'm not necessarily talking about smartphones, because the research I've done says the moment you turn GPS on, you lose 25% of battery. Tablets have the potential to make a resurgence in a number of areas for consumers and I think we'll see some advances in battery (capacity). Tim Siglin: 36:19 Part of that goes to HEVC, which as we know is a much harder codec to decode. I think the consumer companies are being forced into thinking about power consumption as HEVC becomes more mainstream. That's something I think people should pay attention to as well. Then, finally, HDR and surround sound solutions, especially object placement like Dolby Atmos and some of these others, will become much more mainstream as a way to sell flat panels and surround sound systems. Tim Siglin: 36:56 We sort of languished in that space. 4K prices have dropped dramatically in the last two years, but we're not yet ready for 8K. But I think we'll see a trend toward fixing some of the audio problems. In the streaming space, to fix those audio problems, we need to be able to encode and encapsulate into sort of the standard surround sound model. Those are three areas that I would suggest people pay attention. Mark Donnigan: 37:25 Well, thank you for joining us, Tim. It's really great to have you on. We'll definitely do this again. We want to thank you, the listener, for supporting the Video Insiders. Until the next episode. Happy encoding! Announcer: 37:39 Thank you for listening to the Video Insiders Podcast, a production of Beamr Imaging Limited. To begin using Beamr's codecs today, go to Beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

E03: What does the future hold for video codecs? This week, The Video Insiders look at the past and present to assess the future landscape of video encoding as they discuss where AVC, VP9, and VVC fit into the codec stew. The following blog post first appeared on the Beamr blog at: https://blog.beamr.com/2018/12/15/the-future-of-3-character-codecs-avc-vp9-vvc/ Anyone familiar with the streaming video industry knows that we love our acronyms. You would be hard-pressed to have a conversation about the online video industry without bringing one up… In today's episode, The Video Insiders focus on the future of three-character codecs: AVC, VP9, and VVC. But before we can look at the future, we have to take a moment to revisit the past. The year 2018 marks the 15-year anniversary of AVC and in this episode, we visit the process and lifecycle of standardization to adoption and what that means for the future of these codecs. Want to join the conversation? Reach out to TheVideoInsiders@beamr.com. TRANSCRIPTION (lightly edited for improved readability) Mark Donnigan: 00:49 Well, Hi, Dror! Dror Gill: 00:50 Is this really episode three? Mark Donnigan: 00:52 It is, it is episode three. So, today we have a really exciting discussion as we consider the future of codecs named with three characters. Dror Gill: 01:03 Three character codecs, okay, let's see. Mark Donnigan: 01:06 Three character codecs. Dror Gill: 01:09 I can think of … Mark Donnigan: 01:09 How many can you name? Dror Gill: 01:10 Let's see, that's today's trivia question. I can think of AVC, VP9, AV1, and VVC? Mark Donnigan: 01:21 Well, you just named three that I was thinking about and we're gonna discuss today! We've already covered AV1. Yeah, yeah, you answered correctly, but we haven't really considered where AVC, VP9, and VVC fit into the codec stew. So when I think about AVC, I'm almost tempted to just skip it because isn't this codec standard old news? I mean, c'mon. The entire video infrastructure of the internet is enabled by AVC, so what is there to discuss? Dror Gill: 01:57 Yeah. You're right. It's like the default, but in fact, the interesting thing is that today, we're (in) 2018 and this is the twenty year anniversary of AVC. I mean, ITU issued the call for proposals, their video coding expert group, issued the call for proposal for a project. At the time was called H26L, and their target was to double the coding efficiency, which effectively means halving the bit rate necessary for given level of fidelity. And that's why it was called H26L, it was supposed to be low bit rate. Mark Donnigan: 02:33 Ah! That's an interesting trivia question. Dror Gill: 02:35 That's where the L came from! Mark Donnigan: 02:36 I wonder how many of our listeners knew that? That's kind of cool. H26L. Dror Gill: 02:42 But they didn't go alone. It was the first time they joined forces in 2001 with the ISO MPEG, that's the same Motion Pictures Experts Group, you know we discussed in the first episode. Mark Donnigan: 02:56 That's right. Dror Gill: 02:57 And they came together, they joined forced, and they created JVT, that was the Joint Video Team, and I think it's a great example of collaboration. There are standards by dealing with video communication standards, and ISO MPEG, which is a standards body dealing with video entertainment standards. So, finally they understood that there's no point in developing video standards for these two different types of applications, so they got all the experts together in the JVT and this group developed what was the best video compression standard at the time. It was launched May 30, 2003. Mark Donnigan: 03:35 Wow. Dror Gill: 03:36 There was one drawback with this collaboration in that the video standard was known by two names. There was the ITU name which is H.264. And then there's the ISO MPEG name which is AVC, so these created some confusion at the start. I think by now, most of our listeners know that H.264 and AVC are two of the same. Mark Donnigan: 03:57 Yeah, definitely. So, AVC was developed 15 years ago and it's still around today. Dror Gill: 04:02 Yeah, yeah. I mean, that's really impressive and it's not only around, it's the most popular video compression standard in the world today. I mean, AVC is used to deliver video over the internet, to computers, televisions, mobile devices, cable, satellite, broadcast, and even blu-ray disks. This just shows you how long it takes from standardization to adoption, right? 15 years until we get this mass market adoption market dominance of H.264, AVC as we have today. Dror Gill: 04:31 And the reason it takes so long is that, we discussed it in our first episode, first you need to develop the standard. Then, you need to develop the chips that support the standard, then you need to develop devices that incorporate the chip. Even when initial implementation of the codec got released, they are still not as efficient as they can be, and it takes codec developers more time to refine it and improve the performance and the quality. You need to develop the tools, all of that takes time. Mark Donnigan: 04:59 It does. Yeah, I have a background in consumer electronics and because of that I know for certainty that AVC is gonna be with us for a while and I'll explain why. It's really simple. Decoding of H.264 is fully supported in every chip set on the market. I mean literally every chip set. There is not a device that supports video which does not also support AVC today. It just doesn't exist, you can't find it anywhere. Mark Donnigan: 05:26 And then when you look at in coding technologies for AVC, H.264, (they) have advanced to the point where you can really achieve state of the art for very low cost. There's just too much market momentum where the encode and decode ecosystems are just massive. When you think about entertainment applications and consumer electronics, for a lot of us, that's our primary market (that) we play in. Mark Donnigan: 05:51 But, if you consider the surveillance and the industrial markets, which are absolutely massive, and all of these security cameras you see literally everywhere. Drone cameras, they all have AVC encoders in them. Bottom line, AVC isn't going anywhere fast. Dror Gill: 06:09 You're right, I totally agree with that. It's dominant, but it's still here to stay. The problem is that, we talked about this, video delivery over the internet. The big problem is the bandwidth bottleneck. With so much video being delivered over the internet, and then the demand for quality is growing. People want higher resolution, they want HDR which is high dynamic range, they want higher frame rate. And all this means you need more and more bit rate to represent the video. The bit rate efficiency that is required today is beyond the standard in coding in AVC and that's where you need external technologies such as content adaptive encoding perceptual optimization that will really help you push AVC to its limits. Mark Donnigan: 06:54 Yeah. And Dror, I know you're one of the inventors of a perceptual optimization technique based on a really unique quality measure, which I've heard some in the industry believe could even extend the life of AVC from a bit rate efficiency perspective. Tell us about what you developed and what you worked on. Dror Gill: 07:13 Yeah, that's right. I did have some part in this. We developed a quality measure and a whole application around it, and this is a solution that can reduce the bit rate of AVC by 30%, sometimes even 40%. It doesn't get us exactly to where HEVC starts, 50% is pretty difficult and not for every content (type). But content distributors that recognize AVC will still be part of their codec mix for at least five years, I think what we've been able to do can really be helpful and a welcome relief to this bandwidth bottleneck issue. Mark Donnigan: 07:52 It sounds like we're in agreement that for at least the midterm horizon, the medium horizon, AVC is gonna stay with us. Dror Gill: 08:01 Yeah, yeah. I definitely think so. For some applications and services and certain regions of the world where the device penetration of the latest, high end models is not as high as in other parts, AVC will be the primary codec for some time to come. Dror Gill: 08:21 Okay, that's AVC. Now, let's talk about VP9. Mark Donnigan: 08:24 Yes, let's do that. Dror Gill: 08:25 It's interesting to me, essentially, it's mostly a YouTube codec. It's not a bad coded, it has some efficiency advantages over AVC, but outside of Google, you don't see any large scale deployments. By the way, if you look at Wikipedia, you read about the section that says where is VP9 used, it says VP9 is used mostly by YouTube, some uses by Netflix, and it's being used by Wikipedia. Mark Donnigan: 08:50 VP9 is supported fairly well in devices. Though, it's obviously hard to say exactly what the penetration is, I think there is support in hardware for decode for VP9. Certainly it's ubiquitous on Android, and it's in many of the UHD TV chip sets as well. So, it's not always enabled, but again, from my background on the hardware side, I know that many of those SOCs, they do have a VP9 decoder built into them. Mark Donnigan: 09:23 I guess the question in my mind is, it's talked about. Certainly Google is a notable both developer and user, but why hasn't it been adopted? Dror Gill: 09:33 Well, I think there are several issues here. One of them is compression efficiency. VP9 brings maybe 20, 30% improvement in compression efficiency over AVC, but it's not 50%. So, you're not doubling your compression efficiency. If you want to replace the codec, that's really a big deal. That's really a huge investment. You need to invest in coding infrastructure, new players. You need to do compatibility testing. You need to make sure that your packaging and your DRM work correctly and all of that. Dror Gill: 10:04 You really want to get a huge benefit to offset this investment. I think people are really looking for that 50% improvement, to double the efficiency, which is what you get with HEVC but not quite with VP9. I think the second point is that VP9, even though it's an open source coder, it's developed and the standard is maintained by Google. And some industry players are kind of afraid of the dominance of Google. Google has taken over the advertising market online. Mark Donnigan: 10:32 Yes, that's a good point. Dror Gill: 10:34 You know, and search and mobile operating systems, except Apple, it's all Android. So, those industry players might be thinking, I don't want to depend on Google for my video compression format. I think this is especially true for traditional broadcasters. Cable companies, satellite companies, TV channels that broadcast over the air. These companies traditionally like to go with established, international standards. Compression technologies that are standardized, they have the seal of approval by ITU and ISO. Dror Gill: 11:05 They're typically following that traditional codec developer past. ISO MPEG too, now it's AVC, starting with HEVC. What's coming next? Mark Donnigan: 11:16 Well, our next three letter codec is VVC. Tell us about VVC, Dror. Dror Gill: 11:21 Yeah, yeah, VVC. I think this is another great example of collaboration between ITU and ISO. Again, they formed a joint video experts team. This time it's called JVET. Dror Gill: 12:10 So, JVET has launched a project to develop a new video coding standard. And you know, we had AVC that was advanced video coding. Then we had HEVC which is high efficiency video coding. So, they thought, what would be the next generation? It's already advanced, it's high efficiency. So, the next one, they called it VVC, which is versatile video code. The objective of VVC is obviously to provide a significant improvement in compression efficiency over the existing HEVC standard. Development already started. The JVET group is meeting every few in months in some exotic place in the world and this process will continue. They plan to complete it before the end of 2020. So, essentially in the next two years they are gonna complete the standard. Dror Gill: 13:01 Today, already, even though VVC is in early development and they haven't implemented all the tools, they already report a 30% better compression efficiency than HEVC. So, we have high hopes that we'll be able to fight the video tsunami that is coming upon us with a much improved standard video coder which is VVC. I mean, its improved at least on the technical side and I understand that they also want to improve the process, right? Mark Donnigan: 13:29 That's right, that's right. Well, technical capabilities are certainly important and we're tracking of course VVC. 30% better efficiency this early in the game is promising. I wonder if the JVET will bring any learnings from the famous HEVC royalty debacles to VVC because I think what's in everybody's mind is, okay, great, this can be much more efficient, technically better. But if we have to go round and round on royalties again, it's just gonna kill it. So, what do you think? Dror Gill: 14:02 Yeah, that's right. I think it's absolutely true and many people in the industry have realized this, that you can't just develop a video standard and then handle the patent and royalty issues later. Luckily some companies have come together and they formed an industry group called The Media Coding Industry Forum, or MC-IF. They held their first meeting a few weeks ago in Macau during empic meeting one through four. Their purpose statement, let me quote this from their website, and I'll give you my interpretation of it. They say the media coding industry forum (MC-IF) is an open industry forum with a purpose of furthering the adoption of standards initially focusing on VVC, but establishing them as well accepted and widely used standards for the benefit of consumers and the industry. Dror Gill: 14:47 My interpretation is that the group was formed in an effort for companies with interest in this next generation video codec to come together and attempt to influence the licensing policy of VVC and try to agree on a reasonable patent licensing policy in advance to prevent history from repeating itself. We don't want that whole Hollywood story with the tragedy that took a few years until they reached the happy ending. So, what are you even talking about? This is very interesting. They're talking about having a modular structure for the codec. These tools of the codecs, the features, can be plugged in and out, very easily. Dror Gill: 15:23 So, if some company insists on reasonable licensing terms, this group can just decide not to support the feature and it will be very easily removed from the standard, or at least from the way that companies implement that standard. Mark Donnigan: 15:37 That's an interesting approach. I wonder how technically feasible it is. I think we'll get into that in some other episodes. Dror Gill: 15:46 Yeah. That may have some effect on performance. Mark Donnigan: 15:49 Exactly. And again, are we back in the situation that the Alliance for Open Media is in with AV1. Where part of the issue of the slow performance is trying to work around patents. At the end of the day you end up with a solution that is hobbled technically. Dror Gill: 16:10 Yeah. I hope it doesn't go there. Mark Donnigan: 16:13 Yeah, I hope we're not there. I think you heard this too, hasn't Apple joined the consortium recently? Dror Gill: 16:21 Yeah, yeah, they did. They joined silently as they always do. Silently means that one day somebody discovers their logo… They don't make any announcement or anything. You just see a logo on the website, and then oh, okay. Mark Donnigan: 16:34 Apple is in the building. Mark Donnigan: 16:41 You know, maybe it's good to kind of bring this discussion back to Earth and close out our three part series by giving the listeners some pointers. About how they should be thinking about the next codec that they adopt. I've been giving some thought as we've been doing these episodes. I think I'll kick it off here Dror if you don't mind, I'll share some of my thoughts. You can jump in. Mark Donnigan: 17:11 These are complex decisions of course. I completely agree, billing this as codec wars and codec battles, it's not helpful at the end of the day. Maybe it makes for a catchy headline, but it's not helpful. There's real business decisions (to be made). There are technical decisions. I think a good place to start for somebody who's listening and saying “okay great, I now have a better understanding of the lay of the land of HEVC, for AV1, I can understand VP9, I can understand AVC and what some of my options are to even further reduce bit rate. But now, what do I do?” Mark Donnigan: 17:54 And I think a good place to start is to just look at your customers, and do they lean towards early adopters. Are you in a strong economic environment, which is to say quite frankly, do most of your customers carry around the latest devices? Like an iPhone X, or Galaxy 9. If largely your customers lean towards early adopter and they're carrying around the latest devices, then you have an obligation to serve them with the highest quality and the best performance possible. Dror Gill: 18:26 Right. If your customers can receive HEVC, and it's half the bit rate, then why not deliver it to them better quality, or say when you see the end cost with this more efficient codec and everybody is happy. Mark Donnigan: 18:37 Absolutely, and again, I think just using pure logic. If somebody could afford a more than $1000 device in their pocket, probably the TV hanging on the wall is a very new, UHD capable (one). They probably have a game console in the house. The point is that you can make a pretty strong argument and an assumption that you can go, what I like to think of as all in HEVC including even standard definition, just SDR content. Mark Donnigan: 19:11 So, the industry has really lost sight in my mind of the benefits of HEVC as they apply across the board to all resolutions. All of the major consumer streaming services are delivering 4K using HEVC, but I'm still shocked at how many, it's kind of like oh, we forget that the same advantages of bit rate efficiency that work at 4K apply at 480p. Obviously, the absolute numbers are smaller because the file sizes are smaller, etc. Mark Donnigan: 19:41 But the point is, 30, 40, 50% savings applies at 4K as it does at 480p. I understand there's different applications in use cases, right? But would you agree with that? Dror Gill: 19:55 Yeah, yeah, I surely agree with that. I mean, for 4K, HEVC is really an enabler. Mark Donnigan: 20:00 That's right. Dror Gill: 20:01 For HEVC, you would need like 30, 40 megabits of video. Nobody can stream that to the home, but change it to 10, 15, that's reasonable, and you must use HEVC for 4k otherwise it won't even fit the pipe. But for all other resolutions, you get the bang with the advantage or you can trade it off for a quality advantage and deliver higher quality to your users, or higher frame rate, or enable HDR. If all of these possibilities that you can do with HD and even SD content, give them a better experience using HEVC and being able to stream on devices that your users already have. So yeah, I agree. I think it's an excellent analysis. Obviously if you're up in an emerging market, or your consumers don't have high end devices, then AVC is a good solution. If there are network constraints, and there are many places in the world that network conductivity isn't that great. Or in rural areas where we have very large parts of the population spread out (in these cases) bandwidth is low and you will get into a bottleneck even with HD. Mark Donnigan: 21:05 That's right. Dror Gill: 21:06 That's where perceptual optimization can help you reduce the bit rate even for AVC and keep within the constraints that you have. When your consumers can upgrade their devices and when the cycle comes in a few years when every device has HEVC support, then obviously you upgrade your capability and support HEVC across the board. Mark Donnigan: 21:30 Yeah, that's a very important point Dror, is that this HEVC adoption curve in terms of silicon, on devices. It is in full motion. Just the planning life cycles. If you look at what goes into hardware, and especially on the silicon side, it doesn't happen that way. Once these technologies are in the designs, once they are in the dies, once the codec is in silicon, it doesn't get arbitrarily turned on and off like light switches. Mark Donnigan: 22:04 How should somebody be looking at VP9, VVC, and AV1? Dror Gill: 22:13 Well, VP9 is an easy one. Unless you're Google, you're very likely gonna skip over this codec. Not just that the VP9 isn't the viable choice, it simply doesn't go so far as HEVC in terms of bit rate efficiency and quality. Maybe two years back we would consider it as an option for reducing bit rate, but now with the HEVC support that you have, there's no point in going to VP9. You might as well go to HEVC. If you talk about VVC, (the) standard is still a few years from being ratified so, we actually don't have anything to talk about. Dror Gill: 22:49 The important point is again to remember, even when VVC launches, it will still be another 2 to 3 years after ratifying the standard before you have even a very basic playback ecosystem in place. So, I would tell our listeners if you're thinking, why should I adopt HEVC, because VVC is just around the corner, well, that corner is very far. It's more like the corner of the Earth than the corner of the next block. Mark Donnigan: 23:15 That's right. Dror Gill: 23:18 So, HEVC today, VVC will be the next step in a few years. And then there's AV1. You know, we talked a lot about AV1. No doubt, AV1 has support from huge companies. I mean Google, Facebook, Intel, Netflix, Microsoft. And those engineers, they know what they're doing. But now, it's quite clear that compression efficiency is the same as HEVC. Meanwhile, after removing other royalty cost for content delivery, HEVC Advance removed it. The license situation is much more clear now. You add to this the fact that at the end of the day, two to three years, you're gonna need five to ten times more compute power to encode AV1, reaching effectively the same result. Now Google, again. Google may be that they have unlimited compute resources, they will use it. They developed it. Dror Gill: 24:13 The smaller content providers, all the other ones, the non Googles of the world and other broadcasters with growing support for HEVC that we expect in a few years. I think it's obvious. They're gonna support HEVC and then a few years later when VVC is ratified, when it's supported in devices, they're gonna move to VVC. Because this codec does have the required compression efficiency improvement over HEVC. Mark Donnigan: 24:39 Yeah, that's an excellent summary Dror. Thank you for breaking this all down for our listeners so succinctly. I'm sure this is really gonna provide massive value. I want to thank our amazing audience because without you, the Video Insiders Podcast would just be Dror and me taking up bits on a server somewhere. Dror Gill: 24:59 Yeah, talking to ourselves. Mark Donnigan: 25:01 As you can tell, video is really exciting to us and so we're so happy that you've joined us to listen. And again, this has been a production of Beamr Imaging Limited. Please, subscribe on iTunes and if you would like to try out beamer codecs in your lab or your production environment, we are giving away up to $100 of HEVC and H264 in coding every month. That's each and every month. Just go to https://beamer.com/free and get started immediately.

Is AV1 more efficient than HEVC? Dror & Mark get into the middle of a 3 against 1 standoff over whether AV1 is actually more efficient than HEVC. The following blog post first appeared on the Beamr blog at: https://blog.beamr.com/2018/11/23/codec-efficiency-is-in-the-eye-of-the-measurer-podcast/ When it comes to comparing video codecs, it's easy to get caught up in the “codec war” mentality. If analyzing and purchasing codecs was as easy as comparing fuel economy in cars, it would undoubtedly take a lot of friction out of codec comparison, but the reality is that it's not that simple. In Episode 02, The Video Insiders go head-to-head comparing two of the leading codecs in a three against one standoff over whether AV1 is more efficient than HEVC. So, which is more efficient? Listen in to this week's episode, “Codec Efficiency Is in the Eye of the Measurer,” to find out. Want to join the conversation? Reach out to TheVideoInsiders@beamr.com. TRANSCRIPTION (lightly edited to improve readability only) Mark Donnigan: 00:41 Hi everyone I am Mark Donnigan and I want to welcome you to episode two of the Video Insiders. Dror Gill: 00:48 And I am Dror Gill. Hi there. Mark Donnigan: 00:50 In every episode of the Video Insiders we bring the latest inside information about what's happening in the video technology industry from encoding, to packaging, to delivery, and playback, and even the business behind the video business. Every aspect of the video industry is covered in detail on the Video Insiders podcast. Dror Gill: 01:11 Oh yeah, we usually do cover everything from pixels, to blocks, to microblocks, to frames, to sequences. We go all the way up and down the video delivery chain and highlight the most important things you should know before you send any video bits over the wire. Mark Donnigan: 01:28 In our first episode we talked about a very hot topic which asked, “Hasn't this kind of been worn out?” The whole HEVC, AV1 discussion. But I think it was very interesting. I sure enjoyed the talk. What about you Dror? Dror Gill: 01:47 Yeah, yeah, yeah. I sure did. It was great talking about the two leading codecs. I don't want to say the word, codec war. Mark Donnigan: 01:58 No, no, we don't believe in codec wars. Dror Gill: 01:59 We believe in codec peace. Mark Donnigan: 02:00 Yeah, that's true. Why is it so complicated to compare video codecs? Why can't it be as simple as fuel economy of cars, this one gets 20 miles per gallon and that one gets 30 and then I make a decision based on that. Dror Gill: 02:15 I wish it was that simple with video codecs. In video compression you have so many parameters to consider. You have the encoding tools, tools are grouped into what's called profiles and levels, or as AV1 calls them “experiments.” Mark Donnigan: 02:31 Experiments, mm-hmm… Dror Gill: 02:35 When you compare the codecs which profiles and levels do you use. What rate control method? Which specific parameters do you set for each codec? And each codec can have hundreds, and hundreds of parameters. Then there is the question of implementation. Which software implementation of the codec do you use? Some implementations are reference implementations that are used for research, and others are highly performance optimized commercial implementations. Which one do you select for the test? And then, which operating system, what hardware do you run on, and obviously what test content? Because encoding two people talking, or encoding an action scene for a movie, is completely different. Dror Gill: 03:13 Finally, when you come to evaluate your video, what quality measure do you use? There're various objective quality measures and some people use actual human viewers and they assesses subjective quality of the video. On that front also, there're many possibilities that you need to choose from. Mark Donnigan: 03:32 Yeah, so many questions and no wonder the answers are not so clear. I was quite surprised when I recently read three different technical articles published at IBC actually, effectively comparing AV1 versus HEVC and I can assume that each of the authors did their research independently. What was surprising was they came to the exact same conclusion, AV1 has the same compression efficiency as HEVC. This is surprising because some other studies and one in particular (I think we'll talk about) out there says the contrary. So can you explain what this means exactly, Dror. Dror Gill: 04:16 By saying that they have the same compression efficiency, this means that they can reach the same quality at the same bitrate or the other way round. You need the same bitrate to reach that same quality. If you need for example, two and a half megabits per second to encode an HD video file using HEVC at a certain quality, then with AV1 you would need roughly the same bitrate to reach that same quality and this means that AV1 and HEVC provide the same compression level. In other words, this means that AV1 does not have any technical advantage over HEVC because it has the same compression efficiency. Of course that's if we put aside all the loyalty issues but we discussed that last time. Right? Mark Donnigan: 04:56 That's right. The guys who wrote the three papers that I'm referencing are really top experts in the field. It's not seminar work done by a student, not to downplay those papers, but the point is these are professionals. One was written by the BBC in cooperation with the Multimedia and Vision Group at the Queen Mary University of London. I think nobody is going to say that the BBC doesn't know a thing or two about video. The second was written by Ateme, and the third by Harmonic, leading vendors. Mark Donnigan: 05:29 I actually pulled out a couple of phrases from each that I'd like to quote. First the BBC and Queen Mary University, here is a conclusion that they wrote, “The results obtained show in general a similar performance between AV1 and the reference HEVC both objectively and subjectively.” Which is interesting because they did take the time to both do the visual assessment as well as use a quality measure. Mark Donnigan: 06:01 Ateme said, “Results demonstrate AV1 to have equivalent performance to HEVC in terms of both objective and subjective video quality test results.” Dror Gill: 06:10 Yeah, very similar. Mark Donnigan: 06:16 And then here is what Harmonic said, “The findings are that AV1 is not more advantageous today than HEVC on the compression side and much more complex to encode than HEVC.” What do you make of this? Dror Gill: 06:32 I don't know. It sounds pretty bad to me, even two of those papers also analyzed subjective quality so they used actual human viewers to check out the quality. But Mark what if I told you that researchers from the University of Klagenfurt in Austria together with Bitmovin published a paper which showed completely different results. What would you say about that? Mark Donnigan: 06:57 Tell me more. Dror Gill: 06:58 Last month in Athens I was the ICIP conference that's the IEEE International Conference on Image Compression and Image Processing. There was this paper presented by this University in Austria with Bitmovin and their conclusion was, let me quote, “When using weighted PSNR, AV1 performs consistently better for bit rate compared to AVC, HEVC, and VP9.” So they claim AV1 is better than three codecs but specifically it's better than HEVC. And then they have a table in their article that compares AV1 to HEVC for six different video clips. The table shows that with AV1 you get up to 25% lower bitrate at the same quality than HEVC. Dror Gill: 07:43 I was sitting there in Athens last month when they presented this and I was shocked. Mark Donnigan: 07:50 What are the chances that three independent papers are wrong and only this paper got it right? And by the way, the point here is not three against one because presumably there're some other papers. I'm guessing other research floating around that might side with Bitmovin. The point is that three companies who no one is going to say that any of them are not experts and not highly qualified to do a video assessment, came up with such a different result. Tell us what you think is going on here? Dror Gill: 08:28 I was thinking the same thing. How can that be. During the presentation I asked one of the authors who presented the paper a few questions and it turned out that they made some very questionable decisions in all of that sea of possibility that I talked about before. Decisions related to coding tools, codec parameters, and quality measures. Dror Gill: 08:51 First of all, in this paper they didn't show any results of subjective viewing. Only the objective metrics. Now we all know that you should always your eyes, right? Mark Donnigan: 09:03 That's right. Dror Gill: 09:04 Objective metrics, nice numbers, but obviously you need to view the video because that's how the actual viewers are going to assess the (video) quality. The second thing is that they only used the single objective metric and this was PSNR. PSNR, it stands for peak signal-to-noise ratio and basically this measure is a weighted average of the difference in peaks between pixel values of the two images. Dror Gill: 09:30 Now, we're Video Insiders, but even if you're not an insider you know that PSNR is not a very good quality measure because it does not correlate very well with human vision. This is the measure that they choose to look at but what was most surprising is that there is a flag in the HEVC open source encoder which they used that if chosen, the result is improved PNSR. What it does, it turns off some psycho-visual optimizations which make the video look better but reduce the PSNR, and that's turned on by default. So you would expect that they're measuring PSNR they would turn that flag on so you would get higher PSNR. Well, they didn't. They didn't turn the flag on! Mark Donnigan: 10:13 Amazing. Dror Gill: 10:17 Finally, even then AV1 is much slower than HEVC, and they also reported in this data that it was much, much slower than HEVC but still they did not use the slowest encoding standing of HEVC, which would provide the best quality. There's always a trade off between performance and quality. The more tools you employ the better quality you can squeeze out of the video, of course that takes you more CPU cycles but they used for HEVC, the third slowest setting which means this is the third best quality you can get with that codec and not the very best quality. When you handicap an HEVC encoder in this way, it's not surprising that you get such poor results. Dror Gill: 11:02 I think based on all these points everybody can understand why the results of this comparison were quite different than all of the other comparison that were published a month earlier at IBC (by Ateme, BBC, Harmonic). Mark Donnigan: 11:13 It's interesting. Mark Donnigan: 11:14 Another critical topic that we have to cover is performance. If you measure the CPU performance on encoding time of AV1, I believe that it's pretty universally understood that you are going to find it currently is a hundred times slower than HEVC. Is that correct? Dror Gill: 11:32 Yeah, that's right. Typically, you measure the performance of an encoder and FPS which is frames per second. For HEVC it's common to measure an FPM which is frames per minute. Mark Donnigan: 11:42 Frames per minute, (more like) frames per hour, FPH. Dror Gill: 11:45 A year and a half ago or a year ago when there were very initial implementation, it was really FPD or FPH, Frames per hour or per day and you really needed to have a lot of patience, but now after they've done some work it's only a hundred times slower than HEVC. Mark Donnigan: 12:02 Yeah, that's pretty good. They're getting there. But some people say that the open source implementation of AV1 I believe it's AOM ENC. Dror Gill: 12:11 Yeah, AOM ENC. Mark Donnigan: 12:16 ENC exactly has not been optimized for performance at all. One thing I like about speed is either your encoder produces X number of frames per second or per minute, or it doesn't. It's really simple. Here is my next question for you. Proponents of AV1 are saying, “well it's true it's slow but it hasn't been optimized, the open source implementation,” which is to imply that there's a lot of room (for improvement) and that we're just getting started, “don't worry we'll close the gap.” But if you look at the code, and by the way I may be a marketing guy but my formal education is computer science. Mark Donnigan: 13:03 You can see it already includes performance optimizations. I mean eptimizations like MMX, SSE, there's AVX instructions, there's CPU optimization, there's multithreading. It seems like they're already trying to make this thing go faster. So how are they going to close this a hundred X (time) gap? Dror Gill: 13:22 I don't think they can. I mean a hundred X, that's a lot and you know even the AV1 guys they even admit that they won't be able to close the gap. I talked to a few senior people who're involved in the Alliance for Open Media and even they told me that they expect AV1 to five to 10 times more complex than HEVC at the end of the road. In two to three years after all optimization are done, it's still going to be more complex than HEVC. Dror Gill: 13:55 Now, if you ask me why it's so complex I'll tell you my opinion. Okay, this is my personal opinion. I think it's because they invested a lot of effort in side stepping the patents (HEVC). Mark Donnigan: 14:07 Good point. I agree. Dror Gill: 14:07 They need to get that compression efficiency which is the same as HEVC but they need to use algorithms that are not patented. They have methods that use much more CPU resources than the original patent algorithms to reach the same results. You can call it kind of brute force implementation of the same thing to avoid the patent issue. That's my personal opinion, but the end result I think is clear, it's going to be five to 10 times slower than HEVC. It has the same compression efficiency so I think it's quite questionable. This whole notion of using AV1 to get better results. Mark Donnigan: 14:45 Absolutely. If you can encode let's say on a single computer with HEVC a full ABR stack, this is what people want to do. But here we're talking speeds that are so slow let's just try and do (encode) one stream. Literally what you're saying is you'll need five to 10 computers to do the same encode with AV1. I mean, that's just not viable. It doesn't make sense to me. Dror Gill: 15:14 Yeah, why would you invest so much encoding into getting the same results. If you look at another aspect of this, let's talk about hardware encode. Companies that have large data centers, companies that are encoding vast amount of video content are not looking into moving from the traditional software encoding and CPUs and GPUs, to dedicated hardware. We're hearing talks about FPGAs even ASICs … by the way this is a very interesting trend in itself that we'll probably cover in one of the next episodes. But in the context of AV1, imagine a chip that is five to 10 times larger than an HEVC chip and which is the same complexity efficiency. The question I ask again is why? Why would anybody design such a chip, and why would anybody use it when HEVC is available today? It's much easier to encode, royalty issues have been practically solved so you know? Mark Donnigan: 16:06 Yeah, it's a big mystery for sure. One thing I can say is the Alliance for Open Media has done a great service to HEVC by pushing the patent holders to finalize their licensing terms … and ultimately make them much more rational shall we say? Dror Gill: 16:23 Yeah. Mark Donnigan: 16:25 Let me say that as we're an HEVC vendor and speaking on behalf of others (in the industry), we're forever thankful to the Alliance for Open Media. Dror Gill: 16:36 Definitely, without the push from AOM and the development of AV1 we would be stuck with HEVC royalty issue until this day. Mark Donnigan: 16:44 That was not a pretty situation a few years back, wow! Dror Gill: 16:48 No, no, but as we said in the last episode we have a “happy ending” now. (reference to episode 1) Mark Donnigan: 16:52 That's right. Dror Gill: 16:52 Billions of devices support HEVC and royalty issues are pretty much solved, so that's great. I think we've covered HEVC and AV1 pretty thoroughly in two episodes but what about the other codecs? There's VP9, you could call that the predecessor of AV1, and then there's VVC, which is the successor of HEVC. It's the next codec developed by MPEG. Okay, VP9 and VVC I guess we have a topic for our next episode, right? Mark Donnigan: 17:21 It's going to be awesome. Narrator: 17:23 Thank you for listening to the Video Insider podcast a production of Beamr limited. To begin using Beamr codecs today go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.

Join the conversation by jumping into The Video Insiders LinkedIn Group. For more podcast episodes, visit our Show Guide. For more information on Beamr, visit us online. Read the accompanying blog post here. If you would like to be a guest on the show, send an email to thevideoinsiders@beamr.com