Podcast appearances and mentions of doug hubbard

I keep hearing data product, data strategy, and UX teams often struggle to quantify the value of their work. Whether it's as a team as a whole or on a specific data product initiative, the underlying problem is the same – your contribution is indirect, so it's harder to measure. Even worse, your stakeholders want to know if your work is creating an impact and value, but because you can't easily put numbers on it, valuation spirals into a messy problem.   The messy part of this valuation problem is what today's episode is all about—not math! Value is largely subjective, not objective, and I think this is partly why analytical teams may struggle with this. To improve at how you estimate the value of your data products, you need to leverage other skills—and stop approaching this as a math problem.   As a consulting product designer, estimating value when it's indirect is something that I've dealt with my entire career. It's not a skill learned overnight, and it's one you will need to keep developing over time—but the basic concepts are simple. I hope you'll find some value in applying these along with your other frameworks and tools.    Highlights/ Skip to   Value is subjective, not objective (5:01) Measurability does not necessarily mean valuable (6:36) Businesses are made up of humans. Most b2b stakeholders aren't spending their own money when making business decisions—what does that mean for your work? (9:30) Quantifying a data product's value starts with understanding what is worth measuring in the eye of the beholder(s)—not math calculations (13:44) The more difficult it is to show the value of your product (or team) in numbers, the lower that value is to the stakeholder—initially (16:46) By simply helping a stakeholder to think through how value should be calculated on a data product, you're likely already providing additional value (18:02) Focus on expressing estimated value via a range versus a single number (19:36) Measurement of anything requires that we can observe the phenomenon first—but many stakeholders won't be able to cite these phenomena without [your!] help (22:16) When you are measuring quantitative aspects of value, remember that measurement is not the same as accuracy (precision)—and the precision game can become a trap (25:37) How to measure anything—and why estimates often trump accuracy (31:19) Why you may need to steer the conversation away from ROI calculations in the short term (35:00)   Quotes from Today's Episode Even when you can easily assign a dollar value to the data product you're building, that does not necessarily reflect what your stakeholder actually feels about it—or your team's contribution. So, why do they keep asking you to quantify the value of your work? By actually understanding what a shareholder needs to observe for them to know progress has been made on their initiative or data product, you will be positioned to deliver results they actually care about. While most of the time, you should be able to show some obvious economic value in the work you're doing, you may be getting hounded about this because you're not meeting the often unstated qualitative goals. If you can surface the qualitative goals of your stakeholder, then the perception of the value of your team and its work goes up, and you'll spend less time trying to measure an indirect contribution in quant terms that only has a subjectively right answer. (6:50) The more difficult it is for you to show the monetary value of your data product (or team), the lower that value likely is to the stakeholder. This does not mean the value of your work is “low.” It means it's perceived as low because it cannot be easily quantified in a way that is observable to the person whose judgment matters. By understanding the personal motivations and interests of your stakeholders, you can begin to collaboratively figure out what the correct success metrics should be—and how they'd be measured. By just simply beginning to ask and uncover what they're trying to measure, you can start to increase your contributions' perceived value. (17:01) Think about expressing “indirect value” as a range, not a precise single value. It's much easier to refine your estimate (if necessary) once a range has been defined, and you only need to get precise enough for your stakeholder to make a decision with the information. How much time should you spend refining your measurement of the value? Potentially little to none—if the “better math” isn't going to change anyone's mind or decision.  Spending more time to measure a data product's value more accurately takes you away from doing actual product work—and if there isn't much obvious value to the work, maybe the work—not the measurement of the work—needs to change. (19:49) Smart leaders know that deriving a simple calculation of indirect contributions is complex—otherwise, the topic wouldn't keep coming up. There is a “why” behind why they're asking, and when you understand the “why,” you'll be better positioned to deliver the value they actually seek, using valuation measurements that are “just enough” in their precision. What do you think it says to a stakeholder if you're spending an inordinate amount of time simply trying to calculate and explain the value of your data product? (23:22) Many organizations for years have invested in things that don't always have a short term ROI.  They know that ROI takes time, and they can't really measure what it's worth along the way. Examples include investments in company culture, innovation, brand reputation, and many others. If you're constantly playing defense and having to justify your existence or methods by quantifying the financial value of your data products (or data product management team, or UX team, or any other indirect contributor/contribution), then either your work truly does lack value, or you haven't surfaced what the actual success metrics and outcomes are— in the eyes of the stakeholder. As such, the perceived value is “low” or opaque. They might be looking for a hard number to assign to it because they're not seeing any of the other forms of value that they care about that would indicate positive progress. It's easier to write [you] a large check  for a big, innovative, unproven initiative if your stakeholders know what you and your team can accomplish with a small check. (35:16)   Links Experiencing Data: Episode 80 with Doug Hubbard

Check out this episode from the BSW Vault, hand picked by main host Matt Alderman! This segment was originally published on Jan 24, 2023. Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview. Show Notes: https://securityweekly.com/vault-bsw-14

Check out this episode from the BSW Vault, hand picked by main host Matt Alderman! This segment was originally published on Jan 24, 2023. Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview. Show Notes: https://securityweekly.com/vault-bsw-14

Check out this episode from the BSW Vault, hand picked by main host Matt Alderman! This segment was originally published on Jan 24, 2023. Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview. Show Notes: https://securityweekly.com/vault-bsw-14

Check out this episode from the BSW Vault, hand picked by main host Matt Alderman! This segment was originally published on Jan 24, 2023. Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview. Show Notes: https://securityweekly.com/vault-bsw-14

Read the full transcript here. Can we measure everything that matters to us? When is measuring the correlates of a thing pretty much just as good as measuring the thing itself? Why are some people resistant to measuring certain things? What are some things people should be measuring but aren't? What's the connection between measuring things and assigning probabilities to events? How much do we know about how well human intuition performs against "doing the math"? How inconsistent are we at applying our own principles in decision-making? What kinds of calibration training are effective? What is "value of information"? What is the Rule of 5? What are the top three things we can do to improve our decision-making?Douglas Hubbard is an author, consultant, and recognized expert in decision theory and risk analysis. He has written several books on measurement and measuring risk. His work spans various industries including insurance, finance, pharmaceuticals, aerospace, military, energy, government, tech, and nonprofit organizations. Connect with Doug on LinkedIn, email him at dwhubbard@hubbardresearch.com, or learn more about his work at hubbardresearch.com.Further reading:How to Measure Anything: Finding the Value of Intangibles in Business, by Douglas HubbardApplied Information Economics Academy (Doug's online calibration training)Innumeracy: Mathematical Illiteracy and Its Consequences, by John Allen PaulosClearer Thinking's "Calibrate Your Judgement" toolFatebook StaffSpencer Greenberg — Host / DirectorJosh Castle — ProducerRyan Kessler — Audio EngineerUri Bram — FactotumWeAmplify — TranscriptionistsAlexandria D. — Research and Special Projects AssistantMusicBroke for FreeJosh WoodwardLee RosevereQuiet Music for Tiny Robotswowamusiczapsplat.comAffiliatesClearer ThinkingGuidedTrackMind EasePositlyUpLift[Read more]

A key role for the CISO and the team is to identify and plan for mitigation of the most damaging risks. Various approaches have been used over the years with varying levels of success. Are we measuring the right things? Are we using the right instruments? Join us as we discuss some of the flaws present in measuring risk today and considerations to improve our risk management approach. https://www.howtomeasureanything.com/cybersecurity Visit https://securityweekly.com/csp for all the latest episodes! Follow us on Twitter: https://www.twitter.com/cyberleaders Follow us on LinkedIn: https://www.linkedin.com/company/cybersecuritycollaborative/ Show Notes: https://securityweekly.com/csp-146

Today I'm joined by Vishal Singh, Head of Data Products at Starburst and co-author of the newly published e-book, Data Products for Dummies. Throughout our conversation, Vishal explains how the variations in definitions for a data product actually led to the creation of the e-book, and we discuss the differences between our two definitions. Vishal gives a detailed description of how he believes Data Product Managers should be conducting their discovery and gathering feedback from end users, and how his team evaluates whether their data products are truly successful and user-friendly.   Highlights/ Skip to: I introduce Vishal, the Head of Data Products at Starburst and contributor of the e-book Data Products for Dummies (00:37) Vishal describes how his customers at Starburst all had a common problem, but differing definitions of a data product, which led to the creation of his e-book (01:15) Vishal shares his one-sentence definition of a data product (02:50) How Vishal's definition of a data product differs from mine, and we both expand on the possibilities between the two (05:33) The tactics Vishal uses to useful feedback to ensure the data products he develops are valuable for end users (07:48) Why Vishal finds it difficult to get one on one feedback from users during the iteration phase of data product development (11:07) The danger of sunk cost bias in the iteration phase of data product development (13:10) Vishal describes how he views the role of a DPM when it comes to doing effective initial discovery (15:27) How Vishal structures his teams and their interactions with each other and their end users (21:34) Vishal's thoughts on how design affects both data scientists and end users (24:16) How DPMs at Starburst evaluate if the data product design is user-friendly (28:45) Vishal's views on where Designers are valuable in the data product development process (35:00) Vishal and I discuss the importance of ensuring your products truly solve your user's problems (44:44) Where you can learn more about Vishal's upcoming events and the e-book, Data Products for Dummies (49:48) Links Starburst: https://www.starburst.io/ Data Products for Dummies: https://www.starburst.io/info/data-products-for-dummies/ “How to Measure the Impact of Data Products with Doug Hubbard”: https://designingforanalytics.com/resources/episodes/080-how-to-measure-the-impact-of-data-productsand-anything-else-with-forecasting-and-measurement-expert-doug-hubbard/ Trino Summit: https://www.starburst.io/info/trinosummit2023/ Galaxy Platform: https://www.starburst.io/platform/starburst-galaxy/ Datanova Summit: https://www.starburst.io/datanova/ LinkedIn: https://www.linkedin.com/in/singhsvishal/ Twitter: https://twitter.com/vishal_singh

When it comes to measuring risk, the average person assumes they either need a ton of data or they have not enough data. The truth is you often need less data than you think to improve upon your current judgement, and it's probably been measured before. Unfortunately, recognizing that data and applying the math to it doesn't come naturally to the average person and we will probably need some help from technology to access that data. Host Jack Russo asks quantitative analyst Douglas Hubbard if the future of AI includes analyzing data and producing recommendations for things like business strategies, investment strategies & more.   How to Measure Anything How to Measure Anything in Cybersecurity Risk

 Can literally everything be measured? According to quantitative analyst Douglas Hubbard, yes it can – at least when it comes to risk management and decision-making sciences. Hubbard believes that anything that has any impact on your life has observable consequences and is sometimes judged based on flawed empirical data. Today host Jack Russo and Douglas Hubbard discuss measuring risk in areas where even the professionals once believed you could not.   How to Measure Anything How to Measure Anything in Cybersecurity Risk

Today I'm chatting with Osian Jones, Head of Product for the Data Platform at Stuart. Osian describes how impact and ROI can be difficult metrics to measure in a data platform, and how the team at Stuart has sought to answer this challenge. He also reveals how user experience is intrinsically linked to adoption and the technical problems that data platforms seek to solve. Throughout our conversation, Osian shares a holistic overview of what it was like to design a data platform from scratch, the lessons he's learned along the way, and the advice he'd give to other data product managers taking on similar projects.  Highlights/ Skip to: Osian describes his role at Stuart (01:36) Brian and Osian explore the importance of creating an intentional user experience strategy (04:29) Osian explains how having a clear mission enables him to create parameters to measure product success (11:44) How Stuart developed the KPIs for their data platform (17:09) Osian gives his take on the pros and cons of how data departments are handled in regards to company oversight (21:23) Brian and Osian discuss how vital it is to listen to your end users rather than relying on analytics alone to measure adoption (26:50) Osian reveals how he and his team went about designing their platform (31:33) What Osian learned from building out the platform and what he would change if he had to tackle a data product like this all over again (36:34) Quotes from Today's Episode “Analytics has been treated very much as a technical problem, and very much so on the data platform side, which is more on the infrastructure and the tooling to enable analytics to take place. And so, viewing that purely as a technical problem left us at odds in a way, compared to [teams that had] a product leader, where the user was the focus [and] the user experience was very much driving a lot of what was roadmap.” — Osian Jones (03:15) “Whenever we get this question of what's the impact? What's the value? How does it impact our company top line? How does it impact our company OKRs? This is when we start to panic sometimes, as data platform leaders because that's an answer that's really challenging for us, simply because we are mostly enablers for analytics teams who are themselves enablers. It's almost like there's two different degrees away from the direct impact that your team can have.” — Osian Jones (12:45) “We have to start with a very clear mission. And our mission is to empower everyone to make the best data-driven decisions as fast as possible. And so, hidden within there, that's a function of reducing time to insight, it's also about maximizing trust and obviously minimizing costs.” — Osian Jones (13:48) “We can track [metrics like reliability, incidents, time to resolution, etc.], but also there is a perception aspect to that as well. We can't underestimate the importance of listening to our users and qualitative data.” — Osian Jones (30:16) “These were questions that I felt that I naturally had to ask myself as a product manager. … Understanding who our users are, what they are trying to do with data and what is the current state of our data platform—so those were the three main things that I really wanted to get to the heart of, and connecting those three things together.” – Osian Jones (35:29) “The advice that I would give to anyone who is taking on the role of a leader of a data platform or a similar role is, you can easily get overwhelmed by just so many different use cases. And so, I would really encourage [leaders] to avoid that.” – Osian Jones (37:57) “Really look at your data platform from an end-user perspective and almost think of it as if you were to put the data platform on a supermarket shelf, what would that look like? And so, for each of the different components, how would you market that in a single one-liner in terms of what can this do for me?” – Osian Jones (39:22) Links Stuart: https://stuart.com/ Article on IIA: https://iianalytics.com/community/blog/how-to-build-a-data-platform-as-a-product-a-retrospective Experiencing Data Episode 80 with Doug Hubbard: https://designingforanalytics.com/resources/episodes/080-how-to-measure-the-impact-of-data-productsand-anything-else-with-forecasting-and-measurement-expert-doug-hubbard/ LinkedIn: https://www.linkedin.com/in/osianllwydjones/ Medium: https://medium.com/@osianllwyd

Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview. In the leadership and communications section, 8 Questions to Ask Before Selecting a New Board Leader, How Cybersecurity Leaders Can Build Employee Trust—And Why It Is Important, 7 rules to communicate the business value of IT, and more!   Visit https://www.securityweekly.com/bsw for all the latest episodes! Follow us on Twitter: https://www.twitter.com/securityweekly Like us on Facebook: https://www.facebook.com/secweekly   Show Notes: https://securityweekly.com/bsw291

Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview.   Visit https://www.securityweekly.com/bsw for all the latest episodes! Show Notes: https://securityweekly.com/bsw291

Richard Seiersen and our guest, Doug Hubbard, are finishing the second edition of How to Measure Anything in Cybersecurity Risk. Doug is here to share the success of the first edition and preview the second edition. With more insights, the second edition will share more more research data, free tools, and new concepts like FrankenSME. If you're a risk management professional or want to learn more about risk management, don't miss this interview. In the leadership and communications section, 8 Questions to Ask Before Selecting a New Board Leader, How Cybersecurity Leaders Can Build Employee Trust—And Why It Is Important, 7 rules to communicate the business value of IT, and more!   Visit https://www.securityweekly.com/bsw for all the latest episodes! Follow us on Twitter: https://www.twitter.com/securityweekly Like us on Facebook: https://www.facebook.com/secweekly   Show Notes: https://securityweekly.com/bsw291

In this episode, I spoke with Doug Hubbard, from Hubbard Decision Research, and discussed the Applied Information Economics (AIE) decision-making method. Doug discussed the concept of “analysis placebo” and some biases people have when making decisions, such as partition dependence and the illusion of communication. He also discussed some surprising variables that are more important than people give credit to when making decisions. Listen in to find out what he had to say! I hope you enjoy it! Hubbard Decision Research website: https://hubbardresearch.com/

It's the SEASON FINALE of Hey Chums, with special guest Adam Rose, AKA "Blue Cardigan Guy!" (check him out at @RealAdamRose on all things social). Join us as we (sort of) learn about sea moss, boxing gloves, and more. And strap in for a TOP FIVE tank test from Tank Guy Ralph. Plus: a very special message from our friends and presenting sponsors, Circle Square Studios! Presented by Circle Square Studios: If you can dream it, they can do it. And if you can't dream it, that's okay: they'll dream it for you! (also also: a special shout out to our ad reader Doug Hubbard). Hey, Chums! Welcome to the officially unofficial Shark Tank podcast, with friends & co-hosts Alyssa and Sylvie (plus: guest appearances by noted "tank guy," Ralph). We break down each episode's drama, intrigue, and offers (or lack thereof) each week, so swim on back for more chum about Lori Greiner, Mark Cuban, Kevin O'Leary, Daymond John, Barbara Corcoran, and Robert Herjavec.

It's the 15th episode of Hey Chums, just in time for your holiday weekend listening! Packing for a trip? Already having a panic attack on your way to the airport? Take a moment for yourself, and enjoy Alyssa & Sylvie's take on S13E23 of Shark Tank! With tales of makeup for men (?), a must-have dog accessory, and much more. Plus: a very special message from our friends and presenting sponsors, Circle Square Studios! Presented by Circle Square Studios: If you can dream it, they can do it. And if you can't dream it, that's okay: they'll dream it for you! (also also: a special shout out to our ad reader Doug Hubbard). Hey, Chums! Welcome to the officially unofficial Shark Tank podcast, with friends & co-hosts Alyssa and Sylvie (plus: guest appearances by noted "tank guy," Ralph). We break down each episode's drama, intrigue, and offers (or lack thereof) each week, so swim on back for more chum about Lori Greiner, Mark Cuban, Kevin O'Leary, Daymond John, Barbara Corcoran, and Robert Herjavec.

Get off your Squatty Potty and head over to the nearest speaker for the latest episode of Hey Chums, featuring Shark Tank episode S13E22! We thought this was the finale, but ABC/Hulu had other plans. Enjoy this bonus regular-season ep, featuring an eco-friendly chip, a DINO DON update, entrepreneurs who describe themselves as "partners in life and business," and more. Plus: Tank Guy Ralph shaves his face, and puts the Beard King to the test. Also, a HUGE thank you to our presenting sponsors, Circle Square Studios! If you can dream it, they can do it. And if you can't dream it, that's okay: they'll dream it for you! (also also: a special shout out to our ad reader Doug Hubbard). Hey, Chums! Welcome to the officially unofficial Shark Tank podcast, with friends & co-hosts Alyssa and Sylvie (plus: guest appearances by noted "tank guy," Ralph). We break down each episode's drama, intrigue, and offers (or lack thereof) each week, so swim on back for more chum about Lori Greiner, Mark Cuban, Kevin O'Leary, Daymond John, Barbara Corcoran, and Robert Herjavec.

On this episode we complete the story of Doug Hubbard. Doug leaves NIS, joins the British South African Police in Rhodesia and finds himself in a Civil War. Doug eventually will suffer from the effects of his three years in Vietnam but old friends from his time in Vietnam and Deputy Director at NCIS step up to help in his time of need.

Doug Hubbard continues with his time in Vietnam with NIS. After 36 months in country he makes a decision which will change his life forever.

People often talk about upskilling from a physical security role to becoming a cyber or converged security specialist. But what does that really mean?   On this episode of the podcast, we speak with Shaun Southall, an operator that has expertly and effectively converged the two worlds of physical and cyber together as a working security specialist. Join us this week as we talk about:  Shaun's asymmetric career journey into cyber security. How the ‘uninitiated,' physical security specialist can augment their skills. What steps to take and what to avoid when breaking into the field. The single biggest mindset shift that will determine your success.  As we say here, knowledge is power and, in our industry, “what you don't know can hurt you.” So, tune in and get skilled up with our latest expert sharing his tricks of the trade and gems of experience!   More about Shaun: Shaun is a Cyber Security Oversight Specialist for the Civil Aviation Authority, with almost twenty years of experience, a plethora of certifications, and an ambitious hashtag - #cisoby60! He holds a Level 6 diploma in Security Risk Management and an itch for formal self-improvement that has led to becoming an ISO 27001 Lead Implementer and gaining audit experience through voluntary work, before embarking on four ISACA certifications – CISM, CRISC, CDPSE and COBIT Foundation – in just four months. He is heavily influenced by Doug Hubbard, Alexei Sidorenko and Norman Marks and is driven to shift the mentality from ‘red amber green and five by fives' to a holistic appreciation of risk that considers the complexity of the threat landscape. Shaun is also a regular presenter at ASIS CPE events, promoting risk quantification and ‘debiasing the human', an active member of SIRA, FAIR and ISACA London Chapter, is a proud to be part of an organisation tasked with maintaining the safety and security of aviation in the UK. https://www.linkedin.com/in/shaunsouthall/ (Linkedin) More about the Circuit: The Circuit Magazine is written and produced by volunteers, most of who are operationally active, working full time in the security industry. The magazine is a product of their combined passion and desire to give something back to the industry. By subscribing to the magazine you are helping to keep it going into the future. https://circuit-magazine.com/read/ (Find out more >) If you liked this podcast, we have an accompanying weekly newsletter called 'On the Circuit' where we take a deeper dive into the wider industry. http://bit.ly/OntheCircuit (Opt in here >) The Circuit team is: Elijah Shaw Jon Moss Shaun West Phelim Rowe Connect with Us:  https://circuit-magazine.com/ (Circuit Magazine) https://mailchi.mp/the-bba.org.uk/bba-connect (BBA Connect) https://www.theprotectorapp.com/ (NABA Protector) https://the-bba.org.uk/ (British Bodyguard Association)

Finding it hard to know the value of your data products on the business or your end users? Do you struggle to understand the impact your data science, analytics, or product team is having on the people they serve?   Many times, the challenge comes down to figuring out WHAT to measure, and HOW. Clients, users, and customers often don't even know what the right success or progress metrics are, let alone how to quantify them. Learning how to measure what might seem impossible is a highly valuable skill for leaders who want to track their progress with data—but it's not all black and white. It's not always about “more data,” and measurement is also not about “the finite, right answer.” Analytical minds, ready to embrace subjectivity and uncertainty in this episode!  In this insightful chat, Doug and I explore examples from his book, How to Measure Anything, and we discuss its applicability to the world of data and data products. From defining trust to identifying cognitive biases in qualitative research, Doug shares how he views the world in ways that we can actually measure. We also discuss the relationship between data and uncertainty, forecasting, and why people who are trying to measure something usually believe they have a lot less data than they really do.  Episode Description A discussion about measurement, defining “trust”, and why it is important to collect data in a systematic way. (01:35) Doug explores “concept, object and methods of measurement” - and why most people have more data than they realize when investigating questions. (09:29) Why asking the right questions is more important than “needing to be the expert” - and a look at cognitive biases. (16:46) The Dunning-Kruger effect and how it applies to the way people measure outcomes - and Bob discusses progress metrics vs success metrics and the illusion of cognition. (25:13) How one of the challenges with machine learning also creates valuable skepticism - and the three criteria for experience to convert into learning. (35:35) Quotes from Today's Episode “Often things like trustworthiness or collaboration, or innovation, or any—all the squishy stuff, they sound hard to measure because they're actually an umbrella term that bundles a bunch of different things together, and you have to unpack it to figure out what it is you're talking about. It's the beginning of all scientific inquiry is to figure out what your terms mean; what question are you even asking?”- Doug Hubbard (@hdr_frm) (02:33) “Another interesting phenomenon about measurement in general and uncertainty, is that it's in the cases where you have a lot of uncertainty when you don't need many data points to greatly reduce it. [People] might assume that if [they] have a lot of uncertainty about something, that [they are] going to need a lot of data to offset that uncertainty. Mathematically speaking, just the opposite is true. The more uncertainty you have, the bigger uncertainty reduction you get from the first observation. In other words, if, you know almost nothing, almost anything will tell you something. That's the way to think of it.”- Doug Hubbard (@hdr_frm) (07:05)  “I think one of the big takeaways there that I want my audience to hear is that if we start thinking about when we're building these solutions, particularly analytics and decision support applications, instead of thinking about it as we're trying to give the perfect answer here, or the model needs to be as accurate as possible, changing the framing to be, ‘if we went from something like a wild-ass guess, to maybe my experience and my intuition, to some level of data, what we're doing here is we're chipping away at the uncertainty, right?' We're not trying to go from zero to 100. Zero to 20 may be a substantial improvement if we can just get rid of some of that uncertainty, because no solution will ever predict the future perfectly, so let's just try to reduce some of that uncertainty.”- Brian T. O'Neill (@rhythmspice) (08:40) “So, this is really important: [...] you have more data than you think, and you need less than you think. People just throw up their hands far too quickly when it comes to measurement problems. They just say, ‘Well, we don't have enough data for that.' Well, did you look? Tell me how much time you spent actually thinking about the problem or did you just give up too soon? [...] Assume there is a way to measure it, and the constraint is that you just haven't thought of it yet. ”- Doug Hubbard (@hdr_frm) (15:37) “I think people routinely believe they have a lot less data than they really do. They tend to believe that each situation is more unique than it really is [to the point] that you can't extrapolate anything from prior observations. If that were really true, your experience means nothing.”- Doug Hubbard (@hdr_frm) (29:42) “When you have a lot of uncertainty, that's exactly when you don't need a lot of data to reduce it significantly. That's the general rule of thumb here. [...] If what we're trying to improve upon is just the subjective judgment of the stakeholders, all the research today—and by the way, here's another area where there's tons of data—there's literally hundreds of studies where naive statistical models are compared to human experts […] and the consistent finding is that even naive statistical models outperform human experts in a surprising variety of fields.”- Doug Hubbard (@hdr_frm) (32:50) Links Referenced How to Measure Anything: https://www.amazon.com/gp/product/1118539273/ Hubbard Decision Research: https://hubbardresearch.com  

Today we talk with Richard Seiersen, co-author of “How to Measure Anything in Cybersecurity Risk”. Richard shared that at his first CISO position, he was challenged with addressing prioritization of risk, which led to his authoring a book with Doug Hubbard. What can cyber learn from older risk disciplines? The life table used broadly to measure time-to-event data goes back 500 years. Businesses keep falling back to the classic 5x5 "likelihood and impact" matrix which is an inconsistent, non-math-based method. Without math it is really just casting spells in the board room. There are no ratios or explanation of differences, for example. CISOs are called upon to make a bet about something. We will use subject matter expert opinions, and can make them measurably better. Consistency is key. Wild guesses can still help constrain the forecast. There are existing models in cyber such as FAIR that provide a more mathematically applied approach. Statistics came about because people needed to make bets with limited data. Dirty data can be worked with. Embracing uncertainty is okay. Executives are actually very used to uncertainty. Cybersecurity as a practice is in its adolescence with a high mortality risk. We need to adopt the grammar of science. Key Takeaways 0:25 Richard is introduced 1:20 Richard talks about his cyber journey and his day job 3:02 Book talk 5:19 What can cyber learn from older style risk tactics 8:04 5x5 risk matrix 10:05 Improving accuracy 17:00 Gathering an accurate view 19:20 Monte Carlo simulations 22:04 The belief 25:17 Board-ready presentations 26:58 What keeps Richard going in cyber security 28:09 Why statistics were invented Links: Learn more about Richard Seiersen on LinkedIn and Twitter Follow Allan Alford on LinkedIn and Twitter Learn more about Hacker Valley Studio and The Cyber Ranch Podcast Sponsored by our good friends at Axonius

Doug Hubbard is the author of several books and I've read two: *How to Measure Anything* and *The Failure of Risk Management*. I can honestly say that one of my career goals is to implement his methodology into my job today and everything I do in the future. Here's an incomplete list of wow realizations that I had reading Doug:That you can overcome cognitive bias in estimating variationThat we don't measure what's most importantThat we can quantify the value of informationThat we can quantify uncertainty and use it to make decisionsThat expert opinion can be calibrated and aggregated and use in a quantifiable mannerThat Bayesian statistics explain the reduction in uncertainty that accompanies additional information. That last one is an *empirical* observation. I remain floored by that. Floored. I didn't even cover half of what I wanted to cover with Doug. Read Doug Hubbard. Learn from Doug Hubbard. I will continue to!Show notes:https://notunreasonable.com/?p=7298

Jeromy Carriere, Engineering Director at Facebook, shares insight on how to enable high-performing teams on this episode of Storyliving! We hope you enjoy learning from his vast experience at Google, Fidelity, and Microsoft, including an inspiring story about selling a successful startup to AOL during the dot-com era and lessons from Doug Hubbard's measurement research.

A ground shaking exposé on the failure of popular cyber risk management methods. This book is the first of a series of spinoffs from Douglas Hubbard’s successful first book, How To Measure Anything: Finding the Value of “Intangibles” in Business.   Learn more on how to quantify risk in terms of dollars and cents in order to build better "business impact" decision makers, visit: https://hubbardresearch.com/ Visit https://www.securityweekly.com/bsw for all the latest episodes! Show Notes: https://wiki.securityweekly.com/bsw183

This week, it's Security Weekly Virtual Hacker Summer Camp, and we have two interviews! First, we welcome Matt Ashburn, Federal Engagement Lead at Authentic8, to discuss "How Security Spending Overlooks the Biggest Risk of All"! Then, we welcome Doug Hubbard, Founder at Hubbard Decision Research, to discuss "The Failure of Risk Management"!   Show Notes: https://wiki.securityweekly.com/bsw183 Learn more on how to quantify risk in terms of dollars and cents in order to build better "business impact" decision makers, visit: https://hubbardresearch.com/ Visit https://www.authentic8.com/bsw to learn more about them!   Join the Security Weekly Discord Server: https://discord.gg/pqSwWm4 Visit https://www.securityweekly.com/bsw for all the latest episodes! Follow us on Twitter: https://www.twitter.com/securityweekly  

This week, it's Security Weekly Virtual Hacker Summer Camp, and we have two interviews! First, we welcome Matt Ashburn, Federal Engagement Lead at Authentic8, to discuss "How Security Spending Overlooks the Biggest Risk of All"! Then, we welcome Doug Hubbard, Founder at Hubbard Decision Research, to discuss "The Failure of Risk Management"!   Show Notes: https://wiki.securityweekly.com/bsw183 Learn more on how to quantify risk in terms of dollars and cents in order to build better "business impact" decision makers, visit: https://hubbardresearch.com/ Visit https://www.authentic8.com/bsw to learn more about them!   Join the Security Weekly Discord Server: https://discord.gg/pqSwWm4 Visit https://www.securityweekly.com/bsw for all the latest episodes! Follow us on Twitter: https://www.twitter.com/securityweekly  

A ground shaking exposé on the failure of popular cyber risk management methods. This book is the first of a series of spinoffs from Douglas Hubbard’s successful first book, How To Measure Anything: Finding the Value of “Intangibles” in Business.   Learn more on how to quantify risk in terms of dollars and cents in order to build better "business impact" decision makers, visit: https://hubbardresearch.com/ Visit https://www.securityweekly.com/bsw for all the latest episodes! Show Notes: https://wiki.securityweekly.com/bsw183

In this episode Julia talks with Doug Hubbard, author of How to Measure Anything, about why people so often believe things are impossible to quantify like "innovation" or "quality of life." For example, because people often have a deep misunderstanding of the meaning of probability. Or because they're reluctant to violate "sacred taboos" by putting a number on something like the value of human life. Or because it feels vulgar to "reduce" important things to a number. Doug explains how he responds to these objections and others.

Douglas Hubbard (@hdr_frm) wrote a book entitled ‘How to Measure Anything: Finding the Value of Intangibles in Business’. He is also the founder of Hubbard Decision Research (HDR) a consultancy that applies quantitative analysis methods to the most difficult measurements and challenging decisions in business. He explains some techniques for measuring things that are hard to measure and then walks me through an example of how one would measure what is often thought about as intangible quality. We had a fun time talking.

Episode 28 In this episode, Jay sat down with Doug Hubbard and Richard Seiersen to talk about their upcoming book "How to Measure Anything in Cybersecurity Risk". Bob talks about the rOpenSci unconference and the two talk about 2 recent publications. rOpenSci rNOAA When-ish is my Bus (pdf) Dell Secureworks Underground Hacker Marketplace Report How to Measure Anything in Cybersecurity Risk