Podcasts about Reproducibility Project

Brian Nosek is a social-cognitive psychologist, professor at the University of Virginia, and co-founder and director of the Center for Open Science. In 2011, Brian and his colleagues launched the Reproducibility Project which would ultimately transform science forever. In this episode we discuss the following: Reputation is how people perceive us. But integrity is what we get to choose for ourselves. We can hold ourselves accountable for our integrity, but when we worry about our reputation, we're prone to get led astray. If we try to control our reputation, we're prone to avoid risk (e.g., we don't do the things we should do because we might make people mad). If we try to control our reputation, we may deviate from our values in an attempt to keep other people happy. We undermine ourselves when we prioritize reputation over integrity. Our long-term reputation will ultimately derive from our integrity. You can't control your reputation. You can control your integrity. Brian was told he was ruining his career. But by focusing on integrity over reputation, Brian and his colleagues revolutionized science.   Follow Brian: Twitter: https://twitter.com/BrianNosek LinkedIn: https://www.linkedin.com/in/brian-nosek-682b17114/ Follow Me: Twitter: https://twitter.com/nate_meikle LinkedIn: https://www.linkedin.com/in/natemeikle/ Instagram: https://www.instagram.com/nate_meikle/

In this episode, we discuss the replication crisis in psychology which has been an important topic of discussion for the last decade. We revisit some key events from the start of the replication crisis, such as the publication of Daryl Bem's studies on precognition, the paper False Positive Psychology, and the Reproducibility Project and share personal anecdotes about how it was to live through the replication crisis.   Shownotes:  Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100(3), 407–425. https://doi.org/10.1037/a0021524 Ritchie, S. J., Wiseman, R., & French, C. C. (2012). Failing the Future: Three Unsuccessful Attempts to Replicate Bem's ‘Retroactive Facilitation of Recall' Effect. PLOS ONE, 7(3), Article e33423. https://doi.org/10.1371/journal.pone.0033423 Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632 John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953 Fiedler, K., & Schwarz, N. (2016). Questionable Research Practices Revisited. Social Psychological and Personality Science, 7(1), 45–52. https://doi.org/10.1177/1948550615612150 NOTE: Daniel says in the podcast the paper below is by Fiedler and Strack - but it is by Fiedler and Schwarz. Ebersole, C. R., Mathur, M. B., Baranski, E., Bart-Plange, D.-J., Buttrick, N. R., Chartier, C. R., Corker, K. S., Corley, M., Hartshorne, J. K., IJzerman, H., Lazarević, L. B., Rabagliati, H., Ropovik, I., Aczel, B., Aeschbach, L. F., Andrighetto, L., Arnal, J. D., Arrow, H., Babincak, P., … Nosek, B. A. (2020). Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245920958687 Luttrell, A., Petty, R. E., & Xu, M. (2017). Replicating and fixing failed replications: The case of need for cognition and argument quality. Journal of Experimental Social Psychology, 69, 178–183. https://doi.org/10.1016/j.jesp.2016.09.006 Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017). Constraints on Generality (COG): A Proposed Addition to All Empirical Papers. Perspectives on Psychological Science, 12(6), 1123–1128. https://doi.org/10.1177/1745691617708630 Simonsohn, U. (2015). Small Telescopes Detectability and the Evaluation of Replication Results. Psychological Science, 26(5), 559–569. https://doi.org/10.1177/0956797614567341  

For the past several years, social scientists have been increasingly aware of what is called the “replication crisis” in their disciplines. In 2015, the Reproducibility Project at the University of Virginia tried to replicate the results of nearly 100 studies published in psychology journals. Even after going “through extensive measures to remain true to the original studies,” they could only replicate the original findings in 36 percent of the studies, and even then, the effects “were smaller than the initial studies effects.” This problem with replicating results isn't limited to psychology or the social sciences. As Pascal-Emmanuel Gobry of the Ethics and Public Policy Center has noted, medicine is often afflicted with a similar replicability problem. As the saying goes, the first step in fixing a problem is acknowledging you have one. However, given the personal, professional, and political stakes often at work here, that's easier said than done. That's why a recent bit of honesty by a publication you may never have heard of is so welcome. Back in 2014, the journal Social Science & Medicine published a study entitled “Structural stigma and all-cause mortality in sexual minority populations.” The study attempted to measure the impact of anti-gay prejudice on the death rate of gay men and lesbians. The study concluded that “Sexual minorities living in communities with high levels of anti-gay prejudice experienced a higher hazard of mortality than those living in low-prejudice communities.” Specifically, this increased “hazard of mortality” “[translated] into a shorter life expectancy of approximately 12 years . . . for sexual minorities living in high-prejudice communities.” The study was used as scientific confirmation of an emerging cultural narrative. Not only is anti-gay prejudice wrong, we are often told, it can even be lethal. Well, not so fast. Mark Regnerus of the University of Texas did what social scientists in the peer-review process are supposed to do… He set out to replicate the results of that study. Going over the same data set, he found multiple errors, flawed assumptions, and possible “confounding variables,” possible factors that the researchers failed to take into account: ethnicity, age, sex, and smoking, to name but four. In a 2017 study published in the same journal, Regnerus, who by the way is a Christian, concluded “Attempts to replicate the study . . . repeatedly failed to generate the original study's key finding on structural stigma.” To put it mildly, questioning the findings didn't go over so well. Nathaniel Frank of the What We Know project, which is a gay research portal, summed up the views of many who commented when he wrote that, “Mark Regnerus destroyed his scholarly credibility when . . . he allowed his ideological beliefs to drive his conclusions,” and that “there's an enormous body of research showing the harms of [sexual] minority stress, and Regnerus is simply not a trustworthy critic of this research.” Except, as it turns out, Regnerus is trustworthy. Earlier this month Social Science & Medicine announced that the original study had been retracted at the original authors' request. In response to Regnerus' findings, they hired another group to try and replicate their original findings, and that group came to the same conclusion as Regnerus. As one social science website put it, “It was a mistake of the journal Social Science & Medicine to publish that original paper, but, on the plus side, they published Regnerus's criticism. Much better to be open with criticism than to play defense.” Exactly. Mistakes like this are inevitable. All we can ask is that, once the mistake has been pointed out, ideology should not be allowed to get in the way of the truth. The journal did the right thing, which is as welcome as it is rare. Don't forget that BreakPoint's book offer is John Lennox's “Can Science Explain Everything?” Until the end of the month, you can download a free sample chapter by coming to BreakPoint.org/free.     http://www.breakpoint.org/2019/03/breakpoint-social-science-and-cultural-narratives/  

A look at the Robust Research Initiative at the University of Oxford

In this episode, we show how you can decode scientific studies and spot bad science by digging deep into the tools and skills you need to be an educated consumer of scientific information. Are you tired of seeing seemingly outrageous studies published in the news, only to see the exact opposite published a week later? What makes scientific research useful and valid? How can you, as a non-scientist, read and understand scientific information in a simple and straightforward way that can help you get closer to the truth - and apply those lessons to your life. We discuss this and much more with Dr. Brian Nosek.  Dr. Brian Nosek is the co-founder and Executive Director of the Center for Open Science and a professor of psychology at the University of Virginia. Brian led the reproducibility project which involved leading some 270 of his peers to reproduce 100 published psychology studies to see if they could reproduce the results. This work shed light on some of the publication bias in the science of psychology and much more.Does the science show that extrasensory perception is real?Is there something wrong with the rules of the science or the way that we conduct science?What makes academic research publishable is not the same thing as what makes academic research accuratePublication is the currency of advancement in scienceNovel, positive, cleanWhat does “Nulls Hypothesis significance testing” / P-Value less than .05 even mean?Less than 5% of the time would you observe this evidence if there was no relationshipThe incentives for scientific publishing often skew, even without conscious intent by scientists, towards only publishing studies that support their hypothesis and conclusionsThe conclusions of many scientific studies may not be reproducible and may, in fact, be wrong How the reasoning challenges and biases of human thinking skew scientific results and create false conclusionsConfirmation biasOutcome bias“The Reproducibility Project” in psychologyTook a sample of 100 studies Across those 100 studies - the evidence was able to be reproduced only 40% of the timeThe effect size was 50% of what it was What The Reproducibility Project spawned was not a conclusion, but a QUESTIONHow do we as lay consumers determine if something is scientifically valid or not?We discuss the basic keys to understanding, reading, and consuming scientific studies as a non-scientist and ask how do we determine the quality of evidence?Watch out for any DEFINITIVE conclusionsThe sample size is very important, the larger the betterAggregation of evidence is better - “hundreds of studies show"Meta-studies / meta-analysis are important and typically more credibleLook up the original paperIs there doubt expressed in the story/report about the data? (how could the evidence be wrong, what needs to be proven next, etc)Valid scientific research often isn’t newsworthy - it takes lots of time to reach valid scientific conclusions It’s not just about the OUTCOME of a scientific study - the confidence in those outcomes is dependent on the PROCESS Where do we go from here as both individuals and scientists? How can we do better?Transparency is keyPreregistration - commit to a designThe powerful tool of “pre-registration” and how you can use it to improve your own thinking and decision-makingHomework - deliberately seek out people who disagree with you, build a “team of rivals" Learn more about your ad choices. Visit megaphone.fm/adchoices

Papers and articles about reproducing both cancer biology and stem cell results (and how that affected the lives of the scientists). Papers about the Reproducibility Project in Psychology that started at UVA.

In this episode, Brian Earp discusses the 'Reproducibility Project' and questions whether psychology is in crisis or not. In a much-discussed New York Times article, psychologist Lisa Feldman Barrett claimed, “Psychology is not in crisis.” She was responding to the results of a large-scale initiative called the Reproducibility Project, published in Science magazine, which appeared to show that the results from over 60% of a sample of 100 psychology studies did not hold up when independent labs attempted to replicate them. In this talk, Earp addresses three issues: what did the Reproducibility Project really show?; is psychology in crisis or not?; and is there room for improvement?

In this episode, Brian Earp discusses the 'Reproducibility Project' and questions whether psychology is in crisis or not. In a much-discussed New York Times article, psychologist Lisa Feldman Barrett claimed, “Psychology is not in crisis.” She was responding to the results of a large-scale initiative called the Reproducibility Project, published in Science magazine, which appeared to show that the results from over 60% of a sample of 100 psychology studies did not hold up when independent labs attempted to replicate them. In this talk, Earp addresses three issues: what did the Reproducibility Project really show?; is psychology in crisis or not?; and is there room for improvement?

En 2012, on frissonne : Brian Nosek, professeur de psychologie à l’université de Virginie, annonce qu’il va coordonner la réplication de … Lire la suite

Nok en episode psykologlunsj er klar. Nummer 69 i rekken. Denne gangen snakker Jan-Ole om en studie som tar for seg om hvorvidt kvinner har preferanse for maskuline eller feminine menn. Tommy tar for seg en sak som har blitt en gjenganger i lunsjen: the reproducibility project.  Send spørsmål og innspill til oss gjennom vår facebookside eller […] Relaterte poster: #44 – Phineas Gage #55 – Sykkelfantomet og mannen som ikke kunne huske #64 – Julehjerne, bullshitgeneratorer og diagnosebonanza

Vi synger igang en ny episode av Psykologlunsj. Tommy snakker om nyttårsforsetter, og om hvorvidt de holdes eller ikke. Jan-Ole ønsker å snakke om sitt favorittema for tiden: The Reproducibility Project. Sarte sjeler bes spole forbi sangen til Jonas. Denne episoden er redigert av Morten Brøndmo ved Oslo Evenstudio. Send spørsmål og innspill til oss […] Relaterte poster: #27 Parasittpsykologi og hvordan er det å jobbe som psykolog? #64 – Julehjerne, bullshitgeneratorer og diagnosebonanza #56 – Feriepsykologi og fartsnevroner

How good is most psychological research? The field of psychology has long struggled with a reputation for not being a "real" science. Now, thanks to the Reproducibility Project, an ambitious attempt to replicate the findings of dozens of published studies, it seems to many that psychology research is at a crossroads. Mallory Kidwell, a project coordinator at Center for Open Science, explains why reproducibility matters, what steps the field can take to address the issue, and how the public should interpret the Reproducibility Project's shockingly low rate of successful replications.

Brian Nosek of the University of Virginia and the Center for Open Science talks with EconTalk host Russ Roberts about the Reproducibility Project--an effort to reproduce the findings of 100 articles in three top psychology journals. Nosek talks about the findings and the implications for academic publishing and the reliability of published results.

The fourth part of four episodes on anxiety. The workshop has wrapped up, and we are reviewing what was presented at the workshop here in episode 16. The workshop at St. Peter's Episcopal average around 25 people per week. There was a lot of positive feedback about the presentations. I hope to be able to do more workshops in the future. If your group is interested in a presentation on anxiety, depression, relationships, parent/child communication, or a wide variety of content about psychology let me know. I'd be happy to present depending on the schedule.  For our Did You See That segment we review the podcast by Michael Britt. Professor Britt does a podcast titled The Psych Files. He reviewed the Reproducibility Project. His take is that the project does more to bring to light the problems we have with the research culture in general rather than showing that psychological research is suspect itself. Check out his podcast at: http://www.thepsychfiles.com/2015/09/ep-245-the-results-of-the-reproducibility-project-incentives-out-of-whack/ Our main segment reviews what was covered in the fourth workshop. We have covered various practical techniques to manage anxiety, but what about living our lives to the fullest with or in spite of anxiety. In the episode I talk about that anxiety holds us in place or causes us to run away. We are afraid of change. The fear is because we are afraid of losing something, the process of change, or the outcomes. Facing our fears we can see that change doesn't need to be all in one step. It can be step by step. What is one small thing you can do to make a change today?  One of the major problems with facing our fear and making the change is not feeling valuable enough, or being willing to be vulnerable. I reference a Ted talk by Brene Brown to discuss how we can increase our sense of worth to be vulnerable to risk change. http://www.ted.com/talks/brene_brown_on_vulnerability Finally, we talk look at Colossians 3:12-14 and Galatians 3:26-29 to explore how we are children of God. We are heirs of the process. You are worthy. As always connect with me to continue the conversation at: info@drneilstafford.com On Facebook at Dr Neil Stafford Or Twitter at @drseab Be awesome by being you!

Here on the show, we’ve talked about the lack of reproducibility for much of biological research. We’ve bandied around various percentages--is it 50% or up to 90% that can't be replicated? And we’ve poked around various issues that may be causing such poor science. Nicole Perfito is the manager of the Reproducibility Project: Cancer Biology, an effort between Science Exchange and the Center for Open Science. The goal of this project is to take nearly forty “high impact” papers in the field of cancer and try to replicate them.

On today’s Modern Notion Daily podcast, our guest is Roger Giner-Sorolla, a professor of social psychology at the University of Kent and a member of the team working on the Reproducibility Project for Psychology. The Reproducibility Project tasked psychologists with choosing 100 studies that they could replicate in their own labs as a way of…

Songbirds in the city, such as the Great Tit (Parus major), are learning to sing their mating calls at a higher pitch to cut through the urban clamour. But while this may help them be heard, the different tune may hamper their mating success.Sugar is today's popular dietary villain, but does it make sense to demonise a single nutrient? Dr Alan Barclay from the Dietitians Association of Australia explains why we shouldn't just follow the latest fads.Just because a scientific study gets a result, that doesn't mean it's true. The Reproducibility Project tried to replicate 100 psychology papers, and only 36% got a significant result (consistent with a recent high-profile retraction of a paper about canvassing support for same-sex marriage). As John Ioannidis pointed out in 2005, most published research findings are false, and the best way to improve this is to try to remove bias and do larger studies with more statistical significance.

It's been called the "decline effect," "the proteus phenomenon," and "the reproducibility crisis": the startling realization that a lot of seemingly solid scientific research doesn't pan out under repeated testing. The latest blow to scientific confidence comes from the Reproducibility Project, which attempted to replicate 100 published psychology studies and found that, when the experiments were repeated, half or more failed to uphold the original findings. So is it time to start doubting the credibility of research in general? Stanford University psychologist and Reproducibility Project participant Mike Frank joined us to explain what the results really mean, misconceptions about statistical rigor in science, the various ways experimenters blunder and sometimes delude themselves, and the gradual, cumulative nature of scientific progress.

Have you heard that about 100 Psychology studies were replicated and only about 1/3 confirmed the original findings? Why did this happen? Well, one reason has to do with incentives that are out of whack. The "real world" of scientific research is far from the lone researcher looking for the truth. And the other reason has to do with, well, you and the internet. You see, you like to click on things that are surprising or weird (I like to do that too I admit) and that behavior encourages bad research. Let's find out how these things are all connected in this episode of The Psych Files.

  The Reality Check celebrates its 7th birthday! This week we’re bringin’ the science and the sci-fi with two fact-filled segments. First, Darren kicks things off with an analysis of the Reproducibility Project which looks at experimental and correlational studies published in the field of Psychological Science. Adam dives head first into recent DNA studies on octopuses and the resulting headlines suggesting they come from an alien world.

  The Reality Check celebrates its 7th birthday! This week we’re bringin’ the science and the sci-fi with two fact-filled segments. First, Darren kicks things off with an analysis of the Reproducibility Project which looks at experimental and correlational studies published in the field of Psychological Science. Adam dives head first into recent DNA studies on octopuses and the resulting headlines suggesting they come from an alien world.

Justin and Jason discuss Justin's upcoming move back to LA, an update on Appignite, avoiding depression while not running a $1B company, the new Pluggio UI and why some existing customers don't like it, Company 52's new focus, Jason's thoughts on Steve Job's biography, the new interview format, how Jason bought some Facebook shares, Mesh01's upcoming design competition for a "luck surface area" t-shirt, the aborted SpaceX launch, why Steve Blank thinks the age of Silicon Valley is over, when a blow to the head creates a sudden genius, the Reproducibility Project, the genetic basis for why Jason talks so much and the progress being made on AnyFu.