ReproducibiliTea Podcast

In this episode, Will is joined by Lianne Wolsink, PhD candidate at Ruhr University Bochum and current steering committee member of ReproducibiliTea. Will and Lianne discuss the ReproducibiliTea reading lists, created to help journal clubs do deep dives on metascience topics. Lianne created reading lists on replication and science communication; Will created an introductory reading list on Open Science, preregistration, and theory in psychology. Reading lists: https://rpt-rl.netlify.app Books mentioned: Science Fictions by Stuart J Ritchie Against Method by Paul Feyerabend Nobody's Fool by Daniel Simons and Christopher Chabris

Show notes: OSF Preprints | Let's talk about it: positivism and critical theory in dialogue https://osf.io/preprints/osf/t2ywh_v1 Caliban & the Witch by Silvia Federici Pollution is Colonialism by Max Liboiron Against Method by Paul Feyerabend Lecture on Against Method by Tomas Petricek, University of Cambridge : https://youtu.be/jzD1O_gastA S2E10: Limits of ReproducibiliTea for discussion and resources of a feminist lens on open science https://soundcloud.com/reproducibilitea/s1e10-limits The battle for your brain by Nita Farahany Quokka app: https://palm-lab.github.io/QualCA Quokka video guide: https://www.youtube.com/watch?v=9jJK70W0hAI

In this episode, Sarah and Will chat to Josh de Leeuw from Vassar College and the creator of jsPsych. We chat about the history of jsPsych, the unseen process behind creating open-access scientific software, and the current challenges facing software developers in the open scholarship movement. jsPsych is a javaScript framework for creating online experiments, and is always looking for people to contribute to the codebase: https://jspsych.org. Follow Josh de Leeuw on Bluesky: https://bsky.app/profile/joshdeleeuw.bsky.social

Why is Indigenous health research so crucial, especially for adolescents? What are the systemic challenges researchers face, and how can we push for openness, transparency, and equity in health research? In this conversation, hosts Queen Saikia and Sarah Sauvé speak with leading experts from New Zealand - Prof. Sue Crengle (University of Otago), Prof. Terryann Clarke (University of Auckland), and Dr. Andrew Sise (University of Otago) - to explore the current landscape of Indigenous health research and the methodologies used in these investigations. They also discuss key issues around reproducibility, openness, and transparency, the discrepancies in research systems, and how we can drive meaningful change. This episode is a must-listen for anyone passionate about health equity, open research, and improving research culture. Listen now and join the conversation in the comments!

In this episode, Will is joined by Jamie Moffa, a doctoral student in systems neuroscience at Washington University in St. Louis. Jamie has been thinking and working in the science communication space, especially via the In Plain English podcast, which is aimed at bringing scientific knowledge and understanding to the general public. Show Notes: We think about this paper: Volk, S. C. (2024). Assessing the Outputs, Outcomes, and Impacts of Science Communication: A Quantitative Content Analysis of 128 Science Communication Projects. Science Communication, 10755470241253858. https://journals.sagepub.com/doi/10.1177/10755470241253858 Will mentions this paper by C Thi. Nguyen: Nguyen, C. T. (2021). The seductions of clarity. Royal Institute of Philosophy Supplements, 89, 227-255. https://philarchive.org/rec/NGUTSO-2 Will mentions this paper about color constancy and Crocs randomly...: Wallisch, P., & Karlovich, M. (2019). Disagreeing about Crocs and socks: Creating profoundly ambiguous color displays. arXiv preprint arXiv:1908.05736. Follow and reach out to Jamie, especially if you'd like to contribute to the In Plain English podcast! Jamie Moffa – https://copitslab.wustl.edu/people/jamie-moffa/ In Plain English Podcast – https://inplainenglishpod.org/ Our science communicator highlights: Nature and Nurture Podcast by Adam Omary –https://podcasters.spotify.com/pod/show/NatureNurture Cass Eris – https://www.youtube.com/casseris Dr Neurofourier – https://www.youtube.com/c/Neurofourier SciShow by Complexly (Hank and John Green): https://www.youtube.com/@SciShow Science Night Podcast – https://www.scinight.com/episodes Ed Yong (no longer at the Atlantic!) – https://edyong.me/ The Violinist's Thumb by Sam Kean – https://samkean.com/books/the-violinists-thumb/ You can find Will on BlueSky: https://bsky.app/profile/willngiam.bsky.social If you'd like to find out more about ReproducibiliTea, our grassroots initiative to build community in Open Science across institutions, check out https://reproducibilitea.org.

In this episode, Will and Helena are joined by Emmanuel Boakye and Lamis Elkheir to share their experiences as scientists and Open Science advocates in the Global South and how they started the African Reproducibility Network (AREN). African Reproducibility Network Website: https://africanrn.org/ Twitter: https://twitter.com/africanrepro Lamis Elkheir LinkedIn: https://sd.linkedin.com/in/lamis-elkheir-b5844092 Twitter: https://twitter.com/lamiselkheir?lang=en Emmanuel Boakye LinkedIn: https://gh.linkedin.com/in/emmaboakye Twitter: https://twitter.com/thescientistgh

In this episode, we welcome Queen Saikia as a host of the podcast! She and Will Ngiam are joined by Jonny Coates, Associate Director of ASAPBio, a non-profit organisation seeking to Accelerate Science and Publication in Biology. The topic of conversation is preprint review and peer review. Enjoy! Show notes: ASAPBio: https://asapbio.org/ PREreview: https://prereview.org/

We welcome back the ReproducibiliTea Podcast with Will and Helena chatting to Nafisa Jadavji and Nele Haelterman about Reproducibility for Everyone (R4E), a community-led initiative to run reproducibility workshops. Show notes: Repro4Everyone - https://repro4everyone.org

Today, Sarah, Will and Jan sit down to discuss the last season of ReproducibiliTea. We talk about Will's terrible terrible taste in puns, Jan's terrible taste in pizza, and Sarah's fall into FORRT(.org). While we are wrapping up Season 3, stay tuned for a few more fun & exciting episodes we have planned for this year! Not many footnotes this time: Sarah mentioned sysmus: https://sites.google.com/view/sysmus/home For more info go to ReproducibiliTea.org For comments, questions, tips and tricks, and praise use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will sits down with Max Korbmacher, Thomas Rhys Evans, and Flavio Azevedo, some of the authors of the paper "The replication crisis has led to positive structural, procedural, and community changes" to talk about the paper, FORRT, and Open Science communities. Show notes: The paper we discuss for this episode: Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Nature Communications Psychology, 1(1), 3. https://www.nature.com/articles/s44271-023-00003-2 FORRT – The Framework for Open and Reproducible Research Training: https://forrt.org Getting involved with FORRT: https://forrt.org/about/get-involved/ Charlotte Pennington's new book: A Student's Guide to Open Science: Using the Replication Crisis to Reform Psychology https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group UK Reproducibility Network: https://www.ukrn.org/ Project Teaching Integrity in Empirical Research (TIER): https://www.projecttier.org/ Reproducibility Wiki: https://replication.uni-goettingen.de/ Paper Trail: https://thepapertrailjc.squarespace.com/ Brain Imaging Data Structure (BIDS): https://bids.neuroimaging.io/ Collaborative Replication Education Project (CREP): https://www.crep-psych.org/ The Center for Open Science: https://www.cos.io/ Nowhere Lab: http://nowherelab.com/ Advancing Big-team Reproducible Science through Increased Representation (ABRIR): https://abrirpsy.org/ Open Life Science: https://openlifesci.org/ Turing Way: https://the-turing-way.netlify.app/index.html For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will talks to David Reinstein about scientific publishing and The Unjournal. The Unjournal: https://www.unjournal.org The Unjournal on LinkedIn: https://www.linkedin.com/company/unjournal/ To get the latest updates: https://bit.ly/ujupdates To apply for positions at the Unjournal: https://bit.ly/Ujwork Will and David's extended notes for the episode: https://docs.google.com/document/d/13rL6mq71GD6gPv5wBHr_lfX6YSKkLK9n6EAVfGhZhCk/edit?usp=sharing For more info go to https://ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Will sits down with Björn Jorges and Sabrina Hansmann-Roth to discuss the role of academic societies in the science reform movement. The poster session: https://www.visionsciences.org/2023-pre-data-collection-poster-session-satellite/ Korbmacher, M., Azevedo, F., Pennington, C., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Communications Psychology. https://gala.gre.ac.uk/id/eprint/42730842730_EVANS_The_replication_crisis_has_led_to_positive_structural_procedural_and_community_changes.pdf As always: For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today Sarah chats to Zoltan Dienes Live from SIPS! Zoltan's keynote: https://www.youtube.com/watch?v=BxdGXLOC1Cc Reviewing labor: https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-021-00118-2?ref=refind Peer Community In Registered Reports: https://rr.peercommunityin.org/ Flourishing Science Think Tank paper: https://mindrxiv.org/4zrmd There are a few little audio problems in this weeks episode, so special shoutout to our transcript, which is here: https://docs.google.com/document/d/1zd4o40QLn0o5q1Eyp966EoDdvu5XtT6O/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Sarah is joined by Agata Bochynska and Matthew Good from the University of Oslo's Open Research Team to talk about how vital libraries are to Open Science. Links from this episode: QualiFAIR: https://www.uv.uio.no/ils/english/about/organization/tlvlab/qualifair/ TIER2: https://tier2-project.eu/ Carpentries: https://carpentries.org/ ReproducibiliTea at UiO: https://www.ub.uio.no/english/libraries/dsc/open-repro-research/reproducibilitea/index.html Preprint mentioned by Agata in the last segment: https://osf.io/kcvra/ Agata Twitter: https://twitter.com/AgataBochynska Agata mastodon: https://fediscience.org/@agata Digital Scholarship Centre website: https://www.ub.uio.no/english/libraries/dsc/ For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today, Sarah and Will discuss the invisible workload of making open science. The paper on invisible workload: https://journal.trialanderror.org/pub/the-invisible-workload/release/1 The replication crisis has led to positive structural, procedural, and community changes: https://osf.io/r6cvx/ For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Today special guest Nora Serres talks with Sarah Sauve about Registered Report and how cool Bayesian statistics are. Show notes: Appetezer paper: Bayes factor design analysis: Planning for compelling evidence https://link.springer.com/article/10.3758/s13423-017-1230-y Nora's thesis: https://www.duo.uio.no/bitstream/handle/10852/95323/5/Master-thesis_NS.pdf On registered reports: https://www.cos.io/initiatives/registered-reports https://osf.io/preprints/metaarxiv/x7aqr https://www.nature.com/srep/journal-policies/registered-reports PCI Registered Reports: https://rr.peercommunityin.org/PCIRegisteredReports/help/guide_for_authors On Bayesian/statistics: Bayesian Spectacles: https://www.bayesianspectacles.org/ Understanding Psychology as a Science by Zoltan Dienes: https://link.springer.com/book/9780230542303 For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8 The transcript is available here: https://docs.google.com/document/d/1CCk__36WxOxuuFc1JX8lou03HvGVRci4/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true

Today, Will and Jan sit down with special guest and author of the paper "A large-scale study on research code quality and execution" Ana Trisovic to talk about the reproducibility of analysis code. Also windows. Links from this episode: Ana's paper: https://www.nature.com/articles/s41597-022-01143-6 Gratitude package: https://github.com/Pakillo/grateful For more info go to ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

! Important Update ! Since recording, UCU have announced that they will "ringfence £250,000 for members facing deductions for undertaking forms of ASOS, including the marking and assessment boycott." This is reassuring news for workers at universities where the VCs have threatened 50-100% pay deductions. But still no new offers from employers. Today, Sophia and Jan are sitting down with @IlsePit(@fediscience.org) and @KatarinaWarren (k-almeidawarren.com) to chat about the ongoing UCU strikes in the UK, and how it is for ECRs to take industrial actions. "The University and College Union (UCU) represents over 120,000 academics, lecturers, trainers, instructors, researchers, managers, administrators, computer staff, librarians, technicians, professional staff and postgraduates in universities, colleges, prisons, adult education and training organisations across the UK" (https://www.ucu.org.uk/article/1685/About-UCU) You can support the UCU here: https://www.ucu.org.uk/ For more info go to https://ReproducibiliTea.org For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC8

Will, Helena and Jan talk about how we assess research and that it is weird we never really learned how to do Peer Review. Papers we discussed: Responsible Research Assessment Should Prioritize Theory Development and Testing Over Ticking Open Science Boxes: https://psyarxiv.com/ad74m/ A consensus-based tool for evaluating threats to the validity of empirical Research: https://psyarxiv.com/fc8v3 Will's blog post on peer review: https://williamngiam.github.io/blog/my_peer_review_process The tool for checking p-Values: statcheck.io The tool for checking mean values: https://journals.sagepub.com/doi/10.1177/1948550616673876 For more info go to ReproducibiliTea.org. Do you have questions? Comments? Wanna say hi? You can reach us here: https://forms.gle/H6jgUzbbpyauLxUC8

We mentioned lots of resources this week, brace yourselves! Books that changed our worlds How To Read a Paper: The Basics of Evidence-Based Medicine by Trisha Greenhalgh https://www.amazon.co.uk/How-Read-Paper-Evidence-Based-Medicine/dp/1444334360 How to Win Campaigns by Chris Rose https://www.amazon.co.uk/How-Win-Campaigns-Steps-Success/dp/1853839620 Pollution is Colonialism by Max Liboiron https://www.dukeupress.edu/pollution-is-colonialism A Student's Guide to Open Science by Charlotte Pennington https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group The Art of Statistics: Learning from Data by David Spiegelhalter https://www.penguin.co.uk/books/294857/the-art-of-statistics-by-spiegelhalter-david/9780241258767 Tools we mention: FORRT Academic Wheel of Privilege: https://www.youtube.com/watch?v=mzEdTyA06cU&ab_channel=FORRTproject Ioannidis et al. (2019): https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000384 who provide a means to adjust research impact for self-citations using a newly developed tool Baccini et al. (2019): https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0221212 calculate an “inwardness” metric which shows bias to researchers' own country Citation Diversity Statement: tools that can help authors assess the citation diversity statistics of their papers (see, for example, https://github.com/dalejn/cleanBib#instructions) Jane Lawrence Sumner's Gender Balance Assessment Tool (GBAT) https://jlsumner.shinyapps.io/syllabustool/ Groups/projects working on highlighting under-cited work: ABRIR (https://abrirpsy.org/), Marginalia (https://www.marginaliascience.com/) and FORRT (https://forrt.org/neurodiversity/) Other relevant links/resources: Evidence-Based Toxicology https://www.tandfonline.com/journals/tebt20 Evidence-Based Toxicology Collaboration https://www.ebtox.org/ Living a Feminist Life by Sara Ahmed Peer review episodes of Secret Feminist Agenda: https://secretfeministagenda.com/2020/04/07/bonus-episode-season-3-peer-review-of-secret-feminist-agenda/ and https://secretfeministagenda.com/2020/04/07/bonus-episode-response-to-review-of-season-3/ Algorithms are racist & sexist: Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. nyu Press. #MHAWS: Mirya Holman's Aggressive Winning Scholars Newsletter: https://miryaholman.substack.com/ MetaDocencia - https://www.metadocencia.org/

We are back! And we brought... a whole list of reading recommendations! Hosted and Produced by @Sarah_Sauve, @Will_Ngiam, and @VornhagenJB@hci.social Edited by Jan Vornhagen For more information visit ReproducibiliTea.org and send your feedback here: https://forms.gle/8nNLZ92YUcU1mGhc6. For a transcript, please refer to our youtube video: https://youtu.be/S7Ng6t34cSQ This weeks tasty Tea-dbit/Appe-tea-zer: Dienes' paper on democratic governance: https://royalsocietypublishing.org/doi/10.1098/rsos.220808 Things we talked about: Dienes' paper on democratic governance: https://royalsocietypublishing.org/doi/10.1098/rsos.220808 Paul Feyerabend "Against Method" Seductions of Clarity by C Thi Nguyen: https://philarchive.org/archive/NGUTSO-2 Transparency is Surveillance by C Thi Nguyen: https://onlinelibrary.wiley.com/doi/abs/10.1111/phpr.12823 Dissident Knowledge in Higher Education: https://uofrpress.ca/Books/D/Dissident-Knowledge-in-Higher-Education Aspiring to greater intellectual humility in science: https://www.nature.com/articles/s41562-021-01203-8 The questionable positionality statements paper: https://journals.sagepub.com/doi/abs/10.1177/17456916221144988 Also definitely check out Charlotte Penningtons new book "A student's guide to open science": https://forrt.org/educators-corner/014-students-guide-to-open-science/

Sophia and Jan serendipitously meet at a conference and immediately drag special guests @lonnibesancon, @lahariG, Wesley Willett into a room to talk about transparency, Open Science and Human-Computer Interaction. For more Infos about the seminar visit https://www.dagstuhl.de/en/program/calendar/semhp/?semnr=22392 Or go to ReproducibiliTea.org If you have questions or comments, throw them into the pot: forms.gle/HXHHvrNHn54MVHKb9

Today, Sam and Will have choice words about a website. For more Infos go to ReproducibiliTea.org If you have questions or comments, throw them into the pot: forms.gle/HXHHvrNHn54MVHKb9 Transcript: https://drive.google.com/file/d/1TL9lnW_HOL_BuN_wRDrWZb7e-kWUZedr/view?usp=sharing

Links from this Episode: - Paper on the limits of open data for music science: https://emusicology.org/article/view/7646 - Feminist papers on open science: https://journals.sagepub.com/doi/pdf/10.1177/03616843211030926?casa_token=x6twMszrtQgAAAAA:7Wu2Z2V0X58Lo403dJPiLT3jL65YBErd4f6FE_bhcfnie_Sg4oqax9yaMF1R02WEsB4KKNvLu7b32xs and https://journals.sagepub.com/doi/full/10.1177/03616843211036460?casa_token=ZbT7O7sn04YAAAAA:gQ3xo-t3aEh38sIVEfs5SAMa_hJwz_iyDnjuM_q6J-7k8gLn-RuWxtuu-baSkGc1QcybOCFftdAncY0 - My Shiny App: https://sarahsauve.shinyapps.io/BrandAuditDashboard/ -Simulating LMEMs: https://shiny.psy.gla.ac.uk/lmem_sim/ -Preprint of the paper I mention working on reviews for: https://psyarxiv.com/bt6zn/ -That paper covers issues of universality and ethics with citations, but here are two major ones: Decolonizing Methodologies by Linda Tuhiwai Smith and https://journals.sagepub.com/doi/10.1177/1747016117739935 -Will's shiny app: https://williamngiam.shinyapps.io/CDAPower/ -A billion-dollar donation: Estimating the Cost of Researchers' Time Spent on Peer Review by Balazs Aczel, Barnabas Szaszi, Alex O. Holcombe: https://osf.io/preprints/metaarxiv/5h9z4/ For more Infos go to ReproducibiliTea.org If you have questions or comments, throw them into the pot: https://forms.gle/HXHHvrNHn54MVHKb9

Find Kohinoor here: https://www.kohinoordarda.com/ Find all the stuff mentioned in this episode here: The talk by a former senior admin, Dr. Max Liboiron: https://www.youtube.com/watch?v=Rya5Gom5o20 The book on the not-for-profit industrial complex: https://www.dukeupress.edu/the-revolution-will-not-be-funded The organization Sarah is a part of and mentions in the episode, the Social Justice Cooperative of Newfoundland and Labrador: https://www.sjcnl.ca/ The Preprint on Open Science in India: https://osf.io/preprints/aj9gw/ Open data sharing and the global south, who benefits? https://www.science.org/doi/10.1126/science.aap8395 Psychology should generalise from - and not just to- Africa https://www.nature.com/articles/s44159-022-00070-y As usual, you can find more info on ReproducibiliTea here: ReproducibiliTea.org/ Questions? Answers? Comments? Feel free to drop them here: forms.gle/bRFcfiGQof43stoq6 Transcript: https://drive.google.com/file/d/1KyTSdjN8H9qrCWvTbpBSKcLTvt6b6MSZ/view?usp=sharing

What is the point of having a T in your name without being able to use it for a tea pun? ConTeabutorship was right there. More information on CRediT here: https://credit.niso.org/ And more info on ReproducibiliTea here: https://ReproducibiliTea.org/ Questions? Answers? Comments? Feel free to drop them here: forms.gle/bRFcfiGQof43stoq6

Today, Sarah, Will, Sam and Jan discuss what kind of community we would like to foster and how you can get involved. One of those ways, send us your questions and comments, in audio or text right here: https://forms.gle/bRFcfiGQof43stoq6 Another way: Find us on Twitter as @ReproducibiliT or send uns an E-Mail via ReproducibiliTea.org. The latter is also the best place for all infos regarding how to start your own journal club and get involved! Courtesy of Will, please find the transcript here: https://drive.google.com/file/d/1C3P0oc0tf5gje2jUyahhNPvSdBRznREr/view?usp=sharing

Sarah and Sam take a meandering discussion through slow science and what "slow" actually means in this context. Along the way, they also discuss some recommendations to promote slow science made in "Fast Lane to Slow Science" and how often scholarly critique often runs ahead of this slower pace. Links and references: Frith, U. (2020). Fast lane to slow science. Trends in Cognitive Sciences, 24(1), 1-2. Pownall https://www.bps.org.uk/psychologist/slow-science-scholarly-critique https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/ For more infos, visit ReproducibiliTea.org

Sarah and Will went/logged on to this years SIPS respectively and talk about their conference experience. It was cool! More infos www.improvingpsych.org and you can find the conference program here: https://docs.google.com/document/d/1Rka3f0ZY9BBZVIlZoVINrpyaJYeitpq1yKXBLukeMng/edit For more information about us and how to start your own ReproducibiliTea Journal club go to ReproducibiliTea.org. For a transcript check the Youtube version: https://youtu.be/FpW0n3fhx90

Today, Sophia, Sam and Jan discuss conferences for the better part of an hour only to realize they is even still more to say... So stay tuned to this becoming like a regular episode. Transcript will be added asap. Till then you can find subtitles in the youtube version of this episode:

Today @Will_Ngiam talks to @eneayegba_ and @priyasilverst about the Nowhere Lab Check out The Nowhere Lab at https://nowherelab.com/ or @theNowhereLab This episode was produced by William Ngiam and edited by Jan Vornhagen. Learn more on ReproducibiliTea.org The Transcript will be added asap. Till then, you can find subtitles in the youtube version of this episode: https://www.youtube.com/channel/UCGbbo5KrnmAu_7zbnkm2swA

Our new co-hosts @Will_Ngiam, @Sarah_Sauve, and @VornhagenJB discuss how Science needs to be better while also sharing way to much cool ressources. Transcript: https://drive.google.com/file/d/1lD0tTr-E-vv4lH8cJ9ItXedinORxnnQE/view?usp=sharing Ressources like the blogpost that caused this: https://williamngiam.github.io/Science_needs_to_be_better/ Cool stuff Sarah mentioned: Indigenous Canada on Coursera: https://www.coursera.org/learn/indigenous-canada An Indigenous Abolition Study Guide by the Yellowhead Institute: https://yellowheadinstitute.org/an-indigenous-abolitionist-study-guide/ Mapping accountability and relationships I got from Part 3 of Pollution is Colonialism by Max Liboiron: https://www.dukeupress.edu/pollution-is-colonialism Emergent Strategy by adrienne maree brown: https://adriennemareebrown.net/tag/emergent-strategy/ Revolutionary Rehearsals in the Neoliberal Age edited by Colin Barker, Gareth Dale and Neil Davidson: https://www.haymarketbooks.org/books/1653-revolutionary-rehearsals-in-the-neoliberal-age Another cool podcast: https://secretfeministagenda.com/2018/06/29/episode-2-24-being-a-little-bit-bigger-and-more-awesome-than-you-feel-most-days-with-jennifer-askey/ More reading recommendations by Will: https://www.hachettebookgroup.com/titles/damon-centola/change/9780316457330/#:~:text=As%20a%20leading%20sociologist%20and,and%20the%20diffusion%20of And the blogpost about Elsevier whose author's names Jan had forgotten: Eiko Fried and Robin Kok's (https://twitter.com/robinnkok) Blog post: https://eiko-fried.com/welcome-to-hotel-elsevier-you-can-check-out-any-time-you-like-not/ There will be a test on this next week. This episode was produced by William Ngiam and edited by Jan Vornhagen. Learn more on ReproducibiliTea.org

We are back! ReproducibiliTea 2.0 not titled #TheNewNormal. ReproducibiliTea is back with a new ambitions and rotating hosts, but first we hand the whole thing over. For hopefully not the last time brewed by @OrbenAmy, @Sam_D_Parsons and @Cruwelli Find the Transcript here: https://drive.google.com/file/d/1bTwTL6S2Ov9nlfSKdUxxEdweSM9Uwhmh/view?usp=sharing For more info visit ReproducibiliTea.org

Episode 34 - Clarissa Carneiro Early in May we talked to Clarissa (@clari_carneiro) from the Brazilian Reproducibility Initiative (@BrRepInitiative). This project is awesome - think many labs for Brazilian biomedical sciences, with a dash of meta in there too! Tune in to hear about this amazing project. Links: The project: https://www.reprodutibilidade.bio.br/home The team: https://www.reprodutibilidade.bio.br/team Partners: https://www.reprodutibilidade.bio.br/partners Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

Episode 33 with Flavio Azevedo on FORRT Our latest awesome ECR you need to watch out for is Flavio Azevedo @Flavio_Azevedo_. He was recently named one of one of the top 100 global shapers from Portugal! (https://www.100oportunidades.pt/) and by gosh he's just one of our favorite people too. Flavio tells us all about the importance of openness in reproducibility in teaching. He is leading an amazing project: a Framework for Open and Reproducible Research Training, or FORRT. The project advocates for integrating open and reproducible research practices into not only methods training, but subject material too. FORRT will be doing this with a dedicated platform collating teaching materials and the pedagogues to teach them. Listen in to hear what else FORRT has to offer, and how you can get involved! Useful links: Check out Flavio's awesome website; not only for how cool he is academically, but also some sweet photography http://flavioazevedo.com/ Flavio's recent talk about FORRT at Open Science Day at Goethe University https://video01.uni-frankfurt.de/Mediasite/Play/8fae430a4ee24d62a7f61985f74f27121d FORRT website - under development https://forrt.org/ FORRT twitter @FORRTproject The FORRT manuscript "Introducing a Framework for Open and Reproducible Research Training (FORRT)" https://osf.io/bnh7p/ Check out this twitter thread for all things FORRT https://twitter.com/FORRTproject/status/1207303102245801984?s=20 Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

Episode 32 - Research under Crisis with Anne Scheel We have a very special guest in this lockdown episode of ReproducibiliTea: Anne Scheel (@annemscheel). We reflect on research during the COVID-19 pandemic and the wide range of responses from the research community. We talked to Anne about her recent blogpost “Crisis research, fast and slow” (http://www.the100.ci/2020/03/26/crisis-research-fast-and-slow/) - her first in two years! A lot of research is being rushed to testing and to (pre)print; Anne worries that some of this goes against our collective efforts to promote ‘slow’ science. Anne masterfully flips the interview on Amy and Sam to give a teaser about their upcoming studies, and how they are trying to avoid the potential pitfalls of fast research during crisis mode. Listen through the end for some stellar ECR advice from our latest Awesome ECR Useful links: Anne’s Blogpost: Crisis research, fast and slow http://www.the100.ci/2020/03/26/crisis-research-fast-and-slow/ The 100% CI blog http://www.the100.ci/ Emergency Kittens @EmrgencyKittens Anne’s zotero library of COVID-19-related psychology preprints https://www.zotero.org/groups/2472136/covid-19_psych_papers Anne was also a guest on one of our very favourite episodes of the Everything Hertz podcast https://everythinghertz.com/47 Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

Episode 31: Class 2 - What even is a replication? It’s week 2 of Amy’s “Psychology as a Robust Science” course and we are discussing replications. What are they? Is there a distinction between direct and conceptual replications? Do direct replications actually exist? Tune in to (maybe) find out! Related papers and links Open Science Collaboration (2015) Estimating the reproducibility of psychological science https://science.sciencemag.org/content/349/6251/aac4716 Nosek and Errington (2017) Making sense of replications https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5245957/ Brian Nosek’s Queensland talk Amy mentioned (roughly between 5:00-15:00): https://www.youtube.com/watch?v=wsRmyW8GmJs Gilbert, Daniel T., Gary King, Stephen Pettigrew, and Timothy D. Wilson. ‘Comment on “Estimating the Reproducibility of Psychological Science”’. Science 351, no. 6277 (4 March 2016): 1037–1037. https://doi.org/10.1126/science.aad7243. Anderson, Christopher J., Štěpán Bahník, Michael Barnett-Cowan, Frank A. Bosco, Jesse Chandler, Christopher R. Chartier, Felix Cheung, et al. ‘Response to Comment on “Estimating the Reproducibility of Psychological Science”’. Science 351, no. 6277 (4 March 2016): 1037–1037. https://doi.org/10.1126/science.aad9163. Bishop, Dorothy V. M. ‘BishopBlog: Sowing Seeds of Doubt: How Gilbert et al’s Critique of the Reproducibility Project Has Played out’. BishopBlog (blog), 27 May 2018. http://deevybee.blogspot.com/2018/05/sowing-seeds-of-doubt-how-gilbert-et.html Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

SolidariTEA 2 - Green Advocacy With Sander van Bree SolidariTEA is our mini series where we stand in solidarity against the worse side of academia, and stand in solidarity alongside the many awesome advocates for improving academia. This time Sophia talks with Sander van Bree (@sandervanbree) about green advocacy. What actions can you take to reduce your negative impact on the environment; and what can academia do collectively? If you're still not sure on all things green advocacy after listening, hit him up on the twitters! Sander is doubly awesome because he made Sam's life super easy by sharing these useful resources: - Sander's Cusanus presentation (https://tinyurl.com/whatseffective) - A study that plots out the emissions of individual actions (link goes to best figure https://iopscience.iop.org/article/10.1088/1748-9326/aa7541/meta#erlaa7541s3). - Here's Cool Effect, which looks to be the best offsetting organization (https://www.cooleffect.org/). - Here's his Twitter, if people want to read/listen to more of his work (https://twitter.com/sandervanbree) This graph maps out how much CO2 different foods and the different steps in the supply chain cost (https://ourworldindata.org/uploads/2020/02/Environmental-impact-of-food-by-life-cycle-stage.png) Music credit: River Meditation - Jason Shaw freepd.com/misc.php

Episode 30 – What are we doing? AKA the golden pancake We are starting a new 8 part series following the structure of Amy’s “Psychology as a Robust Science” course (https://www.amyorben.com/docs/syllabus.pdf). We will take a specific aspect from each week and dive into the detail. This week: Introductions and Merton’s norms, what is Science? We discuss each of Mertons norms in turn; communalism, universalism, disinterestedness, and organised scepticism. Useful resources: Merton, Robert K. ‘The Normative Structure of Science’, 1942. https://www.panarchy.org/merton/science.html. this YouTube video for explanation: https://www.youtube.com/watch?v=00btFojQPiU&list=PLAKyhL4GNnqMVIdZDvSt3bqlHFJVRNOsF&index=2) https://www.youtube.com/watch?v=00btFojQPiU https://journals.sagepub.com/doi/pdf/10.1177/053901847601500406

Episode 29 - Four Hero Origin Stories To Start 2020 Welcome to 2020. We are back to our regular recording and releasing schedule. Sorry for the wait, and thank you for your patience. We begin with Sophia and Sam's starts to the year at the Advanced Methods for Reproducible Science workshop, aka #Repro20. All three of us have attended this workshop over the last 3 years. Last year Amy returned to talk about her year after #Repro18, and Sam returned to do the same about lessons from #Repro19. The workshop schedule is here: https://osf.io/cdrfk/ And, slides from the talks and workshops are in the repository: Check them out for some awesome materials. The workshop had 30 future leaders, aka heroes, of open and reproducible research - all coming together to learn, share experiences, and discuss how to build a bright future of research. Shout out to two projects: 1. Open Research Calendar (@OpenResearchCal)- Collecting #openresearch events in one place! (Alexandra Lautarescu, @AleLautaresc; Bradley Kennedy, @bradpsych; Cassandra Gould van Praag ,@cassgvp; Esther Plomp ,@PhDToothFAIRy) 2. Open CV - a project to help ECRs incorporate open and reproducible research practices into their CVs (Loukia Tzavella ,@LTzavella; Sam Parsons, @sam_d_parsons; Alexander Kirchner-Häusler, @KirchnerHausler; Jiv Ramduny, @JRamduny) In addition to the awesome people leading these projects, our special feature in this episode is four quick fire interviews with four of the heroes we met at the workshop - here's their origin stories: 1. Dr Robby Thibault (@rt_thibault) 2. Ms Kelly Lloyd (@KellyElizaLloyd) 3. Mr Jiv Ramduny (@JRamduny) 4. Dr Pamela Jacobsen (@pamelacjacobsen) Have no doubt, we will be hearing from these, and more, of the awesome people from the workshop. Tune in to the pod for more! Music credit: Be Jammin - Alexander Nakarada freepd.com/world.php

Episode 28 - Jade Pickering and Marta Topor This time we are joined by the amazing Jade Pickering (@Jade_Pickering) and Marta Topor (@MartaTopor). Jade and Marta share their own ReproducibiliTea experiences and an exciting new project on building tools for systematic reviews. Watch this space as the duo get ready to release this tool to the wilds! Keep an eye out for the most positive and wholesome answers to our usual guest question "what advice do you have for other ECRs?" Music credit: Be Jammin - Alexander Nakarada freepd.com/world.php

In the first of our new SolidariTea sub-series, Amy Orben talks to Robert Stenson from the University of Nottingham, UK, where he is finishing a PhD at the School of Cultures, Languages and Area Studies. SolidariTea episodes are short podcast conversations with Early Career Researchers around the world that highlight injustices, campaigns, struggles and discussions mainly about working conditions, diversity and equality, employment, and work-life balance. Robert Stenson tells us about how his university has outsourced all grad student teachers to a temporary staffing company and how this has affected their working conditions and ability to strike (see his tweet here: https://twitter.com/robert_stenson/status/1199260126319910912). The episode highlights how increasing amounts of universities are outsourcing workers (see https://unitemps.com) and how this precariousness affects grad students and early-career scientists. The practice implemented by the University of Nottingham sets a dangerous precedent for outsourcing teaching by the most junior and vulnerable workers, who are often crucial to delivering the courses departments offer to their students. Have you, or someone you know, been affected by precariousness, outsourcing or temporary contracts? Please get in touch with us! We are planning to highlight more ECR stories in future SolidariTea episodes. Music credit: River Meditation - Jason Shaw https://freepd.com/misc.php

Episode 27 - More advice on starting a PhD We return to thinking about starting a PhD. This is part 2 of our series. Sophia brought Amy and Sam some more questions and they had thoughts. If you have any thoughts, comments, and different perspectives we would love to hear them. Please reach out and tell us your thoughts and experiences. Question 4 - How much time did you spend on different PhD activities? e.g. writing papers, learning skills, reading, etc. AKA. work-work balance Question 5 - How did you manage your supervisor and build a support team? Question 6 - What was the hardest part of your PhD? Amy's blogpost: https://medium.com/@OrbenAmy/why-leaving-oxford-for-a-summer-was-the-best-decision-of-my-phd-33da4616f4af Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

Episode 26 - Advice on starting a PhD In a slight change of pace, Sophia interviews Amy and Sam for advice on starting a PhD. This is part 1 of a two part series on advice for new PhDs. If you have any thoughts or comments, hit up our twitter and join the conversation. Caveat: Amy and Sam both completed their PhDs in Experimental Psychology at Oxford University. We did try to make any advice or comments as widely relevant as possible, however we should recognise the limits of our experience. We would love to hear from you, whether your institution/department would be very different, whether you agree or disagree with our comments, and if you have anything to add we’d love to comment! Highlights Question 1 – If you could give yourself some advice at the start of yo9ur PhD, what would you say? Sams PsyPAG (@PsyPAG) blogpost: http://www.psypag.co.uk/what-advice-would-you-give-to-yourself-at-the-beginning-of-your-postgraduate-studies/ Related: check our Anne Scheel’s (@annemscheel) episode on Everything Hertz (@hertzpodcast) https://everythinghertz.com/47 Question 2 – What should you do at the start of your PhD? Question 3 – how do you, and have you been successful at, developing a good work/life balance? A part of this question we faced is, how do you define work? Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

Episode 25 - Starting A ReproducibiliTea Journal Club With Jade Pickering And Angelika Stefan Why did we start the ReproducibiliTea Journal club, and how can you start your own? Sam and Amy share their experiences before we listen in on a discussion with two very special guests. Jade Pickering (@Jade_Pickering) and Angelika Stefan (@ephemeralidea) discuss their experiences starting and running a ReproducibiliTea Journal club. You can find ours, and others', ReproducibiliTea JC materials (including a starter pack) here: https://osf.io/3qrj6/ visit reproducibilitea.org for more information on starting your own JC, and all the info on the other jcs! Music credit: Kevin MacLeod - Funkeriffic freepd.com/misc.php

Episode 24 - Conferences This time we talk academic conferences! What do we thing about them as ECRs? Amy was at the ABCD workshop during recording (check out https://abcdstudy.org/) giving a hands-on picture of conference activities. Of course, we couldn't talk conferences without mentioning our favourite SIPS!(see https://improvingpsych.org/meetings/previous-sips-meetings/). We compare conferences, discuss how we try to make the most out of our conference experience, and what conferences are like for ECRS Music credit: Be Jammin - Alexander Nakarada

Episode 23 - Nick Fox Sophia and Amy have a great chat with Nick Fox (@NickFoxstats). Nick is a Research Scientist in the Center for Open Science. Nick tells us his winding story from being a psych-hating undergrad, through biochem and behavioural neuroscience, to social psychology and meta-science (the full story is on his blog https://nickfox.netlify.com/post/trip-through-grad-school/). Nick's thesis on "Scientist as Subject: How researcher behaviors influence psychological knowledge" as awesome, and you can read it here https://psyarxiv.com/6m7cn/ Nick gives an overview of SCORE - "Systematizing Confidence in Open Research and Evidence". The project is huge! Reach out to Nick if you want to get involved! you can read more about it here https://cos.io/about/news/can-machines-determine-credibility-research-claims-center-open-science-joins-new-darpa-program-find-out/ Music credit: Be Jammin - Alexander Nakarada

Episode 22 - PsychBrief This time we chat to another awesome ECR, PsychBrief! You may have seen his very popular blog on psychologocal methods http://psychbrief.com/ or seen him dropping truth bombs on twitter @PsyBrief No, we don't reveal PsychBriefs real identity, but he does let us in on why he started and maintains his anonymous profile and his experiences in a not-so-open field. This was a great discussion, enjoy! Music credit: Be Jammin - Alexander Nakarada freepd.com/world.php

Episode 21 - Katia Damer This episode we talked to the co-founder and CEO of Prolific Academic (https://prolific.ac/), Katia Damer. Prolific Academic is a platform connecting researchers with a pool of research participants for online data collection. We discuss Katia's experience of founding Prolific as a start-up during her PhD and how prolific brings researchers and participants together. Find Katia on twitter @ekadamer, and shutout to Prolific co-founder @Phelimb Some highlights: - Katia's journey to Prolific - How we can use Prolific and whether it is the right tool for our research - How participants can join - Sam has used Prolific, it was da bomb - Critical voices about online research - Katia's start up experience, and bringing that experience back the the PhD - Dealing with failures and mistakes; how Prolific owns their mistakes. - Katia's advice for ECRs Music credit: Be Jammin - Alexander Nakarada freepd.com/world.php

Episode 20 - Priya Silverstein Buckle up to hear Priya discuss running (and publishing) replication research and diversity in science. This was perhaps the most eventful recording session we have ever had, but we really enjoyed the conversation. The more diversity the better! Music credit: Be Jammin - Alexander Nakarada freepd.com/world.php

Episode 19 - Katie Drax Introducing Katie Drax! an upcoming force in open and reproducible research! Katie is leading MAPS: Mapping the Analytical PathS of a crowdsourced data analysis. Katie also leads the Bristol branch of the ReproducibiliTea Journal Club - with much more success in cataloging the JC than we manage. Katie also has some great advice for ECRs on their open science journey. We hope you enjoy listening as much as we did. Katie's twitter: @katiedrax MAPS: https://osf.io/9qke2/ ALSPAC study: http://www.bristol.ac.uk/alspac/ Music credit: Be Jammin - Alexander Nakarada freepd.com/world.php