Clinical Trial Optimization

During the 2022 WCG MAGI Clinical Research Conferences in Boston and Las Vegas, Linda Sullivan invited attendees to stop by the CTO podcast booth and share their ideas about the top three things that the industry should do to improve how clinical trials are designed and executed. Don't miss this rare opportunity to hear ideas from investigative site staff, clinical trial participants, and other industry stakeholders about opportunities to improve clinical research.

Hannah Reimer and Tara White at Temple University joined Linda Sullivan to discuss their collaboration to develop a systematic way of including researchers in the clinical care of patients. Faced with the challenge of not having patients eligible to enroll in clinical study due to low adherence to standard of care guidelines, the research team reached out to the clinical care team to explore how to address the problem. Working collaboratively with the quality improvement group, they were able to drive changes that resulted in improved adherence to standard of care guidelines which enabled the research team to achieve study enrollment goals. The new approach of including research staff in a quality improvement project was viewed positively and has opened the door for collaborations in the future.

Patricia Roselle, global head, patient stakeholder engagement at Sanofi ,joins  Linda Sullivan to share insights about the success of an innovative model that directly and intentionally collaborates with patient advisors and advocacy groups from around the world to co-design Sanofi's end-to-end approach to medicines development.

Have you ever had a resume come across your desk that looked too good to be true? You bring the candidate in for an interview where they say all the perfect things, only to hire them and find out they were nothing like they led you to believe? This is the question explored during Linda Sullivan's interview with Kylie Sands, Carly Lovelett, and McKailey Lyndaker at St Lawrence Health. The organization revamped their job interviewing process to better learn how a potential candidate could fit into their department and help everyone succeed. The new approach that assesses “soft skills”, as well as technical skills, is helping the organization identify job candidates that have the right critical thinking skills to be successful members of their clinical research team.

Brandi Bratrude at Boston Children's Hospital and Mona Li at the Dana Farber Cancer Institute joined Linda Sullivan to discuss a successful Intern-to-FTE pipeline program at the Boston Children's Hospital. The program, made possible through a collaboration with the Office of Workforce Development at the Dana-Farber Cancer Institute, works to ensure that youth and adults from underserved and underrepresented Boston neighborhoods have the access, skills, and resources necessary to pursue healthcare careers.

Is there a way for sponsors and clinical trial sites to donate unused lab kits and supplies for humanitarian aid? This is the question explored during Linda Sullivan's interview with Mark Ryan vice president, head site management Americas region, and Donna Libretti-Cooke director of contracting and budgeting at Bayer and founder, Greg Folz and program coordinator, Josh Kravitz at Kits4Life. MedSurplus Alliance's Kits4Life program was started to provide the means for unused clinical research supplies to be repurposed by organizations around the world that are qualified to accept humanitarian aid. Bayer has successfully implemented the program across clinical trial sites in the United States and is working with Kits4Life and other sponsors to expand the program globally.  Additional information about Kits4Life is available at: https://www.medsurplusalliance.org/clinical-research-programs

How are organizations now tracking and assessing risk with risk-based quality management (RBQM) systems in the wake of new clinical trial designs and the COVDI-19 pandemic? That's one of the key questions explored during Linda Sullivan's interview with Duncan Hall, CEO and Founder at TRI — The Risk-Based Monitoring Company, which specializes in making RBQM simpler and more accessible.  Duncan, who appeared on CTO Episode #4 (August 2020), discusses how RBQM has been transformed and become more utilized in the last two decades, how the pandemic accelerated further RBQM changes, and how accompanying key risk indicator metrics have evolved as well. Duncan notes that the pandemic forced clinical trial managers to accelerate plans to implement RBQM through centralized monitoring. He also discusses how the broader adoption of DCT modalities, including the increasing use of wearables, will impact the design and implementation of risk-based monitoring.   Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com.

How can organizations use advanced analytics to complement, enhance, and accelerate current QA practices? That's one the key questions explored during Linda Sullivan's interview with Timothé Ménard, Head Quality Data Science & Bioethics Coach in Product Development Quality at F. Hoffmann-La Roche. In his role at Roche, Ménard has been leading the product development Quality Data Science Team since January 2018. From simple analytics methods to machine learning, Ménard's team is creating and implementing data-driven solutions that help understand, early detect, and predict clinical and PV quality issues. Sullivan notes that, in the last six years, many organizations have been implementing risk-based quality management approaches, based on evolving revisions to guidance such as ICH E6R2 and E8R1, which require specific data sets. According to Ménard, we should leverage and focus not just on clinical trial data but also on operational data, which is sometimes more challenging more than clinical data. In regard to identifying and tracking quality issues, Ménard points out that analytics now can be applied on program and study levels, which expands critical analysis far beyond the site level and enables biopharmaceutical companies to accelerate the drug approval process. Ménard notes that his organization has been able to glean key insights from data that might not have been available a decade ago. Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com.

What's the best way to develop world-class Quality Assurance (QA)  professionals in the clinical trials industry? That's one of the key questions explored during Linda Sullivan's interview with David Fryrear, Executive Vice President and Head of Quality Assurance at Astellas Pharma, Inc. Clinical QA was first defined as “auditing,” says Fryrear, but it has morphed into a critical element in Quality Management Systems and is continuing to transform itself today. In addition, Fryrear notes, QA is now a resource for business partners and is a key contributor and independent voice in critical decision-making in the clinical trials process. Today, Fryrear says, top-notch QA professionals must demonstrate an ability to go beyond simply being well versed in regulatory and compliance issues, but must have different competencies, including an understanding of risk management, the ability to interpret data and apply critical thinking to solve problems as well as communicate effectively with peers and business partners. Fryrear notes that Astellas has mapped out the core competencies needed and provides employees with opportunities to work closely with experienced staff to learn about and broaden their knowledge and expertise. “Quality is not the responsibility of QA – it is something that everyone needs to get involved in,” says Sullivan. Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com

In the final part of our three-part series on the value of Quality Tolerance Limits (QTLs), WCG Senior Advisor Linda Sullivan talks about methods for early detection of risk with Steve Young and Keith Dorricott. Young is the Chief Scientific Officer at CluePoints, who oversees the research and development of advanced methods for data analytics, data surveillance, and risk. Dorricott is a Master Black Belt who has spent more than 15 years applying process improvement techniques to address clinical research problems. Young and Dorricott co-led a groundbreaking, WCG Metrics Champion Consortium QTL working group, which developed a guidance on QTL emerging best practices for Consortium members. In April 2022, a three-part series summarizing those discussions and recommendations was published on the Applied Clinical Trials Journal website: https://www.appliedclinicaltrialsonline.com/view/defining-quality-tolerance-limits-and-key-risk-indicators-that-detect-risks-in-a-timely-manner-reflections-from-early-adopters-on-emerging-best-practices-part-1 The three-part podcast series is designed to be a valuable podcast companion resource to the website article. In this episode, Young, Dorricott, and Sullivan discuss how to set the threshold and secondary tolerance limits to be proactive in dealing with risks, noting that it is sometimes difficult to determine. The group also talks about when to consider using leading indicator metrics as QTL parameters and/or companion KRI metrics to provide an earlier signal of that a risk will exceed the threshold before the end of the study. The first part of the series focused on the relationship between QTLs and KRIs. The second part of the series discussed the process of defining QTLs. If you subscribe to CTO, you will automatically receive those episodes. Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com

In the second installment of our three-part series on defining and using Quality Tolerance Limits (QTLs), WCG Senior Advisor Linda Sullivan talks about the process of defining QTLs with Steve Young and Keith Dorricott. Young is the Chief Scientific Officer at CluePoints, who oversees the research and development of advanced methods for data analytics, data surveillance, and risk. Dorricott is a Master Black Belt who has spent more than 15 years applying process improvement techniques to address clinical research problems. Young and Dorricott co-led a groundbreaking, WCG Metrics Champion Consortium QTL working group, which developed a guidance on QTL emerging best practices for Consortium members. In April 2022, a three-part series summarizing those discussions and recommendations was published on the Applied Clinical Trials Journal website: https://www.appliedclinicaltrialsonline.com/view/defining-quality-tolerance-limits-and-key-risk-indicators-that-detect-risks-in-a-timely-manner-reflections-from-early-adopters-on-emerging-best-practices-part-1 The three-part podcast series is designed to be a valuable podcast companion resource to the website article. In this episode, Young, Dorricott, and Sullivan discuss what factors go into the decision of whether to establish a QTL; the importance of key potential study failure points in developing QTLs; and how to determine the number of QTLs that should be implemented. The first part of this series discussed the relationship between QTLs and KRIs and the final installment of the series will focus on methods for early detection of risk. If you subscribe to CTO, you will automatically receive those episodes. Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com

In the first of this three-part series on defining and using  Quality Tolerance Limits (QTLs), WCG Senior Advisor discusses the relationship between QTLs & KRIs  with Steve Young and Keith Dorricott. Young is the Chief Scientific Officer at CluePoints, who oversees the research and development of advanced data analytics, data surveillance, and risk methods. Dorricott is a Master Black Belt who has spent more than 15 years applying process improvement techniques to address clinical research problems. Young and Dorricott co-led a groundbreaking, MCC QTL working group, which developed a guidance on QTL emerging best practices for Consortium members. In April 2022, a three-part series summarizing those discussions and recommendations was published on the Applied Clinical Trials Journal website here. The three-part podcast series is designed to be a valuable podcast companion resource to the website article. In this episode, Young, Dorricott, and Sullivan discuss parameters and thresholds as they apply to QTL and KRIs, concluding QTLs should be considered as a designated subset of KRIs. The group also discusses the working group's mission; its process for determining best practices; the vendor perspective on QTLs; the importance of tracking risk; and the relationship of QTLs to centralized monitoring. The second part of the series will concentrate on defining QTLs and the final part of the series will focus on methods for early detection of risk. If you subscribe to CTO, you will automatically receive those episodes. Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com

What's the current status of diversity efforts in clinical trials given the recent FDA draft guidance and the introduction of the Diverse and Equitable Participation in Clinical Trials (DEPICT) in Congress? That's the main topic explored during WCG Senior Advisor's interview with Lori Abrams, Vice President, Patient Advocacy & Clinical Research Diversity, at WCG. Abrams has a long history of improving clinical drug development by including patients, caregivers, and advocacy groups into every aspect of the development continuum. In her last corporate role, Lori was Director of Diversity & Patient Engagement at Bristol-Myers Squibb (BMS) where she built one of the first clinical trial advocacy groups in R&D in pharma. Sullivan and Abrams discuss the most important diversity and patient voice developments in clinical trials, including General Considerations for Clinical Studies/Guidance for Industry, adopted by ICH in October 2021; FDA's guidance on Enhancing the Diversity of Clinical Trial Populations — Eligibility Criteria, Enrollment Practices, and Trial Designs Guidance for Industry released in Nov 2020; FDA's draft guidance on Diversity Plans to Improve Enrollment of Participants from Underrepresented Racial and Ethnic Populations in Clinical Trials Guidance for Industry released on April 13, 2022, and the DEPICT Act, which was introduced in the United States Congress on February 9, 2022. The legislation aims to boost diversity in clinical trials by requiring enhanced data reporting on clinical trial demographics and providing resources to improve access to clinical trials. Abrams also talks about how these changes have increased the urgency that sponsors now feel to implement patient diversity programs. Want to suggest a topic for CTO? Just email Linda Sullivan at lsullivan@wcgclinical.com

What are the most important lessons learned by clinical research sites from having to deal with the COVID-19 pandemic the last two years, and what are the challenges these sites face going forward?. That's the key topic explored during Senior Advisor Linda Sullivan's interview with Sandys Smith, Senior Vice President, Clinical Solutions and Strategic Partnerships for WCG. Smith, who has spent her career partnering with physicians and other healthcare stakeholders to drive customer value and clinical excellence in patient-centric oncology services, discusses how the lockdown affected clinical research site operations, and what enabled certain sites to pivot more easily. Smith discusses how telehealth and remote monitoring took off, hospitals functioned with insufficient staff, and the Great Resignation occurred. What that meant, she says, was that fewer personnel and resources were available to health care organizations and clinical sites. Smith also talks about where clinical sites are today in terms of trial backlogs, a recent uptick in new starts in March, still existing staffing shortages and the need for additional training for less experienced investigators, and how sites can be better supported. Finally, Smith talks about the advantages of adopting a hybrid, decentralized clinical trial model — a combination of onsite and virtual participant visits — that enables sites to enroll patients who don't live near clinical sites. Clinical sites are looking at how to get the work done most effectively, such as partnering with outside organizations to do budgets and/or utilizing remote staffers and deploying technology-based research solutions, according to Smith.

What's the current status of the WCG Avoca Quality Consortium  (AQC) — the exciting new collaboration between Metrics Champion Consortium (MCC) and Avoca Quality Consortium, whose mission is to elevate clinical trial quality, drive efficiency, and mitigate risk through industry collaboration by  providing a more holistic approach to improving clinical trial operations.  That's the key topic explored during Senior Advisor Linda Sullivan's interview with Michelle Webb, Vice President of WCG's AQC. The consortium is a diverse, global organization of more than 180 members that includes pharmaceutical and biotech companies, sites, and clinical service providers, who come together to achieve alignment within the industry, according to Webb. The integration of MCC with the AQC has enabled members to access information in one resource and proves that “the whole is greater than the sum of its parts,” she adds. True integration, Webb explains, has begun as members are participating in both Communities of Practice, which was traditionally MCC-based, and in Leadership Advisory Boards, which were historically AQC-based.  The main 2022 workstreams for the consortium, she notes, including innovation adoption, inspection readiness, provider qualification, risk-based quality management, site quality, safety and metrics development workgroups, all intertwine with the overall themes of expansion and inclusion.  Webb and Sullivan also discuss the importance of brainstorming in consortium meetings representing a unique approach to engaging members. Finally, Webb points out how the consortium will be keeping members current on regulatory updates and new guidance documents.

What are best strategies for utilizing the promise of machine learning, deep learning, and artificial intelligence (AI) in clinical trials? That's the key topic explored during MCC's Linda Sullivan's interview with Jonathan Rowe, Principal at ZS, where he leads the R&D, quality, and operations risk management function, and former head of clinical development, quality performance, and risk management at Pfizer. In general, Rowe notes, AI systems today are becoming increasingly useful in the industry and are providing encouraging results in developing higher quality, lower risk, protocols based upon learning from the performance of previous trials. AI is currently being developed to be able to review multiple forms of text-based clinical trial information such as audit reports, protocol deviations and monitoring visit reports. Using AI in this regard allows for the identification of issues and risks across a portfolio. Eventually AI might prove effective in people helping capture better text-based information across the trial enterprise. Well-trained AI may ultimately help determine whether particular study events were truly significant or not as well as producing improvements in report writing and enhancing risk management. It's important to train AI systems to be more sensitive to particular text writing in reports, and we shouldn't lose that capability,  asserts Rowe. “We're getting pretty good in categorizing findings,” Rowe says. “But we're not at 100 percent accuracy”, he adds. Sullivan and Rowe also discuss the pros and cons of using AI to assist in looking for trends in CAPA issues and root cause data.

How will the collaboration between Metrics Champion Consortium (MCC) and Avoca Quality Consortium (AQC) help develop new deliverables for the clinical trials industry? That's the key topic explored during MCC's Linda Sullivan's special interview with Patty Leuchten, Founder and CEO of WCG Avoca. In September 2021, WCG announced exciting news: AQC and MCC were joining forces in 2022 to form a single industry consortium. AQC and MCC are similar in having SMEs facilitate the research and development of consortium deliverables with the input of consortium members rather than having member companies do the “heavy lifting.” Leuchten and Sullivan discuss how their two organizations will be working together to integrate the extensive AQC knowledge center assets and MCC metric sets and tools. This collaboration will enable the single consortium to produce new resources for the industry to utilize in several clinical trials areas. Leuchten talks about her experience with clinical trials during the COVID pandemic, noting how it spurred innovation with new technology without incurring “undue risk.” In conclusion, Sullivan and Leuchten agree that the new consortium platform will provide an exciting resource and added value for members to explore and drive innovation in conducting clinical trials.

What were the top takeaways from the interactive metric sessions led by   Keith Dorricott at the recent 4th Annual WCTG/MCC Clinical Trial Risk and Performance Management Virtual Collaborative Summit? That's the key topic explored during Linda Sullivan's interview with Dorricott, a Master Black Belt and veteran CRO executive who serves as an expert MCC work group leader and contributor.  Dorricott discusses outcomes from several Summit breakout group sessions, including, Selection, Implementation and Use of KPIs in a Sponsor-CRO Collaboration and Using Metrics to ID Issues and Drive Improvement Activities. One key takeaway from the Summit, according to Dorricott: the importance of industry alignment on key performance questions, which can then be followed by developing and using metrics to help answer those performance questions. If you don't have key performance questions, you don't have the “grounding” necessary to get to the relevant metrics, explains Dorricott. He also described the key learnings from the very popular workshop on using metrics to drive process improvement, which provided participants with a methodology for reviewing metric dashboards and the opportunity to apply their skills to analyze a real-life case study. For more information about how to select the right metrics for your program and use metrics to improve performance, check out MCC's website, https://metricschampion.org.  

How can small sponsors develop the right strategies to deal with the challenges of aggregating vital trial data? That's one of the key topics explored during Linda Sullivan's interview with Rohit Nambisan and Todd Johnson of Lokavant. The guests discuss the difficulties of getting needed data from outside vendors, standardizing the data and implementing the necessary analysis. Some smaller sponsors, the guests explain, historically have not realized the benefits of aggregated data, especially merging historical and ongoing trial data.  That planning is far more complex that simply adding columns to a spreadsheet, and sponsors should focus their priorities on the highest possible use  cases, the guests add. Finally, the guests say, small sponsors need to consider aggregating and standardizing data (“the new oil”) across multiple studies  and CROs, and these sponsors need to negotiate access to this data in their contracts. For more on this topic, check out MCC's website,  https://metricschampion.org. While you're there, also check out the upcoming MCC Virtual Summit, at which the guests will be running an interactive session on this topic, www.mcc-summit.com

How can sponsors with limited resources be most successful in outsource RBQM? That's one of the key topics explored during Linda Sullivan's interview with Liz Wool, founder, and president of the Wool Consulting Group. Wool is a recognized industry expert on CRO-vendor oversight and management, certified CRA (ACRP), and an instructional designer, master trainer and expert in delivering virtual training courses that engages the learners.  Wool and Sullivan discuss the evolution of risk-based quality management and the regulatory challenges faced by both large and small sponsors. Small sponsors face several problems if they outsource RBQM – they often don't have the expertise to be an effective partner in RBQM, Wool notes. They need to work with a consultant who can upskill and train them to establish RBQM expectation with their CRO. Key factors in successfully outsourcing RBQM  is to work in multi-disciplinary teams, not silos, and to fully understand the methodology being used by the CRO. For more on this topic, check out MCC's website,  https://metricschampion.org. While you're there, also check out the upcoming MCC Virtual Summit, at which Liz Wool will be running a mini workshop on this topic, www.mcc-summit.com

How can organizations best use data to tier sites into groups to support “fit for purpose” risk monitoring? That's one of the key issues explored during Linda Sullivan's interview with two CTO guests: Tammy Finnigan, Chief Operating Officer and Elizabeth Robertson RBM operations consultant at Triumph Research Intelligence. Some sponsors are still running traditional monitoring models while they begin developing “fit for purpose” monitoring – an approach that deploys monitoring resources to the sites based on their risk levels, according to the two guests. Grouping sites into risk categories at the beginning of a study enables you to assign more experienced monitors with the sites most likely to need monitoring oversight. Example of site data being examined under the “fit for purpose” model include historical data, such as recruitment and retention, consent, inclusion criteria, and missing data. There are human aspects such as site staff changes that should be included in this evaluation as well, the guests note. For more on this topic, check out MCC's website,  https://metricschampion.org. While you're there, check out the upcoming MCC Virtual Summit, at which these two guests will lead a mini-workshop about this topic, www.mcc-summit.com

Is there a common set of key risk indicators (KRIs) the clinical trials industry is monitoring to achieve world-class Risk-Based Quality Management (RBQM)? That's one of the key issues explored during Linda Sullivan's interview with two returning CTO guests: Artem Andrianov, CEO of Cyntegrity and Johann Proeve, Chief Scientific Officer at Cyntegrity. Andrianov and Proeve note that they do see a set of “golden” KRIs – a subset of KRIs that many of their customers are monitoring with their RBQM system. When asked by Sullivan, Executive Director of WCG's Metric Champion Consortium, about the impact of COVID-19, both guests say that there has been huge demand for implementation and training of their RBQM platform. They specifically advise clients to begin with pilot programs when implementing RBQM. A successful strategy for illustrating the value of using a RBQM system is to conduct a retrospective analysis of a completed study to determine how early emerging issues could have been identified compared to when (or if) the issues were identified using traditional monitoring approaches. The bottom line, according to the Cyntegrity officials: RBQM is a platform that enables sponsors to identify data or process problems early on, track them proactively and avoid late surprises. And, yes, they add, it's vital that the persons reviewing the RBQM data are capable of interpreting that data and are able to implement appropriate remedial measures. For more on this topic, check out the MCC website,  https://metricschampion.org. And while you're there, check out the upcoming MCC Virtual Summit, www.mcc-summit.com

Centralized monitoring clinical trial programs can catch outlying data points in almost real time and quickly notify the clinical trial study team. That's one of the key takeaways from Crupa Kurien's interview with Linda Sullivan, MBA, Executive Director of WCG's Metric Champion Consortium. Kurien is the central monitoring manager in the RBQM program at Pfizer, who currently manages and provides oversight for the company's COVID-19 portfolio. For the most part, Kurien says, Pfizer's central monitors disseminate data analysis internally.  In addition, she notes, it is important that the signals from the central monitoring platform or dashboard are fine-tuned and broadly understand to a wide audience to make optimal use of the data analytics. Pfizer's central monitoring program runs two basic types of analyses: key-risk indicator analysis and data quality assessment, Kurien says. Central monitoring specialists ae excellent critical thinkers, she notes, trained to question best practices and communicate effectively with other team members and corporate staffers.  The pandemic accelerated the deployment of central monitoring at Pfizer, she notes, ensuring that any risks were mitigated in as close to real time as possible. Effective central monitors specialist listen to study teams, understand their protocols and work very quickly to prioritize any flagging issues. For more on this topic, check out the MCC website,  https://metricschampion.org. And while you're there, check out the upcoming MCC Virtual Summit,, www.mcc-summit.com

Sponsors should approach Risk-Based Quality Management (RBQM) implementation focusing on supporting some important critical factors, including company culture, trial design, trial operations, as well as continuous improvement. That's one of the key takeaways from Sam Sather's interview with Linda Sullivan, MBA, Executive Director of WCG's Metric Champion Consortium. Sather co-founded Clinical Pathways, a North Carolina-based consulting firm and is the company's Vice President who leads quality management consulting services. RBQM is dynamic and should be embedded in all aspects of a stakeholder's operations and decision-making, notes Sather. RBQM is not an optional activity or intermittent, she explains, and needs cross-functional representation. Finally, Sullivan and Sather take a deep dive into the differences between how small and large sponsors implement RBQM which varies based on how companies view risk-taking, what is at stake for success, access to data, outsourcing strategies, and regulatory inspection experience. Whatever you do, Sullivan and Sather conclude, document it. For more on this topic, check out the MCC website,  https://metricschampion.org. And while you're there, check out the upcoming MCC Virtual Summit, www.mcc-summit.com

With the anticipated adoption of ICH-E8-R1 this summer, there's going to be even more attention on patient voice and engagement and critical-to-quality factors. MCC's Quality by Design (QbD) - Risk Based Quality Management (RBQM) – Quality Improvement community of practice group is has already developed practical tools to help optimize quality by design. These are some of the key topics discussed in Keith Dorricott's interview with Linda Sullivan, MBA, Executive Director of WCG's Metric Champion Consortium. Dorricott is a Director at Dorricott Metrics & Process Improvement Ltd in the UK and facilitates the MCC QbD-RBQM- Quality Improvement Group.  The Cost of Poor Quality Estimator Tool enable users to estimate the financial impact of quality problems that arise during study conduct, notes Dorricott.  Other tools developed by MCC members in the quality by design area, note Sullivan and Dorricott, include the protocol development quality scoring tool and the protocol operational complexity scoring tool, which support the process of identifying potential quality problems during protocol development. Both  tools include consideration of the patient perspective, and the protocol operational complexity scoring tool calculates a patient burden measurement as well, adds Sullivan. MCC members will be reviewing the tools over the next few months to determine what updates will be needed to align with ICH-E8-R1, she notes. Finally, Sullivan says, MCC will be launching a Patient Diversity community of practice group this month to provide a place for members to consider how to assess the impact of patient engagement programs and to ensure that the “voice of the patient” is incorporated into QbD tool updates. For more on this topic, check out the MCC website,  https://metricschampion.org. And while you're there, check out the upcoming MCC Virtual Summit,, www.mcc-summit.com

How can you best engage patients in the design and execution of your clinical trials?  That’s one of the important topic discussed in Lori Abrams’s  interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Abrams is the Executive Director of Patient Advocacy and Diversity at WCG and the former Director of Diversity & Patient Engagement at Bristol-Myers Squibb (BMS).  It’s incredibly important to involve minority populations in your clinical trials to deal with health disparities in the United States, notes Abrams.  That effort brings awareness, voice, and diversity to your clinical trials, she explains. By involving the voice of patients and caregivers in your clinical trials, Abrams adds, it can help you identify new investigators and overcome obstacles to recruitment in the clinical trial process.  When you work with advocacy groups on engagement and diversity issues, make sure that the relationship is not transactional and you give them credit for the success of the project, she asserts. At BMS, Abrams started regular educational seminars that involved patients and caregiver voices, and diversity and engagement became GDO’s portion of company’s goals.  FDA recently asked a company to submit a diversity plan for a Phase III study, she notes, which illustrates that we are at a “crossroads” in the clinical trials diversity effort, she notes. For more on this topic and the upcoming launch of MCC”s Patient Diversity of Practice Group, check out the MCC website,  https://metricschampion.org. And while you’re there, check out the upcoming MCC Virtual Summit,, www.mcc-summit.com

MCC launched its groundbreaking monitoring process metrics working group last year with the purpose of developing new metrics that take into account changes in the clinical monitoring process, including risk-based assessment and centralized monitoring integration. That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  The collaborative working group started with the challenging process of developing a process map that depicts the new monitoring approach that integrates centralized monitoring with remote and onsite monitoring, Dorricott notes. The group has been utilizing the MCC metric development framework described in a previous podcast to identify critical success factors, key performance questions, and ultimately metrics that organizations can utilize to oversee and management new monitoring models. The group anticipates the first installment of metrics to be published by the end of May. Sullivan notes that MCC has knowledge portals, including those on monitoring, that members can utilize to optimize their clinical trials. For more on this topic, check out the MCC website,  https://metricschampion.org. And while you’re there, check out the upcoming MCC Virtual Summit,, www.mcc-summit.com

MCC has developed a valuable risk assessment and mitigation management tool that organizations can use as the basis for implementing the risk-based quality management process outlined in ICH E6 (R2), section 5.0. That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott facilitates MCC’s QbD-RBQM-Quality Improvement Community of Practice Group. Dorricott describes how MCC members have worked collaboratively over the last five years to develop a Risk Assessment and Mitigation Management Tool (RAMMT) that moves beyond the RACT tool that was developed by another industry group. The initial drive was from members wanting MCC metrics to be aligned with individual risks to assist with risk scoring. Sullivan points out that when ICH E6 (R2) was released, the group conducted a gap analysis to determine whether the tool fully supported the approach outlined in the guidance. The tool was updated to support a new approach that includes identifying critical processes and data and utilizing MCC process maps and key risk indicator metrics already developed by MCC metric development groups. Sullivan and Dorricott note that it’s important for risk indicators to detect risk and emerging issues as early as possible to reduce, and maybe eliminate, the negative impacts. However, Sullivan and Dorricott add, when you’re scoring your risks, you should score the ‘as-is’ situation and document any additional mitigations such as improved detection, or additional training and guidance. Otherwise, the mitigations may not actually be implemented and the risk will be higher than anticipated. Indeed, Sullivan says, documentation is key to risk assessments now being required by regulators. This topic is covered in a White Paper on the MCC website,  https://metricschampion.org. And while you’re there, check out the upcoming MCC Virtual Summit.

MCC metric toolkits include comprehensive sets of time, cost, and quality metrics defined at portfolio, study, and site levels – but how do organizations decide which metrics to include in their metric reports? That’s the important topic discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott is a Director at Dorricott Metrics & Process Improvement Ltd in the UK and facilitates many MCC metric development work groups. Dorricott describes several metric selection tool options included in MCC metric toolkits — each designed to help organizations select the right metrics for their programs. Some organizations opt to start off with metrics that are designated as “basic” metrics and add other metrics as needed. Alternatively, organizations can select metrics that answer important key performance questions and/or help to manage common process issues. Sullivan discusses an often vexing question: If a metric is always “green,” does it still need to be deployed? Answer:  It depends — because the measurements of time, cost, and quality are not always static. What you decide to measure can say a lot about your organization – it sends a signal to your employees and vendors about what is important, she points out. Be sure to consider the unintended consequences of removing a metric – does it mean that achieving a high level of performance is no longer important?

How do MCC metric development work groups determine what metrics to include in a metric toolkit? That’s one of the important topics discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott is a Director at Dorricott Metrics & Process Improvement Ltd in the UK and facilitates many MCC metric development work groups. Dorricott describes the MCC Metric Development Framework – a multi-step, structured approach MCC work groups use to develop metric toolkits. Rather than starting with metric definitions – which may end up focusing on metrics that are easy to measure rather than the right metrics, this framework focuses on the process and measuring what matters.  The starting point for this approach is to have the group to develop a process map, which helps define a glossary of terms and the aim of the process, according to Dorricott. While the high-level process map may not look exactly like the process at any given organization, it becomes the Rosetta Stone, enabling conversations with the multi-stakeholder members of the work group, adds Sullivan. Once the process map is developed, the work group collaboratively defines critical success factors – the desired outcomes of the process. Then key performance questions are defined. These are the questions that, when answered, help you determine whether a critical success factor is being achieved. Finally, the group defines metrics that provide information to answer each key performance question. There’s a tremendous amount of detail in the metric sets of the toolkit to help provide context and for implementation, which often surprises stakeholders, note Dorricott and Sullivan. In addition, they explain, toolkits contain target goals and companion metrics that can be used for further analysis, with the eye on the prize of finding root causes and improving overall clinical trials. Finally, Sullivan notes, it’s important that metric results are displayed in the most informative and user-friendly way as possible.

How are benchmarks on protocol design practices changing and what’s the impact of the exploding number data points in clinical trials? That’s one of the important issues discussed in Ken Getz’s  interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Getz is the newly appointed director and a professor at the Tufts Center for the Study of Drug Development (CSDD), Tufts University School of Medicine. Getz discusses the results of a recently released study by the Tufts CSDD, which updated previously published benchmarks while also exploring new clinical trial developments, including the collection of data points. One key takeaway from the study, according to Getz, is that when you connect longer cycle times, higher costs, and poor retention, you must arrive at one inescapable conclusion; It’s becoming more difficult to predict how any given protocol will perform and whether it will be meet expected deadlines. Another key takeaway: In typical  Phase 3 clinical studies, there are now more than 3.5 million data points, which is about three times the number of data points collected ten years ago in similar studies. A final takeaway, Getz notes, is the challenge they encountered benchmarking the impact of protocol complexity on study timelines because many companies utilize “moving plan targets”. As far as post-COVID clinical trials are concerned, the space will continue to be complex and customized and better outcomes will be achieved only if we revise standards operating procedures, according to Getz.

How can you best measure whether vendors are meeting study milestone goals? That’s one of the important issues discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott is a Director at Dorricott Metrics & Process Improvement Ltd in the UK and facilitates many MCC metric development and community of practice groups. Timeliness metrics — measurements that assess whether study milestones are achieved by their target dates —- are valuable but don’t always tell the entire story, according to Sullivan and Dorricott, such as whether the milestone dates are realistic and how on-time measurements influence human behavior. Then, when you move from measuring a single study to cross-studies or portfolio level, aggregation of data can get even more complicated. There are many ways to “cut” the data, they note, depending on the performance questions you want answered. With timeliness, Dorricott says, it always comes back to expectations — and whether those expectations are reasonable. In this respect, measuring how long it takes to achieve milestones is important and should be reviewed to determine if timeliness expectations are achievable. In terms of rolled-up data, it’s easy to aggregate results across studies but it’s important to determine which studies or process steps are not meeting expectations and need attention. When you aggregate results from a large number of studies, it can be challenging to see when you have a few studies that are not performing well. For this reason, the MCC Vendor Oversight Metric Toolkit includes several portfolio-level milestone timeliness metrics that answer difference performance questions, such as what the overall performance of the portfolio is in terms of milestones met across active studies, performance over the last 3 months and performance reported by study phase. Additional information about the MCC Vendor Oversight Community of Practice group is available on the MCC website (www.metrcischampion.org).

How can sponsors and CROs assess relationships through the use of streamlined surveys and careful review of results? That’s one of the important issues discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott is a Lean Sigma Master Black Belt with extensive experience in process improvement at CROs and is a Director at Dorricott Metrics & Process Improvement Ltd in the UK. He leads the MCC Vendor Oversight Metric Work Group which examined the topic in detail and produced a comprehensive and useful guidance document on this issue. The guidance document contains tips and tricks, from what questions to include to how to convert responses into insightful metrics and communicate results. The starting point of surveys, Dorricott says, should not be the questions themselves but the purpose of the survey and what you’re trying to achieve by running it.  In the end, Dorricott notes, the group determined that there were a handful of strategic elements to measure. They considered these strategic areas and defined five relationship assessment questions. While additional questions can be asked, smaller surveys, he says, respect the time restrictions of those completing the survey and are likely to have a higher response rate – which is important for the validity of the survey.  The MCC website, metricchampion.org, contains a White Paper that explains the guidance document, https://metricschampion.org/mcc-white-papers-index/

How can clinical trial organizations achieve TMF excellence? That’s one of the important issues discussed in Sholeh Ehdaivand’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Ehdaivand, President and CEO of LMK Clinical Research Consulting, a certified woman-owned TMF consulting company, notes that the TMF (Trial Master File) is the collection of documents and data associated with the clinical trial. Often referred to as “the story of the clinical trial,” it is designed to serve the needs of all stakeholders, including sponsors, sites, regulators, and auditors. The TMF serves as confirmation that participant rights have been protected and that the data are reliable, Ehdaivand says. While that may seem like a simple concept, she adds, not everyone understands what is actually included in the TMF. The TMF is not a “document dump,” but should be an active file containing all relevant and, especially in 2020, applicable data, according to Ehdaivand. Her company’s motto is “if it isn’t written down, it didn’t happen.” She also notes that the TMF should be compiled and organized in a timely manner and in compliance with regulations from governing authorities. She explains how the organization of the documents in the TMF is just as important as the documents themselves. Sullivan and Ehdaivand discuss LMK’s launch of TMF University in January 2021, the first and only internationally accredited TMF training program in the industry. The curriculum includes completion required assessments to ensure that students truly learned and fully understand the components of an inspection-ready TMF. The certification program is designed to elevate the importance of the TMF so that stakeholders will not just look at it as a place where “documents go to die.” There are three learning levels within the TMF University accreditation program: Discovery level, which covers the fundamental topics concerning the TMF; Immersion level, which covers essential topics that are critically important to TMF quality; and Mastery level, which covers advanced topics of TMF Management.  Completion of learning levels one and two (six courses) along with the satisfactory completion of the required assessments will earn students the internationally recognized status and credentials of Qualified TMF Associate, while the completion of all three learning levels (9 courses) and assessments will earn students the internationally recognized status and credentials of Qualified TMF Manager.  

Can metrics play a crucial oversight role in establishing an effective and efficient relationship between sponsors and CROs? That’s one of the important issues discussed in Steve Crow’s and Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Crow, Associate Director, Performance and Training, Clinical Operations at GW Pharmaceuticals (known as Greenwich Biosciences in the U.S.) explains that the growth of the company’s portfolio of studies led them to seek CRO services - which began their journey of establishing a core set of CRO metrics to assist with oversight. The organization joined MCC to learn about metrics and gain access to the metrics toolkits available to members. His advice to other sponsors in similar situations, “Don’t reinvent the wheel … MCC has the resources you need to get started”. He notes that it’s important to find a “common purpose” by focusing on the key questions you seek to answer rather than giving CROs a list of performance metrics. Opening the discussion with key performance questions that are important to your organization results in enthusiastic discussion, collaboration, and the removal of barriers, explains Crow. Keith Dorricott, a Lean Sigma Master Black Belt with extensive experience in process improvement at CROs, is now Director at Dorricott Metrics & Process Improvement Ltd in the UK, notes that metrics are only beneficial if they answer specific performance questions. Indeed, Sullivan says, that’s the basic approach MCC advocates by including key performance questions in the metric toolkits and implementation support tools available to MCC members. Finally, Dorricott and Sullivan point out, context is vital not only when selecting metrics but also paramount when reviewing results. Collaborators should remind themselves of why they’re looking at the data, what the key questions were, and what actions should be considered if performance does not meet expectations. The metrics should drive decisions and action. Otherwise, they note, they’re just looking at numbers on a dashboard.

What’s the best way to get rid of unproductive waste in clinical trials? That’s one of the important issues discussed in Keith Doricottt’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Keith Dorricott, a Lean Six?? Sigma Master Black Belt with extensive experience in process improvement at CROs, is now Director at Dorricott Metrics & Process Improvement Ltd. in England.  This sentence makes no sense. Dorricott describes the origins of the Total Cost of Quality Model in the US and its early adoption in Japan. He also discusses the basic idea that there is a balance between good and poor cost of quality – there are costs to building quality in, but much greater costs when quality does not meet the required standard. The costs of quality failure  can include protocol amendments, low enrollment, extended timelines, rework, reputational damage and delay to market. Dorricott suggests- that there be greater accountability for those responsible for certain processes rather than a simple reliance on others to find issues later and then have the cost of fixing them. Dorricott says that there are now ways to estimate the costs of poor quality, and that indeed the biggest challenge for the clinical trials industry is how to become more efficient and ultimately bring more high-quality therapeutics to market sooner to impact patients’ lives.   

Should Risk-Based Quality Management (RBQM) begin with technology or process? That’s one of the important issues discussed in Gary Tyson’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Tyson founded Pharma Initiatives eight years ago after the sale of Campbell Alliance to what is now Syneos, where he had built the clinical development and  medical affairs practice area. It’s not unusual for Tyson’s clients to engage his organization to help them select and implement RBQM software systems. But RBQM “is not a classic software implementation” project because RBQM implementation doesn’t start with technology selection. Technology is only one of the three elements of transformative change needed to implement successful RBQM programs – you should take a three-pronged approach that includes people, process, and technology, says Tyson. Organizations need to educate staff about RBQM fundamentals and establish their  RBQM process before considering RBQM software.  Since RBQM is not highly prescriptive in terms of mandates established by regulatory agencies, it is more dependent on the particular needs of the clinical trial stakeholder. And small sponsors should engage with their CROs to establish how they will work together to analyze risk and implement risk controls. In addition, says, Sullivan, your CAPA process should also be connected to RBQM. Tyson suggests to his clients that they begin RBQM with a pilot project, using affordable tools available on the market today.

How can sponsors, CROs, and sites improve the performance of clinical researchers? That’s one of the important issues discussed in SAM Sather’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Sather, Cofounder and Vice President of Clinical Pathways (CP), LLC, a consulting firm in North Carolina, leads the quality management consulting services for the company. She talks about the critical importance of human performance improvement (HPI) for the clinical trials industry, a science that is particularly significant today given the integration of quality risk management and GCP. HPI is about structuring your organization in ways that enable your employees to be star performers and engage in critical thinking, Sather says. “It’s based on doing a business analysis and defining what matters … and is very linked to metrics and what is an enforceable agreement, and [involves] the concepts of gap analysis and root cause as well.” she explains. One HPI model successfully used by Sather is Gilbert’s Behavior Engineering Model (BEM), which considers factors on two levels: environmental and individual. All factors included in these levels are equally important and need to be present for individuals to perform well. For example, organizations implementing risk-based management should consider whether it truly provides team members with the information, resources, incentives, knowledge, capacity, and motivation they need to succeed. The pandemic has pushed the industry to be agile and some stakeholders have succeeded, and others haven’t, she concludes.

How can CROs and sponsors detect potential problems and issues in clinical trial conduct and data quality? That’s one of the critical issues discussed in Steve Young’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Young, Chief Scientific Officer at CluePoint, discusses how risk-based quality management (RBQM) methodology, data analytics, and data surveillance are being utilized today to go beyond just ferreting out fraud.  Now, Young says, clinical trial stakeholders are deploying advanced statistical engines to detect quality and data breaches in investigational studies. According to Young, while there are still instances, to varying degrees, of major misconduct in clinical trials, CROs and sponsors are increasingly turning to online solutions to risk planning. It’s important, he adds, that when risks are discovered, the solutions are integrated into the workflow of clinical trials. In fact, Young asserts, one of the biggest challenges facing the industry today is to embrace change and accept the use of centralized monitoring data to aid in the discovery and resolution of risk.         Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit Sept. 8-10? Visit https://www.centerwatch.com/mcc-summit-2020. If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.

Can patients be trained to report their symptoms more accurately? That’s one of the key topics discussed in Nat Katz’s and Art Morales’ interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Katz, the Chief Science Officer of WCG-Analgesic Solutions and Morales, Vice President, Technology Solutions at WCG, assert that, with proper design and implementation, you can structure clinical trials, such as pain management studies, so that patients learn how to better report their symptoms. With proper training, patients learn to accurately report their symptoms, resulting in more reliable data, according to Katz and Morales. During COVID-19, they add, sponsors have had the opportunity to examine ways to introduce new technologies, such as telehealth, to clinical trial operations and to use more science-based tools for evaluating the performance of clinical trials. COVID-19 also has accelerated the scientific evaluation of clinical sites by optimizing the analysis of patient data. The result: Clearer and more reliable data collected during the conduct of study — without unblinding the data. Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit Sept. 8-10? Visit https://www.centerwatch.com/mcc-summit-2020. If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.

How are clinical trial companies pivoting to implement Risk-Based Quality Management programs during the pandemic? That’s one of the deep dives explored in Duncan Hall’s and Rachel Oakley’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Hall, founder and CEO of Triumph Research Intelligence (TRI) and Oakley, Vice President of RBQM Services at TRI, note that, when the COVID-19 pandemic first hit, CROs and sites performed risk assessments on their existing studies. Then those organizations took advantage of innovative technology that had a key tool for optimizing clinical trials — centralized monitoring. According to Hall and Oakley, that tool enables critical oversight and monitoring of investigational sites, even when in-person visits are not feasible. They point out that regulatory agencies are issuing new guidance during the pandemic, and that sponsors, CROs, and sites need to figure out how to comply with both those new mandates as well as existing rules. According to Hall and Oakley, there has been a renewed sense of community in the industry in dealing with the pandemic as stakeholders try to stay current through a variety of trusted sources, including MCC’s weekly COVID calls. Finally, they note, one component for optimizing clinical trials is taking center stage — training, especially in the area of RBQM. Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit Sept. 8-10? Visit https://www.centerwatch.com/mcc-summit-2020. If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.  

What’s the secret to success for implementing a world-class holistic approach to a Risk-Based Quality Management (RBQM) system? That’s one of the key topics from Artem Andrianov‘s and Johanna Proeve’s interview with Executive Director of WCG’s Metric Champion Consortium (MCC), Linda Sullivan, MBA. Andrianov, CEO, and Proeve, Chief Science Officer at Cyntegrity emphasize the importance of developing a clinical trial platform that enables your organization to manage the entire quality management cycle rather than continuing with traditional silo-based approaches. This comprehensive approach, they explain, enables pharmaceutical companies to become more efficient in reducing risk, which is especially important during the current pandemic. In addition, they note, in the COVID-19 era, more organizations have had to turn to holistic risk-based quality management systems because, for example, Excel doesn’t always work for managing RBQM, especially in larger clinical trials. Today, both CROs and sponsors are implementing innovative RBQM technology and asking for additional training to deal with data quality deterioration and other issues brought about by the onslaught of the pandemic, according to Andrianov and Proeve. Listen to this episode to find out how COVID-19 has magnified festering clinical trial shortcomings and why it’s essential that pharmaceutical company leaders become “data scientists.” Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit Sept. 8-10? Visit https://www.centerwatch.com/mcc-summit-2020. If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.  

"What impact does a remote clinical team operating model have on clinical trial performance?" That’s one of the hot topics from Ken Getz’s interview with Executive Director of WCG’s Metric Champion Consortium (MCC), Linda Sullivan, MBA. He describes several projects that the Tufts Center for the Study of Drug Development (Tufts CSDD) is currently working on; ones that will provide robust, data-driven analysis and strategic insights that help drug development professionals improve pharmaceutical R&D. Getz is the Deputy Director of Tufts CSDD and Professor at the Tufts University School of Medicine. At any given time, Getz states that Tufts is conducting between 12 and 15 grant-funded projects. The ideas for their studies are often generated from conversations in working groups, like MCC’s member networks. They also arise from thought-provoking conversations at industry conferences. But no matter where the studies come from, the studies always provide a “think tank for that very high-level, macro-level view of what’s happening in drug development,” Getz explains. Listen to this podcast to learn how Tufts – and the data it publishes – offers insights for organizations to transform clinical trials. “Everyone I speak with tells me they’ve tried to measure and benchmark. There are lots of internal forces that have tried to tackle benchmarking, so we’re excited to have an evidence-based approach that provides an opportunity to help organizations shape their practices moving forward.” Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit September 8-10? Visit https://www.centerwatch.com/mcc-summit-2020. If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.

In this episode, host Linda Sullivan, MBA, Executive Director of WCG's Metric Champion Consortium (MCC) chats with Laurie Halloran, founder of the Halloran Consulting Group, about how she developed an on-demand drug development team that can fill in gaps in the clinical trial process for life science companies, whether these involve regulatory strategy or execution, or quality, clinical, or manufacturing issues. The COVID-19 pandemic has resulted in these companies having to pivot to more virtual experiences. At the beginning of the outbreak, she notes, her group organized town halls that made it clear to stakeholders that life science companies had to implement telehealth and other remote monitoring solutions to protect patient safety. As a result of the pandemic, Halloran explains, her group is reviewing clinical trial protocols to determine how technology can be better utilized in the collection of clinical trial data. In conclusion, Halloran says, “don’t waste this opportunity” to make your clinical trial program more resilient. Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit September 8-10? Visit https://www.centerwatch.com/mcc-summit-2020 If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.