Biometrics Expert Discusses Data Integrity for COVID-19 Clinical Trials
Issues can include bias, result repeatability
July 1, 2020
The first remdesivir double-blind, placebo-controlled clinical trial for treating COVID-19 was published recently in Lancet.1 A member of the trial’s data safety monitoring board (DSMB) is Weichung Shih, PhD, director of the biometrics shared resource at Rutgers Cancer Institute and professor of biostatistics at Rutgers School of Public Health. Shih discussed the role of biostatisticians in protecting data integrity for clinical trials and the challenges during the COVID-19 pandemic.
Shih and colleagues quickly analyzed and transmitted data from 10 participating hospital sites to the database. They found remdesivir was not the magic bullet solution for severely ill COVID-19 patients, as was the hope. But the drug did shorten the recovery time for patients with moderately severe illness.
The following interview with IRB Advisor was conducted via email and edited for length and clarity:
IRB Advisor: What was your role in guarding data integrity of the first double-blind, placebo-controlled clinical trial of remdesivir?
Shih: I served as an expert of biostatistics in the DSMB for that trial. There were five members in the DSMB: two medical experts, one epidemiologist, and two statistical experts, all independent from the trial sponsor and investigators. The DSMB was charged and guided by the DSMB Charter to ensure the interests of patients and the objectivity of the trial sponsor. Investigators are protected, as they were masked with respect to the treatment assignments on patients throughout the study, while the DSMB may access the unblinded data all the time.
IRB Advisor: What are some risks when there are questions or issues related to data integrity in a clinical trial? What are the most effective safeguards to mitigate risks and prevent a worst-case scenario, such as the now-retracted hydroxychloroquine study findings?2
Shih: Potential issues with a clinical study include bias, lack of generalizability, and/or repeatability of the results. In general, from the design viewpoint, a randomized, double-blind clinical trial with a properly selected control treatment and sufficient number of subjects/participants from different study centers is the gold standard for investigating a new experimental therapy. From the conduct viewpoint, composing an independent DSMB is a common and useful practice to mitigate potential risks and to prevent a worst-case scenario for a clinical trial.
As I understand, the Lancet-retracted publication of hydroxychloroquine study was not a prospective clinical trial, let alone a well-designed and carefully conducted trial, but a secondary analysis of observational COVID data that were supposed to have been collected and warehoused in a huge commercial database. Usually, high-quality medical journals would require the authors certify they have full access of the data for audit when they submit their paper for publication. In the case you mentioned, it seems the authors later admitted they could not vouch for the veracity of the data sources after the paper had been published.
IRB Advisor: Researchers, public health officials, IRBs, and regulatory agencies are under a great deal of pressure to find solutions to the COVID-19 pandemic as quickly as possible. What are some of the risks, as well as regulatory and best practice safeguards, of collecting data and assessing its integrity under outside pressures?
Shih: Medical evidence has hierarchies like a pyramid. At the bottom of the lowest level of evidence is authoritative opinion — based on, perhaps, some anecdotal case report without critical appraisal or consensus. Next is retrospective, observational studies, which should be interpreted with all sorts of possible confounding factors in consideration. A higher level of evidence is individual prospective cohort studies, or clinical trials that we just mentioned. On the top of the highest level of evidence is systematic reviews or meta-analyses of relevant randomized, controlled trials. Naturally, the lower the level of evidence, the easier and faster to obtain the data, and vice versa. The lower level of evidence also is associated with higher risk of introducing bias, lack of generalizability, and repeatability. During the COVID-19 pandemic, when many research studies are being conducted, there could not be a systematic review or meta-analysis ready yet. Hence, we should evaluate individual clinical studies with vigilance as they are presented.
IRB Advisor: How can researchers, IRBs, and DSMBs help maintain public trust in the research enterprise, even when mistakes (or deceit) occur?
Shih: The researchers, IRBs, journal editors, news media, and health officials or decision-makers need to frame their judgment with an eye on the quality of data in terms of the level of evidence every time they are in touch with a clinical study for the public.
The public is most interested in the result of a study. That’s why the result is always the headline of the news. But we know the process of the getting the result is critical for the result. We should explain to the public that a single research paper rarely establishes any finding with great certainty. Nothing is firmly established until many studies have confirmed it, by different researchers, using different methods, under different settings, investigating different aspects of it. From time to time, we learn lessons when rushing research with impatience, such as the recent incidence you referred to. But retracting as soon as realizing mistake (or deceit) was made is a responsible and critical step for winning back public trust in science.
IRB Advisor: Is there anything else you could explain about your work and how it relates to human research protection and scientific integrity?
Shih: I am a faculty member of the Rutgers Biomedical Health Sciences. I teach a course on design and analysis of medical experiments at the Rutgers School of Public Health. I have authored a textbook on this subject based on many years of research and experience practiced at the Biometrics Division of the Rutgers Cancer Institute of New Jersey with my colleagues. I have served on the Scientific Review Board of Rutgers Cancer Institute of New Jersey to help ensure all studies conducted at our institution follow the standard requirements of human subject protection rules and regulations.
REFERENCES
- Wang Y, Zhang D, Du G, et al. Remdesivir in adults with severe COVID-19: A randomised, double-blind, placebo-controlled, multicentre trial. Lancet 2020;395:1569-1578.
- Mehra MR, Ruschitzka F, Patel AN. Retraction — Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: A multinational registry analysis. Lancet 2020;395:1820.
The first remdesivir double-blind, placebo-controlled clinical trial for treating COVID-19 was published recently in Lancet. A member of the trial’s data safety monitoring board is Weichung Shih, PhD. Shih discussed the role of biostatisticians in protecting data integrity for clinical trials and the challenges during the COVID-19 pandemic.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content