Data safety monitoring boards (DSMBs) have played a big role in delivering vaccines and treatments to the public faster in the past 18 months. DSMBs reviewed results and safety data of the various studies involving treatments and vaccines for COVID-19. IRB Advisor asked two scientists about how DSMB work has helped improve research protections during the pandemic: Weichung (Joe) Shih, PhD, director of the Biometrics Shared Resource at Rutgers Cancer Institute and professor of biostatistics at Rutgers School of Public Health, and Dirk Moore, PhD, member of the biometrics faculty at Rutgers Cancer Institute, associate professor in the department of biostatistics and epidemiology at Rutgers School of Public Health, and chair of the DSMB that oversees clinical trials conducted at Rutgers Cancer Institute. The following interview has been lightly edited for length and clarity:
IRB Advisor: Dr. Shih, can you please describe your role in guarding the data integrity of the first double-blind, placebo-controlled clinical trial of remdesivir?
Shih: I served as an expert of biostatistics on the DSMB for that trial. There were five members in the DSMB: two medical experts, one epidemiologist, and two statistical experts, all of whom were independent from the trial sponsor and investigators. The DSMB Charter charged and guided the DSMB to ensure the interests of patients and the objectivity of the trial sponsor and investigators were protected by masking them with respect to the treatment assignments on patients throughout the study. The DSMB may access the unblinded data all the time.
IRB Advisor: Data monitoring committees have been around since the 1960s, although few trials sponsored by pharmaceutical/medical device companies used DSMBs until after 2000. How important do you see the role of the DSMB in today’s research environment — especially with the COVID-19 pandemic and rapid development of vaccines and treatment?
Moore: From the outset, DSMBs have been designed to be independent reviewers of a drug, vaccine, or device. This has been interpreted in terms of members having no financial stake in the outcome, either through stock ownership or other connection to the corporation making the drug. Also, while DSMB members may be compensated for their time, that compensation may not be tied in any way to whether the drug is ultimately approved by the FDA.
[Regarding] COVID-19 vaccines, there has been the appearance of some high-ranking U.S. government officials and news media outlets injecting political considerations into the approval process for these vaccines. Political encroachment on the review process — or the appearance of encroachment — has been mitigated by the unprecedented decision of many pharmaceutical companies to make the vaccine study protocols publicly available. In that way, one can read the detailed review criteria that have been put in place in advance of the study start date. This information can reassure the public that DSMB approval decisions are dictated not by external political pressure but rather by meeting preapproved safety and efficacy goals.
IRB Advisor: What are some of the risks if questions or issues related to data integrity arise in a clinical trial? What are the most effective safeguards to mitigate risks and prevent a worst-case scenario, such as with the now-retracted hydroxychloroquine study in 2020?1
Shih: Potential issues with a clinical study include bias, lack of generalizability, and/or repeatability of the results.
In general, from the design viewpoint, a randomized, double-blind clinical trial with a properly selected control treatment and sufficient number of subjects/participants from different study centers is the gold standard for investigating a new experimental therapy. From the conduct viewpoint, composing an independent DSMB is a common and useful practice to mitigate potential risks and prevent a worst-case scenario for a clinical trial.
As I understand, the Lancet-retracted publication of hydroxychloroquine study was not a prospective clinical trial, let alone a well-designed and carefully conducted trial, but a “secondary” analysis of “observational” COVID data that was supposed to have been collected and warehoused in a huge commercial database.1 Usually, high-quality medical journals would require the authors certify that they have full access of the data for audit when they submit their paper for publication. In this case, it seems the authors later admitted they could not vouch for the veracity of the data sources after the paper had been published.
IRB Advisor: Is there anything else you could explain about your work and how it relates to human research protection and scientific integrity?
Shih: I am on the faculty of the Rutgers Biomedical Health Sciences. I teach a course on design and analysis of medical experiments at the Rutgers School of Public Health. I have authored a textbook on this subject based on many years of research and experience practiced at the Biometrics Division of the Rutgers Cancer Institute of New Jersey with my colleagues. I have served on the scientific review board of Rutgers Cancer Institute to help ensure all studies conducted at our institution follow the standard requirements of human subject protection rules and regulations.
IRB Advisor: With some AIDS research and now with COVID-19 research, we are seeing how important it is for a DSMB to monitor interim results. This can help either when unexpected adverse events arise, or if intervention works so well that it better serves public health to bring it to the affected population quickly. Would you please describe how diligently and carefully DSMBs review studies, particularly during a crisis, to ensure safety remains a priority even when there is an urgent need for a new treatment or vaccine to be distributed to the public?
Moore: DSMBs meet regularly to review the safety and, in some cases, efficacy of drugs and vaccines. For example, the Rutgers Cancer Institute, being a National Cancer Institute-designated Comprehensive Cancer Center, has a DSMB comprised of physicians and other researchers with relevant experience. It meets twice a month to review, in sequence and on a set schedule, all investigational drug trials conducted at the institute. Any adverse events that have occurred are reviewed to ensure patient safety, and any protocol deviations are reviewed and passed on to the principal investigator for remediation. Any trials that are found to not adequately balance patient risk and safety are referred to the scientific review board for review and possible closure.
IRB Advisor: With the COVID-19 pandemic, researchers, public health officials, IRBs, and regulatory agencies are under a great deal of pressure to find solutions as quickly as possible. What are some of the risks, as well as regulatory and best practice safeguards, of collecting data and assessing its integrity under these sorts of outside pressures?
Shih: Medical evidence has hierarchies like a pyramid. At the bottom of the lowest level of evidence is authoritative opinion based on, perhaps, some anecdotal case report without critical appraisal or consensus. Next is retrospective, observational studies, which should be interpreted with all sorts of possible confounding factors in consideration. A higher level of evidence is individual prospective cohort studies or clinical trials that we just mentioned. On the top of the highest level of evidence is systematic reviews or meta-analyses of relevant randomized, controlled trials. Naturally, the lower the level of evidence, the easier and faster to obtain the data, and vice versa. The lower level of evidence also is associated with higher risk of introducing bias, lack of generalizability and repeatability. During the COVID-19 pandemic, when many research studies are being conducted, there cannot be a systematic review or meta-analysis ready yet. Hence, we should evaluate individual clinical studies with vigilance as they are presented.
IRB Advisor: Will DSMBs continue monitoring the coronavirus vaccines after FDA approval? If so, how will results be made transparent to maintain public trust? If not, which group will oversee post-approval monitoring?
Moore: Even after approval, vaccines are reviewed by collecting reports of adverse outcomes. In the U.S., the Vaccine Adverse Event Reporting System (VAERS) is an early warning system for potential problems. Anyone (patients or physicians) can report suspected vaccine reactions to VAERS. The VAERS database is open to independent investigators for analysis of early- or late-term adverse reactions. This transparency can help maintain public trust. We can expect that the coronavirus vaccines will be similarly monitored.
IRB Advisor: In 1999, a young man named Jesse Gelsinger died during a gene therapy clinical trial. As a result, the FDA and other government agencies put in place more regulatory oversight to enhance protection of research participants. How do DSMBs and their independent role help prevent that kind of tragedy from recurring?
Moore: The process of ensuring patient safety in clinical trials requires a multipronged approach involving the IRB, the DSMB, and nurses and doctors treating the patient, as well as other regulatory officials. Safety measures must be established and clearly formulated into a data safety monitoring plan. The plan must include procedures for ensuring safety monitoring procedures are followed by all participants in the process, and it also must lay out requirements and procedures for rectifying any plan deviations. In this way, outcomes such as the one that befell Jesse Gelsinger may be prevented.
IRB Advisor: How can researchers, IRBs, and DSMBs help maintain public trust in the research enterprise — even when mistakes (or deceit) occur from time to time?
Shih: The researchers, IRBs, journal editors, news media, and health officials or decision-makers need to frame their judgment with an eye on the quality of data in terms of the level of evidence every time they are in touch with a public clinical study.
The public is most interested in the result of a study. That’s why the result is always the headline of the news. But we know that the process of getting the result is critical for the result. We should explain to the public that a single research paper rarely establishes any finding with great certainty. Nothing is firmly established until many studies have confirmed it, by different researchers, using different methods, under different settings, investigating different aspects of it, and more. From time to time, we learn lessons when rushing research with impatience, such as the [hydroxychloroquine study]. But retracting as soon as we realize the mistake (or deceit) was made is a responsible and necessary step for winning back public trust in science.
REFERENCE
- Mehra MR, Ruschitzka F, Patel AN. Retraction — Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: A multinational registry analysis. Lancet 2020;395:1820.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content. TESTING
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content