After an incident involving a protocol modification, the Northwell Health IRB of New Hyde Park, NY, found that the board needed certain expertise that it had been lacking.
An investigator submitted a modification that included changes to the study design. The IRB sent the modification to an expert who found a problem with the study design and thought the revised study could expose research participants to risk, recalls Hallie Kassan, MS, CIP, director of the human research protection program at Northwell Health.
The IRB suspended the study, believing it no longer met criteria for approval.1
“The study sounded fine as written, but when you studied the numbers you could see it wasn’t a proper design,” Kassan says. “We put the study on hold until the design was revised.”
That near-miss highlighted the need for the IRB to have a professionally trained eye look at studies’ statistical designs when they pose greater than minimal risk, she adds.
The IRB decided to assign a statistical IRB reviewer to all studies that were investigator-initiated and greater than minimal risk. The IRB also created a two-page statistical reviewer checklist. The reviewers can have valuable effect on the 46.111 approval criteria, specifically numbers one and two.1
Adding a statistician to the IRB seemed like a good solution. But would it cause IRB review delays?
“We were interested in whether the addition of statistical reviewers on a consistent basis would lead to additional delays,” says Jon Newlin, CIP, assistant director of human research protection program at Northwell Health.
“We looked at data where there were no scientific reviewers, and we looked at it after the change, and what we found reassured us,” Newlin says.
Adding a statistical reviewer did not increase the rate of study deferrals.1 “When studies were deferred after the change, it was for reasons that may include statistical issues, but they were not the sole reason,” Newlin says. “The studies would have been delayed, anyway.”
Finding statistical reviewers proved to be easy, as IRB chair Martin Lesser is an associate vice president of the biostatistics unit and could refer experts from his office, Kassan notes.
“Within the organization, we have a research institute, and because of that they created a biostatistics unit,” she says.
When the IRB rolled out this change, IRB leaders met with the biostatistics unit and discussed the human subjects research criteria for approval and how these apply to the biostatistics issue, Newlin explains.
“We talked about issues one and two, and built the checklist around the kinds of things that might impact one and two in the criteria,” he says. “For instance, the example we gave was having a sample size larger than needed to meet study objectives.”
Additional examples of statistical problems that could affect the criteria for approval are included in the statistical reviewer checklist. The following are some sample items:
• deficiencies in randomization, study design structure, clearly defined aims, etc. may result in the inability to achieve meaningful results;
• inappropriate or lacking power or precision analysis could decrease the likelihood of developing knowledge to the point that risks no longer are reasonable.
The statistical reviewer checklist also includes reviewer tips, such as the following:
• “If the PI refused to adopt a suggested change, would you assess that the study does not meet the criteria for approval? If not, then the recommendation should be a suggestion instead of a requirement.”
• “Will risks to subjects be reduced by the suggested change? If so, and the change will not make the study undoable, then the change is required by criterion 1.”
Since adding the role of statistical reviewer, the IRB has had positive outcomes. “We have this extra level of review with additional feedback,” Kassan says. “It will ultimately protect our participants and ensure there are better study results that come out of our studies.”
Performing a statistical review of investigator-initiated studies that are greater than minimal risk has improved the research and ensured that participants’ time on studies has value, she adds.
The IRB’s statistical reviewers undergo the same human research protection training as all IRB members, including a general orientation and CITI-training, Newlin says.
During IRB meetings, the board encourages statistical reviewers to voice their opinions during discussions of protocols.
“We ask them if they have any comments, and they are specifically given the floor,” Kassan says.
While not all of the statistical reviewers’ comments might pertain directly to criteria for approval, those that do affect the criteria can affect the study’s status.
“What’s interesting about having statistical reviewers on the IRB is they have a very unique expertise, and they are the only ones at the meeting with that expertise,” Newlin says. “So we rely on the statistical reviewer to be the expert, and with any statistical issues that come up, you have to be careful they do relate to the criteria for approval.”
Other IRB members might approve of a study, but if the statistical reviewer finds that the study’s aims do not match the statistical analysis plan, then there’s a problem noted.
“We incorporate statistical feedback in the IRB feedback letters, listed alongside all other referral issues,” Newlin explains. “Sometimes the issues are significant, and we ask investigators to make an appointment with the biostatistics department service.”
While investigators are not required to use the biostatistics service, it could be useful.
When a statistical reviewer finds a problem with a study, the IRB’s action to suspend or deny the application is based on IRB criteria for approval. “We try to explain carefully in our letters why this is a question and how we can’t find the criteria for approval to be met,” Kassan says.
REFERENCE
1. Lesser M, Newlin J, Kassan H. Using statistical reviewers on an IRB. Poster presented at PRIM&R’s 2017 Advancing Ethical Research Conference, held Nov. 5-8, 2017, in San Antonio.
After an incident involving a protocol modification, the Northwell Health IRB of New Hyde Park, NY, found that the board needed certain expertise that it had been lacking.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content