Two eyes are better than one: Secondary reviews improve protocol review process
September 1, 2002
Some IRBs find these are worth the trouble
Whether an IRB calls it a secondary review or simply multiple primary reviews, there is a growing trend among IRBs to request that two board members take a close look at each new research proposal. With the media attention in recent years on human subjects research and the occasional accidents in which subjects have died, some IRBs have been concerned about whether their protocol review process is adequate for studies created in-house.
"We felt an extra level of protection was needed when we were authors of research," says Clayton J. Heydorn, CIP, director of human subjects administration at the New York Medical College Office of Research Administration in Valhalla, NY. "We noticed too that we had a fair amount of problems with some research protocols produced by our own faculty and that they needed a little more guidance," he adds.
The IRB recently began to use a secondary reviewer system for studies by the college’s own investigators. Now instead of having one person take a close look at each case, there are two people assigned to review the protocol before it is presented to the full board, Heydorn reports. "Basically the single reviewer system is allowed by federal regulations, and originally it was understood that everyone on the IRB would receive the full protocol and discuss it," he explains. "But that’s impractical in today’s world."
Instead, the system has evolved for most IRBs into one in which the reviewer takes the closest look at the protocol and then gives the board his or her impressions. Other members of IRBs may not have read through each protocol before the meeting.
Heydorn says the IRB discussed the idea of having a secondary reviewer assigned to each in-house protocol, and it wasn’t a hard sell. "IRB members were well aware that they had been receiving protocols that needed a lot of work before they could be approved, so having two pairs of eyes looking at them in detail is a help," he says.
The School of Dentistry IRB at the University of North Carolina at Chapel Hill also uses secondary reviewers for all full committee reviews, says Gena Everhart, IRB coordinator. "I’m an advocate for having a secondary reviewer," Everhart says. "I think it works, and it’s a good system to have."
For IRBs that object to the term "secondary reviewer" because it may imply that person’s review is less important, an alternative would be to assign more than one person to do a primary review, suggests Paul W. Goebel Jr., CIP, vice president of Chesapeake Research Review Inc. in Columbia, MD. Goebel was chair of the Food and Drug Administration’s (FDA) IRB, which used three primary reviewers. Each reviewer would take a separate portion of the project and use his or her expertise to review that section.
"The whole reason for having primary reviewers is to have a knowledgeable person spend the time it would take to thoroughly review the project and identify any areas that might require further explanation or a little polishing," Goebel says. "The way this usually works is the primary reviewer then refers these questions to IRB staff who in turn send them to the clinical investigator for comment."
Multiple reviewers yield benefits
Primary reviewers have two goals — the first is to identify problem areas and obtain further explanation. The second goal is to save time at the IRB’s meeting by having answers to these questions available, Goebel says. Goebel, Heydorn, and Everhart offer these additional suggestions about using a secondary or multiple primary reviewer process:
• Look for different expertise in reviewers. At the UNC at Chapel Hill School of Dentistry IRB the primary reviewer typically is selected because of his or her expertise in the area of the protocol, Everhart says. For the secondary reviewer, someone might be selected who is a community member of the IRB, she says. "We want a different point of view, and the community reviewer might see things that the scientist might not see," Everhart says.
The FDA’s IRB also selects reviewers from different backgrounds, Goebel says. "The FDA’s IRB also [usually] had a nonscientist perform a review of the informed consent document and see if the document will be readily understood by the average person," he says. "The FDA also had a statistician review the study and answer the question of whether the study has enough statistical power to answer the question the project is attempting to answer."
Both of these reviews provide unique perspectives that might otherwise be overlooked in a typical primary review process.
The private IRB with which Goebel currently is associated has two reviewers for protocols. One reviewer has as scientific background, and the other does not, he says. "The scientific reviewer should be qualified to evaluate the science and medical specialty area," Goebel says. "The nonscientific reviewer usually focuses on the consent document and also may read the protocol, asking questions, such as, If this was my infant, my mother, someone in my family, would I feel comfortable having them participate in this research?’"
At New York Medical College, the IRB’s primary reviewer typically is the person whose discipline best fits the protocol that’s to be reviewed, Heydorn says. While the IRB may choose a secondary reviewer who is from the same discipline, the secondary reviewer also could be someone who is at the site where the research is being conducted to address site-specific concerns, or an epidemiologist/statistician to address issues regarding the evaluation section of the protocol, or a behavioral expert to address the behavioral components of a particular study, he adds.
• Multiple reviewers provide flexibility and may improve quality. One of the primary reasons the School of Dentistry IRB uses a secondary reviewer is so that if the primary reviewer is unable to attend the meeting at which a protocol is being discussed, there will be someone else there to explain the project to the board, Everhart says. "Many of the people on our committee travel extensively, and if they are stuck somewhere and can’t get back for the meeting, we want to make sure the project is covered and does not have to be deferred," Everhart explains.
Improving review quality is another reason cited for having a secondary reviewer. "Sometimes it’s very interesting to see that one reviewer finds no problems with a study, but the other one finds a few problems," Heydorn says. "So we’re happy to see that we caught something we might not have caught otherwise," Heydorn says. "This gives us that extra comfort level of knowing that more than one pair of eyes have scrutinized the material."
Using multiple reviewers is a good way to make certain each protocol has been thoroughly reviewed and that concerns are addressed before all IRB members see it, Goebel says.
• Convince IRB members that the extra time will be worth the effort. The two-reviewer concept was an easy sell to IRB members, he recalls. "The whole purpose of the primary reviewer system is so you don’t get into a situation where the study is presented cold to the members," Goebel notes. "This way the investigator has the opportunity to think these things through ahead of time and address the concerns."
Even when two reviewers are used, an IRB typically will be able to use its own members and not have to go outside the IRB for expertise, he says. "IRB members expect to do that as part of their appointment as committee members," Goebel adds.
Time is a concern when secondary reviewers are used, particularly among people who already are very busy, Everhart acknowledges. "But I would argue that the reasons are strong enough that this needs to be done," she says. "I think the whole flow works much better when you use this system, and in the end that will benefit everyone — the institution, the researcher, and the subjects."
The School of Dentistry IRB trains IRB committee members at an orientation about the review process and the roles of primary and secondary reviewers, Everhart says. "We tell them that when they are appointed secondary reviewers, they are as responsible for a thorough review as the primary reviewer," she adds. "And they should be prepared to present that project at the meeting in the same way that the primary reviewer is presenting it."
Of course, the entire committee also should look at the protocols, but they may take about half the time as the primary and secondary reviewers, Everhart explains. Primary and secondary reviewers are expected to read through the materials, make certain everything is complete, obtain answers from investigators on all concerns, know the regulations that apply to the research, and help investigators satisfy those regulations, Everhart says. "The rest of the committee is responsible for reading the protocol, and then if they have questions at the meeting, they ask the primary and secondary reviewers who will respond in lieu of the investigator," she says.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content