Survey helps evaluate IRB's efficiency, performance
Provides checks and balance
How well is your IRB doing? It depends on who you ask. The subjectivity of any answer makes it a difficult question to measure.
This is why an IRB chair at the Boston University School of Medicine in Boston, MA, decided to take the direct approach and survey IRB members anonymously about their own impressions of the IRB's successes and shortcomings.
"As a chair of the IRB, it made some sense to find out what committee members felt about the committee's performance," says James A. Feldman, MD, MPH, a research director for the department of emergency medicine and an associate professor of emergency medicine at BUSM.
Feldman reviewed available literature about IRB performance measures and found little out there. While some IRBs have used trained observers of IRB meetings to measure performance, Feldman decided to simply ask board members for their opinions, under the theory that an anonymous survey would elicit more candor.
So Feldman and a co-investigator developed an anonymous survey instrument to compare several different IRBs within one institution.1
The survey tool used a 6-point Likert scale to determine whether the IRB meetings were efficient, and it also had a section for comments.1
"We took the feedback seriously, and we went through all of the qualitative comments and discussed them," Feldman says.
Researchers presented the survey's results and comments to the IRBs and encouraged discussion about any suggestions or comments that might show a need for improvement.
For example, one comment was that reviewers would present their conclusion before all IRB members had a chance to speak. The comment said this was intimidating for anyone who might disagree with the reviewer's conclusion, Feldman recalls.
"When I told IRB members about this comment, they said they were surprised because they hadn't thought about it and had only presented conclusions first to help move the meeting along," Feldman says.
Although the survey results showed that overall IRB members agreed strongly that the chair allowed all sides to be heard on important issues, the qualitative comment provided an additional perspective that would be addressed.1
Feldman's IRB has four reviewers, and it is possible to see how they might bias the committee's opinion of whether a protocol is approved or rejected, he notes.
Efficiency and fairness
The three IRBs surveyed all showed that members felt their IRB meetings were efficient, Feldman says.
"It was reassuring that people had quite positive feelings," Feldman says. "People on the boards feel we do focus on major risks to human subjects, and when we don't approve a protocol, it's based on the regulations."
Feldman sees this as the kind of response research professionals might desire from the IRB.
"You want a sense that people on the board feel that they're coming to decisions that are founded and grounded in the regulations and that they don't reflect some other factor of why someone thinks a study should not be done," Feldman says.
For instance, investigators would be more comfortable with an IRB that concentrated on the risks to human subjects and did not spend a lot of time micromanaging the informed consent form, he says.
As chair, Feldman feels responsible for keeping the IRB meetings no longer than necessary, while still covering everything that's needed.
The survey showed that IRB members were satisfied with the meetings' timeliness, and that was gratifying, he says.
"It's a tremendous, unpaid time commitment for people," Feldman notes. "To devote time to read detailed and accomplished protocols it takes a lot of time for me as the chair and for members of the committee."
Feldman created a PowerPoint presentation with the survey's results.
With this he showed how respondents strongly agreed that inviting principal investigators to meetings helped clarify some issues. Also, he showed that IRB members agreed that the educational sessions were useful, but their agreement was less enthusiastic than it was for other questions.
Here are some sample questions, answered with a 6-point range from strongly disagree to strongly agree, from the anonymous IRB survey:
- Board meetings make efficient use of my time;
- The chair allows all sides on important issues to be heard;
- The IRB is effective in its mission to protect human research subjects;
- I feel free to present my opinion to the board;
- Inviting principal investigators to board meetings has helped clarify important issues; and
- When protocols are not approved by the board, the reasons are based upon the 111 criteria.
In another section of the survey, participants are asked to describe some items according to whether they are too short, about right, or too long on a 5-point scale.
Here are some samples of those items:
- duration of meetings;
- time reviewers take to present protocols;
- amount of time spent discussing new protocols; and
- amount of time spent reviewing protocols to prepare for an IRB meeting.
Feldman presented the qualitative comments to the IRBs, as well. But he omitted some that were not relevant to a performance improvement discussion, such as a comment saying, "The chair is great," he says.
Here are some sample items from the qualitative comments:
- "Would like us to remind investigators to write study summary in terms that all can understand."
- "Reviewers should never read out loud a report that's already in system just hit the high points. Most do this."
- "We don't necessarily need to review the entire protocol for a progress report just discuss the big picture of how the study is doing."
- "Possibly an ombudsperson for PIs planning to submit new protocols."
- "I'd like us to think more about an approval that is a true final approval but excises minor changes by the investigators who subsequently report compliance."
- "More board education!"
- "Should have a standard of confirming comprehension for studies that deserve special scrutiny a description to be made by the board."
In all, the anonymous survey proved to be a useful tool for finding areas that an IRB could improve and discuss, Feldman says.
"I've received a very positive response to it, so it's well worth doing," Feldman says.
Reference
- Feldman J, Rebholz C. The use of an anonymous survey of board members to evaluate the performance of an institutional review board. Abstract presented at the 2007 Annual Public Responsibility In Medicine & Research (PRIM&R) Human Research Protection Programs Conference; Boston, MA; Dec. 1-4, 2007.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content