QA assessments render positive review changes
Experts perform QA assessment
Many research institutions now have quality assurance/improvement projects that include research site audits or quality checks. But how many have thought about performing such a quality check on the IRB's work?
The institutional review board at New York University (NYU) Langone Medical Center in New York, NY, pilot-tested a project that assesses the quality of its IRB's reviews of exempt and expedited study submissions with the goal of finding problems and correcting them.
"We traditionally have been focused on our IRB's turnaround time, but what about the quality of the IRB review?" says Helen Panageas, CIP, associate director of the human research protection program at the IRB at NYU Langone Medical Center.
The IRB has five people, including Panageas who are trained to act as expedited reviewers. The other reviewers include physicians, nurses, and scientists.
"We review minimal risk research and decided it was time to look at the quality of our reviews," she says. "We wanted to see if the quality assurance findings were useful and whether these were something we could employ as part of a day-to-day process in our office."
So the IRB had experts who were not involved in the expedited and exempt reviews perform the quality assurance assessment.
"Our quality assurance manager would do external audits and actual protocols that the IRB has approved, and we decided to see if this could be part of his responsibility," Panageas says.
The IRB's education analyst also was involved.
The office developed standard operating procedures (SOPs) for the quality assurance reviews. They include the following:
1. Identify an issue: "We looked at exempt and expedited first, using the 111 criteria as our benchmark," Panageas says. "We looked at 70 projects over a six month period, assessing 50% of our volume and 50% of exempt and 50% of expedited reviews."
2. Analyze compliance: For exempt reviews, the team looked at whether or not research really met the categories, she says.
For example, an institution's research might include retrospective chart reviews where identifiers are not recorded. A project to review educational practices would not need identifiers, Panageas says.
The quality assurance reviewers would read through these and make sure they're assigned to the right category.
3. Check minimal risk: For expedited reviews, which have to meet 111 findings, the QA team could make sure these meet the clearly defined federal regulations of what minimal risk is.
"We need to make sure the expedited review fits into one of seven categories allowed for initial expedited review," Panageas says. "And those are things like surveys and focus groups with identifiers are allowed if it's minimal risk research."
4. Create a checklist:"First, we decided what to look at, and then we created a one-page checklist," Panageas says. "It's very straightforward."
They created the checklist with Excel software.
"We found when looking at the categories assigned to an expedited review that we wanted to make sure they were put in all of the right categories," Panageas says. "There could be a blood draw in the survey, and we were assigning it to the blood draw category, but not recognizing that it was also a survey."
IRBs sometimes overlook this point.
"You should check all that apply," she says. "The regulations don't spell out that, but our process involves making sure we check all of these appropriate categories."
5. Assess performance: "Once we gathered all of the information we looked at our findings and started to evaluate the percentages of what our findings essentially were," Panageas says. "We had a very high rate of making those 111 findings and being correct about minimal risk."
There typically are some areas identified that need improvement, and IRBs should focus on making changes that target their weaknesses, she adds.
6. Share results: "We've brought our findings to the attention of the reviewers and highlighted why it's important to make sure they include all of the categories for the projects," Panageas says. "What we were thinking about doing when we started the pilot project was to apply the same checklist to the full board review, measure the quality of the board's actual committee reviews."
Instead, the team came up with standards for each of the 111 criteria.
"If we say the risks to subjects are minimized, how in fact does the protocol substantiate the fact that the risk to subjects were minimized?" Panageas says. "We want benchmarks and indicators that say, Yes, the risk to subjects are minimized because of the following things.'"
7. Incorporate reviewer's checklist: "For the next phase of the project, we're incorporating the checklist they have into our project and seeing if we can add to that," Panageas says.
The IRB has addressed protocol language about minimal risk through staff education. IRB reviewers go back to investigators and say, I think you need to do a better job with this in this section,' if they didn't document the informed consent process, she adds.
"They're now able to go out on a case-by-case basis and say to the principal investigator, This is what we need from you to meet our criteria and approve this project,'" Panageas says. "Our purpose was to identify whether we were missing anything in our reviews and then help improve our analysts' reviews, helping them identify the weaknesses."
These types of quality assurance reviews could be held every three or four months. Eventually they should help improve the consistency of IRB reviews, Panageas says.
Once IRBs have conducted such a checks-and-balance of their protocol reviews, it's a good idea to meet with the full IRB to give members additional education and to share the results, she adds.
"We could target information where they are weakest," she says. "So if we find that maybe there's always a paragraph addressing subject selection, but it's not strong, then that could be an area where we target education and training for our members so they can better understand what the standards are and what to look for."
Many research institutions now have quality assurance/improvement projects that include research site audits or quality checks. But how many have thought about performing such a quality check on the IRB's work?You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content