Research tries to get a handle on IRB costs and variability
Smaller IRBs are less efficient, but connection of costs to quality of review still unclear
As IRBs seek to improve efficiency in their own operations, they often suffer from a lack of the most basic data about the costs of reviewing protocols.
The issues involved are not just financial — an IRB that doesn't know what resources it's using for which types of protocols can't determine where to make changes to improve efficiency while retaining quality.
But in a useful development for human subjects protection, researchers are increasingly turning their attention to this issue, seeking out hard data about how much IRBs spend on all aspects of their work.
The studies already are uncovering interesting patterns and raising questions for institutions about how best to deploy their resources to protect research participants. But researchers are still trying to work out an important piece of the puzzle: How quality fits into the picture, and how best to measure that quality.
Economies of scale
Todd H. Wagner, PhD, a health economist with the Veterans Administration Palo Alto Health Care System in California, surveyed 81 IRBs at VA and VA-affiliated hospitals across the country and found strong economies of scale in IRB operations for a study published in August 2004 in the journal Medical Care.
Wagner's group tallied the costs of operating an IRB, and the number of actions, or reviews of any kind, that the board conducted in a year.
It found that for small IRBs (those conducting up to 151 actions per year), the cost per action was significantly higher, at $2,781, than at a medium-sized IRB (up to 826 actions, at $416 per action) or a large IRB (larger than 1,637 actions, at $187 per action). Even within the group of 22 small IRBs, there was large variability of costs per action.
A more recent study found similar patterns, with lower-volume IRBs spending, on average, more per protocol than mid- or high-volume IRBs.
That study, published in the August issue of Academic Medicine, drilled down into the data even further, examining the variability within each size of IRBs, and even within the various costs of IRB operations.
Author Margaret M. Byrne, PhD, a health economist and professor at the Miller School of Medicine, University of Miami, in Florida, says that deeper analysis of the numbers yielded some surprising results about what might be driving that persistent variability.
"What we found is there is a huge variability in the ratio of staff to board time," Byrne says. She noted that at some IRBs, staff would report spending a third of the time that IRB members did reviewing protocols. At other institutions, staff reported spending 12 times as much time as IRB members performing that function.
"That tells you that IRBs across the country are doing things in very different ways," she says. "We don't know, of course, which ones are doing it correctly — most efficiently, to the highest quality. But it's kind of unlikely that everyone is doing it well in such different ways."
In addition, Byrne's group found strong economies of scale for IRB costs where they wouldn't necessarily be expected. The cost of IRB board member time spent on a protocol (based on the amount of time members spent on review, multiplied by a wage price) was substantially lower at higher-volume institutions.
"This indicates that board members at high-volume institutions are spending approximately one-third less time in reviewing each protocol than their counterparts at low-volume institutions," Byrne wrote in the journal article.
What isn't clear is the reason for these differences. Are the low-volume institutions simply less efficient? Are the high-volume IRBs too rushed in their reviews?
Byrne says that currently, we don't have enough information to know what these numbers really mean as far as the quality of IRB review.
Getting at quality
"We could have one institution that is doing things very inefficiently, but also doing a very bad quality of reviews, and it could have the same costs as an institution that's doing a very high quality of reviews but doing it very efficiently," she says.
"So even if you found one institution that's doing things great, and they had a certain cost, you couldn't just look at the cost and say everyone who's working at this cost is doing things right," Byrne says. "You have to get down and measure the quality and the efficiency of the review process, and that is extremely difficult, of course."
Byrne's study was part of a larger project by the Consortium to Examine Clinical Research Ethics, a national group established at Duke University Medical Center to compile comprehensive data about oversight of clinical trials.
She says future consortium projects will try to get a better handle on the role of quality of IRB review, and its relationship to IRB costs.
"What we're going to have to do is come up with some measures," Byrne says. "What's a measure of a good quality of review? Is it that you don't have any adverse effects? That certainly is one measure, but you wouldn't want it to be the only one."
Wagner's group tried to get at the thorny problem of determining the quality of an IRB, using three measures: whether the IRB's administrator was certified by one of several nationally recognized certification groups; estimating the percentage of the IRB chairperson's work that was compensated by the IRB; and the administrator's response to a number of graded questions about the IRB's efficiency and quality.
Wagner acknowledges that those methods have limitations, but says that other suggested measures, such as accreditation, have their own shortcomings.
"We really need IRBs to start to look inside their own systems and say, 'What's going on here?' 'How do we compare to other standards - locally, nationally?' They can look at their own system, but can they compare them to anything else to say whether it's good?"
While there's still much to know about IRB costs and their relationship to quality, Wagner says studies to date do give IRBs, particularly smaller ones, some questions to ponder about their own operations.
"For the very small IRBs in the world — those who do something less than 100 actions per year — the question is, would it be easier and better for your institution to affiliate with a larger one that's a neighbor and do it all through them?" Wagner says.
"You give up some of the control, but maybe you get a faster, better product. This really is for those (smaller IRBs), because we know that the economies of scale curve is so steep on the small side."
Wagner says medium and large operations should consider a different issue — possibly contracting out particular types of protocols that require large expenditures of resources or that are in areas that require special expertise.
He notes that many boards are beginning to contract out industry-sponsored protocols to commercial IRBs to better manage their workloads.
"If an IRB is an expert in clinical trials, then maybe they should send out health services or qualitative work because maybe the IRB members just don't know it as well," Wagner says.
Currently, though, Wagner says IRBs seeking to make these decisions are hampered by the lack of data.
"We're beginning to see protocol tracking systems that can tell you what protocol is at what stage and which ones are coming up for renewal. But none of them track it to the level of how much effort you put into a particular protocol."
He predicts that the next decade will be one of rapid change in IRB management, as institutions try to get a handle on these issues.
"I think we're going to see a sort of hold-on-to-your-seats type change, and I really hope that people keep their eye on the data — that it be evidence-based, not I-heard-it-through-the-grapevine change, but evidence-based change."
Sources
Byrne MM, et al. Variability in the Costs of Institutional Review Board Oversight. Acad Med. 2006;81:708-712.
Wagner TH, et al. Economies of Scale in Institutional Review Boards. Med Care. 2004;42:817-23.
As IRBs seek to improve efficiency in their own operations, they often suffer from a lack of the most basic data about the costs of reviewing protocols.You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content