By Sue Coons
Finding ways to evaluate IRB ethical quality and effectiveness has been an elusive ideal. Two research professionals are advocating for directly measuring quality of board oversight, rather than relying on the structure of the IRB. An upcoming U.S. Government Accountability Office (GAO) evaluation of commercial IRBs also may promote the conversation.
For many IRBs, quality measures include licensing from the Association for the Accreditation of Human Research Protection Programs, confirmation of regulatory compliance, board member expertise and training requirements, investigator and board member satisfaction, and efficient turnaround of submitted protocols, wrote Holly Fernandez Lynch, JD, MBE, and Stephen Rosenfeld, MD, MBA, in a recent issue of the Annals of Internal Medicine. “Yet it is not clear that these industry-standard approaches offer meaningful assessments of ethical quality, nor is it clear what ought to be viewed as satisfactory in this regard.”1
“As we articulate in the paper, we really don’t have a clear sense about how best to evaluate IRBs,” says Fernandez Lynch, assistant professor of medical ethics in the Perelman School of Medicine at the University of Pennsylvania. “If we had a mechanism of directly assessing their quality and effectiveness, we wouldn’t have to rely on proxy measures based on their structure. But in the absence of those measures, we have to ask if their structure might create concerns about the influence of payment, obligations to investors, or competing interests of research institutions, all of which may distract from attention to the interests of the research participants.”
Commercial IRB Structure
According to a 2016 report, more than 70% of reviews are handled by commercial IRBs. Multisite protocols are challenging and, per regulation, NIH policy sites cannot review them individually.2 “It needs to be a single IRB oversight, and there is value in the professionalism that commercial boards offer. For people who sit on IRBs at academic sites, this may be just one of their committee assignments. They are also trying to get their own research done and all of the other things that come along with their day jobs. On the for-profit boards, this is what they do. And they can get things done quickly.” The professional board members can have many more board meetings; they can participate in a board meeting a week or a couple times a week. “As a matter of efficiency, you can see why these boards are attractive compared to the academic boards.”
However, some people hold concerns about the structure. “If you are beholden to investors, then it is possible you will choose the pathway that is most efficient and legally permissible, but perhaps not ethically superior.” Fernandez Lynch says she is not suggesting that for-profit boards are violating the regulations. “They clearly are going to be following them, but the regulations are not the be-all, end-all of ethical research.” If IRBs are pressured to be responsive to investors and profit motives, are they doing the bare minimum of regulatory compliance and saying something is approvable? Or are they considering ways to make it ethically better? “How can we put an ethical gloss over all of our reviews? That’s the concern about incentives set up under the commercial model.” Fernandez Lynch emphasizes this is not about any nefarious intent, but rather the limits imposed by structural conflicts of interest.
The discussion is about the business model, not the actual board members of the commercial IRBs, Rosenfeld agrees. “This has gotten confused recently. When we talk about these companies as IRBs, the actual IRB [is comprised of] the scientists and laypeople who are convened to discuss protocols. They are not the business. They have to be kept separate,” explains Rosenfeld, chair of the U.S. Department of Health and Human Services Secretary’s Advisory Committee for Human Research Protections (SACHRP) and president at Freeport Research Systems in Freeport, ME. “The people who are in marketing and business development, and even potentially compliance, don’t sit on those boards. The people who do give their time to those boards are very well-intentioned and worry about this stuff. They are doing their jobs, but they are guided by the SOPs [standard operating procedures] and standards that are established by the companies. This [discussion] is about the companies that convene the boards and what authority and scope they allow those boards to address.”
Rosenfeld started working in the independent, for-profit IRB industry in 2008. He is a former board chair of Advarra and Quorum Review, and former president and CEO of Western IRB.
“When I started in that industry — and when that industry itself started, which was decades before that — there was real question about whether ethical review could be done for profit because the people paying for the reviews were obviously the sponsors and scientists, not the research participants,” he explains. “A consequence was that there were a lot of structural controls put in place to separate the deliberations of the actual IRB, the board of experts and representatives, from the concerns of running a business.”
Typically, the people on the board were not employees. The business convened an actual IRB panel and supported all the administrative tasks, distribution of materials, and getting things to and from the clients. “The actual deliberations of the board were kept entirely separate. It was that structure that made this whole IRB model acceptable to the community,” Rosenfeld says. A community of independent IRBs existed as well. “The combination of competitive forces and the structural concern that I raised kept our focus on research participants, which of course is what the IRB is supposed to focus on.”
Since then, the independent IRBs have been bought and consolidated into the large commercial firms. At the same time, changes in the regulations and general acceptance of the model seems to have decreased the concern about structural controls, Rosenfeld says. In addition, the regulations around IRB decisions are broad. “They leave a lot to the discretion of the committee. That is necessary because IRBs have to respond to changing science and changing social conditions,” he says.
That also can mean limited reviews that can answer the letter of the regulations but not necessarily put the interests of research participants first can be considered compliant. “These are structural issues,” Rosenfeld notes, emphasizing that he is not suggesting any bad intent in the commercial IRB industry. “But I think it is naïve to believe that these very large companies at this point will not behave in their own business interest, and that may not be completely aligned with the interests of research participants.”
Rosenfeld also is concerned about new ethical questions that have arisen in the past decade, such as privacy concerns, internet-based research, group harm, social justice, and genomics. Although the regulations are broad enough for IRBs to address these issues, companies may decide not to send them to the board for consideration. “By restricting what the IRB actually looks at, you are perversely actually adding to the burden of research as people look to other avenues to address these emerging ethical issues. The incentives for IRBs don’t support them embracing these things unless they have to.”
An Independent, Nonprofit Alternative
Rosenfeld has been advocating for the formation of an independent, nonprofit IRB as an alternative to the current models. “His idea is a nonprofit commercial IRB, which is intended to take the benefits of the academic model, particularly not being beholden to investors, and the benefits of the for-profit model, especially having extensive professional support and expert review,” Fernandez Lynch explains. “You get the best of both worlds, allowing attention to ethical considerations that might go beyond the regulatory requirements.”
Nonprofit IRBs likely will have more leeway to emphasize the ethical side of review than IRBs separated from research institutions. This might prevent the associated conflicts that come with institutional affiliations, including the need for research funding, Fernandez Lynch and Rosenfeld noted in their article. “Independent, nonprofit IRBs could take advantage of economies of scale that have made for-profit boards so efficient, as well as the professional model of membership that treats protocol review as a full-time, expert position, all while reinvesting resources in structures and processes likely to promote high-quality review,” they wrote. “Without the need to constantly grow market share, this type of board also might be positioned to serve as ‘laboratory,’ testing different approaches to research ethics oversight and sharing results to inform others.”1 If he or someone else could start this nonprofit and people responded to the idea of being more responsive to ethics, then the commercial IRBs likely would do the same as competitive businesses, Rosenfeld says.
The challenge in creating this model, Rosenfeld says, is that people are not particularly interested primarily in IRBs themselves. Instead, they are interested in scientific research and moving the needle on cures and knowledge. “This is one of the great challenges. The current structure gets IRBs out of the way of that,” he says. “In some ways, it is very responsive to the needs of the scientific community. That’s what most grantors are interested in, understandably: promoting cures and advancing knowledge.”
The idea would attract people who are primarily motivated by protecting the rights and welfare of research participants, he says. However, the current structure is not always viewed as a problem, and most people outside the research community do not know about IRBs or how they are structured. “If they do know IRBs exist, they just assume [IRBs] are doing their job,” Rosenfeld says.
The AEREO Consortium
Concerns about the shortcomings of common-sense approaches to evaluating IRB quality and effectiveness and the difficulties of learning about IRB policy, practice, and performance motivated Fernandez Lynch to launch the Consortium to Advance Effective Research Ethics Oversight (AEREO) in 2018. AEREO is a collaboration of IRB leaders, academics with expertise in research ethics and empirical methods, and other stakeholders aiming to make progress toward defining and measuring IRB effectiveness. Fernandez Lynch says AEREO is undertaking several projects, including:
- Efforts to build a set of precedents against which IRB decisions can be compared and from which IRBs can learn;
- Develop stronger understandings about what type of expertise and perspective is needed for ethical review and how to best facilitate ethical deliberation among a diverse group of decision-makers;
- Consider whether the protections imposed by IRBs are likely to achieve their goals;
- Pay attention to the views and experiences of the research participants whom IRBs aim to protect.
“These are long-term engagements that do not lend themselves to a simple quality checklist, and go well beyond questions of regulatory compliance,” she says.
Implications from the GAO
Some of the conversation may change with the GAO’s investigation into the practices of commercial IRBs at the request of U.S. Sens. Elizabeth Warren, D-MA, Sherrod Brown, D-OH, and Bernie Sanders, I-VT. (For details, see related story in this issue.) The agency has its work cut out for it, Fernandez Lynch says. “What the senators are looking for is some assessment of quality, effectiveness, and efficiency of these review boards. This is something bioethicists have been working on for decades, and we really have not made much headway on it.”
The ideal often is described as some set of metrics or a checklist, she says, with the notion that if an IRB sticks to the list, there is confidence that participants are protected. “But it’s not so easy because each board is being asked to apply ethical standards that are kind of fuzzy,” Fernandez Lynch says. “If they were clear, we wouldn’t need IRB review.”
Boards can reach conclusions that are different from one another. “There is not a clear way of saying ‘This board got it right and this board got it wrong,’ unless it was egregious where everyone would agree that a particular board got it wrong,” she explains. “These are all research ethics debates: What kind of consent do you need for standard of care research? Is a placebo control arm appropriate? Do you need to have concrete plans for post-trial access? It’s hard to say that there is one right answer, which makes these checklist-type approaches very difficult.”
The system of research ethics does not lend itself to an easy checklist of items that indicate if something was performed ethically. For example, consider the metric of how many protocols a board approves on first submission. Fernandez Lynch says that does not really say much about quality. “It could be that a board is approving 100% of protocols, but they aren’t rubber-stamping things. Instead, it could be that they have a robust system in place where the sponsors of the research and the researchers are engaged with the staff of the IRB before they even submit anything so that submitted protocols are in really great shape. They have worked out all the kinks. They are clearly resolving ethical concerns so that by the time they are actually getting to the board, the protocols are approvable. This is the IRB actually working exactly as we would had hoped in the most efficient manner of dealing with any problems before things get to the board.”
Rosenfeld says he is not sure what will come out of the investigation. “I am quite confident that at least the two big IRBs are completely compliant with the regulations. I think that has become their role.” In the GAO’s last investigation of commercial IRBs, Coast IRB in Colorado Springs approved a fraudulent study. The IRB later shut down.3 “I don’t think there is anything [like the Coast IRB issue],” he says. “They were obviously not doing the right thing. I don’t think there will be any smoking gun this time.” If complying with the regulations is the definition of the “right thing,” then commercial IRBs will point to the regulations and say they are doing what they are supposed to do, he says.
“In my view, IRBs are about protecting the rights and welfare of research participants — not demonstrating compliance with the regulations, but the regulations are the mechanism that we enforce that protection through,” he adds. Therefore, if the IRBs are compliant with the letter of the regulation, then the GAO may not find anything unexpected.
However, Rosenfeld hopes the investigation opens the door to a conversation about the purpose of IRBs. “I think one of the issues with IRBs is that the people they are meant to protect, in most cases, don’t even know they exist,” he says. “There is no real oversight or accountability to people who may one day be research participants, or to society in general. Maybe the GAO will give us an opportunity to have that conversation.”
REFERENCES
- Fernandez Lynch H, Rosenfeld S. Institutional review board quality, private equity, and promoting ethical human subjects research. Ann Intern Med 2020;173:558-562.
- Kaplan S. In clinical trials, for-profit review boards are taking over for hospitals. Should they? Stat, July 6, 2016. https://www.statnews.com/2016/07/06/institutional-review-boards-commercial-irbs/
- Koziatek S. IRB seals fate by approving fake protocol in federal sting. IRB Advisor, July 2009. https://www.reliasmedia.com/articles/113357-irb-seals-fate-by-approving-fake-protocol-in-federal-sting
Finding ways to evaluate IRB ethical quality and effectiveness has been an elusive ideal. Two research professionals are advocating for directly measuring quality of board oversight, rather than relying on the structure of the IRB. An upcoming U.S. Government Accountability Office evaluation of commercial IRBs also may promote the conversation.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content