The ins and outs of Pitt’s IRB monitoring program
As increasing numbers of IRBs consider establishing an auditing and monitoring program, there is at least one trailblazer from which they can learn, and that is the University of Pittsburgh. The university’s research conduct and compliance office, which oversees the IRB, established in 1997 an extensive auditing program for protocols submitted to the IRB. "At the time, it was very unique to an academic medical center to have this type of program," says Kelly Dornin-Koss, RN, BSN, CCRC, education and compliance coordinator.
Initially, the program was developed in the Office for Research for Health Sciences. Then in August 1999, the research conduct and compliance office was created to separate compliance activities from the office of research, which is responsible for generating funds for research, she reports.
Dornin-Koss is in charge of an office of four employees who conduct all of the audits of human subjects research. One of the four staff members is in charge of education and computer-based training, and another is an auditor solely for an area Veterans Affairs (VA) medical center with whom the university is sharing office space and resources, she says. "Most of the investigators who do research in the VA also are faculty at Pitt, so it serves two purposes," Dornin-Koss says.
Since the program began, the standard operating procedures (SOPs) and audit forms have been revised and expanded numerous times. The auditors attended a lecture on the subject by the Greater Pittsburgh Chapter of Clinical Research Professionals and used their sample format in the development of SOPs, Dornin-Koss says. "These were created over time, and I did audits before developing the SOPs," she explains.
As the audit and informed consent process SOPs evolved, it became clear that the process of educating investigators and others also needed to expand and improve. "I went to the Public Responsibility for Medicine in Research meeting this past fall, and one speaker said the investigators should be aware of your SOPs," Dornin-Koss recalls. "We had them internally, but I didn’t think about making them available to investigators until he said that, and so we put them on the web site, and it’s been helpful."
Here is an overview of how the auditing program works:
• Selecting protocols. Auditors query the IRB database for protocols that have received federal or internal funding, an active approval status, IRB approval within the past five years, and have an expected subject enrollment of 10 or more people. Once these protocols are identified, auditors focus on protocols with moderate to high risk, Dornin-Koss says. "For high risk, we’ll look at it with 10 or fewer subjects, but we might choose a minimal risk protocol if there are 300-400 subjects enrolled," she explains. Other criteria considered during audit protocol selection are:
• Involvement of a gene transfer intervention.
• Requirement of periodic review by federal regulations (e.g., research protocols involving approval by the radioactive drug research committee).
• Research terminated by the IRB due to failure by the investigator to submit a subject for continuation of IRB approval.
• Involvement of multiple study sites. Occasionally, the IRB executive committee will instruct the office to conduct 100% audits based on a particular criteria. "A directive from the IRB executive committee last year told us to do audits on all gene transfer protocols, and we had about 10-15 of these," Dornin-Koss says. "Basically, these are all high risk, and they don’t enroll very many subjects."
• Auditing schedule and table. The program’s goal is to conduct one audit per week, but this will be difficult to attain because the audits are very detailed and time-consuming, she explains. In preparing an audit table, coordinators will assess eligibility criteria, screening procedures, study procedures, and follow-up procedures. "We develop audit tables that are specific to each protocol, and we are trying to find a way to do audits that are not so detailed," Dornin-Koss says. "Some of these protocols have 30-40 inclusion/exclusion criteria, and we look at every one of these and every follow-up procedure."
For example, some inclusion criteria may include:
- subject is younger than 18 years of age;
- subject is HIV-negative;
- subject does not have a history of invasive fungal infection within 14 days prior to procedure X;
- subject does not have a history of allergy or intolerance to medications "A," "B," or "C."
- subject is not pregnant or breast-feeding.
"We look at every admission from 1997 forward, if the protocol was in place since 1997," she explains. "We take into consideration if there have been modifications."
Get every step of the audit in writing
• Selection and notification. One goal is to have every step of the audit in writing. Researchers are hand-delivered the audit notification that they must sign, Dornin-Koss says. Audits typically cover 20%-25% of the enrollment of a protocol that has fewer than 100 subjects; for those with 100 or more subjects, about 10% will be selected, she says. "If it’s a for-cause audit, then we do a 100% record review and a 100% informed consent review," Dornin-Koss adds. "We review the entire IRB file and look at when the protocol initially was submitted and look at the timeliness of the investigator’s responses to the IRB and if there are any lapses." When investigators are notified of the audit, they are asked to meet with the auditing coordinator within the next two weeks.
• Pre-audit interview. "We like to sit down and have a face-to-face conversation with the investigator or research coordinator or whoever they bring to this meeting," Dornin-Koss says.
At this meeting, which may last 30-60 minutes, auditing coordinators ask who is responsible for which protocol-related activity and how investigators handled subject recruitment. Other questions include:
- Do they have any training programs for their staff?
- Who is responsible for obtaining informed consent?
- What is a description of the informed consent process?
- Where are records stored?
- Who is responsible for storing records and analyzing data? (To see sample pre-audit interview form, click here.)
• Audit report. Using auditing forms, coordinators document basic information about protocols, including:
- investigator’s name;
- source of funding;
- level of risk;
- abstract of protocol;
- pre-audit interview;
- IRB correspondence;
- number of centers/sites;
- number of subjects enrolled;
"We ask for a list of all subjects enrolled to get unique identifiers and dates enrolled, and we look for any subjects who were enrolled during a lapse of IRB approval," Dornin-Koss explains.
The audit form mirrors the information that the auditors entered into the database after the audit, and includes inclusion/exclusion criteria, study procedures, adverse events, and the last section of the audit report is a description of the research records and study documentation. Altogether, the audit report could be anywhere from 10-50 pages in length, Dornin-Koss says.
• Identifying problems. "What we have found is that there is a certain level of background noise in the audits, and you’re not going to find everything just perfect," Dornin-Koss says. When problems are discovered, they are documented in the auditing reports and reviewed by the IRB executive committee, which makes the decision about how these will be handled, she reports. "They’ll ask the investigator to respond to the audit and list a corrective plan of action that they’re going to do, and it depends on how many mistakes we find throughout the audit," Dornin-Koss explains. "If we find several mistakes, the IRB executive committee might suspend enrollment."
The executive committee, consisting of the IRB chair, four IRB vice chairs, the director of research conduct and compliance, and legal counsel, also decides which problems should be reported to federal agencies, she adds.
• Reviewing consent forms. Auditors verify that the documentation of informed consent is performed according to federal policy and according to the university’s policies. Files are audited for the following documentation:
- the signature of the subject or of the subject’s legal representation;
- the signature of the investigator, who must be a physician investigator in protocols involving drug device or surgical procedures;
- witness’ signature;
- dates written adjacent to each signature, in the hand of the signatory;
- narrative documentation in the case history regarding the informed consent process;
- the research subject’s initials on each page of the informed consent document, except for the signatory page.
Auditors also will assess informed consent documents for these items:
- Presence of the IRB approval date in the right hand corner of the informed consent document, along with the renewal date and, if applicable, the date of the most recent protocol modification.
- Utilization of the current version of the IRB approved consent form.
- Consistency between the type and frequency of side effects listed in the informed consent document to those that actually occurred.
- Presence of any extemporaneous modification to the informed consent document.
- Determination that informed consent was obtained prior to the initiation of any research-related procedures.
- Implementation of all study procedures outlined in the informed consent document.
Since the auditing began, there has been considerable improvement in the area of informed consent documentation, Dornin-Koss says. "When we first were doing audits, we’d see extemporaneous modification of the consent form, and that’s something we haven’t seen now for a couple of years," she reports. For instance, in previous years, an investigator might have crossed out the part that reads, "You have four visits," and written in, "You have three visits," and had the subject initial the change. "That’s wrong because every modification to the protocol has to have prior approval from the IRB, and the only exception is to avoid an immediate hazardous research incident to subject," Dornin-Koss explains.
• Other audit goals. "What we like to see is if the investigator has copies of all IRB correspondence and copies of correspondence with the sponsor," she says. "We like to see if they have a status data safety monitoring board; we’d like to see the minutes of that," Dornin-Koss adds. "We like to see a section on training, and our university has implemented requirements, so we have our own computer-based training module that everyone involved in human subjects research is required to complete." So every protocol submitted to the IRB must be checked to see if the investigator has completed the education.
Protocols that use drugs or medical devices must have records of receiving drugs and dispensing drugs and devices. This accountability should be a standard practice, Dornin-Koss says. "We have an investigational drug service as well, and they do audits also," she adds.
• Education and training. None of the auditing forms are copyrighted, and they can be viewed by anyone who is interested in seeing how the university’s auditing program works at the web site: www.rcco.pitt.edu. For example, here is a list of the links to audit SOP forms:
- protocol selection;
- audit notification;
- selection of research subject records;
- preparation of protocol timeline;
- preparation of audit tables;
- pre-audit interview;
- review of investigator’s consent form documents;
- review of research participant records and source documentation
- review of regulatory file;
- test article accountability;
- review of electronic records;
- audit report;
- reporting of serious noncompliance.
Then there are further links to the informed consent process SOPs:
- observation of informed consent process;
- investigator interview;
- subject evaluation of informed consent process;
- subject interview;
- audit of consent form for 24-hour adverse event contact information.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content