Site level considerations in study completion speed
Expert's research highlights main issues
[Editor's note: in this Q&A story, Jeffrey James DiFrancesco, MSc, MEng, CSSBB, an assistant professor of pharmaceutical business in the College of Graduate Studies, University of the Sciences in Philadelphia, Philadelphia College of Pharmacy in Philadelphia, PA., and a partner with PrécisTrial LLC in Blue Bell, PA, discusses his research into study completion performance. DiFrancesco's co-author in the study is Harold E. Glass, PhD, professor of pharmaceutical business, University of the Sciences in Philadelphia, Philadelphia College of Pharmacy.]
CTA: Would you please provide background information about your research?
DiFrancesco: Seven companies took part in the study: Bristol-Myers Squibb, GlaxoSmithKline, J&J, Novartis, Pfizer, Sanofi-Aventis, and Wyeth. The study focused on several indications: Asthma/COPD, Depression, Schizophrenia, Bipolar disorder, Hypertension, Diabetes and Menopausal Syndrome. The global study included data from 521 phases 2 and 3 studies at 4,092 sites and were completed after 2002.
All phase 3 studies were multinational, involving the U.S., and at least one western and one eastern European country.
CTA: What do you see as the main site level considerations in study completion speed?
DiFrancesco: When we began our research, we interviewed each sponsor to gather what they subjectively believed were the site level considerations in study completion speed. One of the strongest and most consistent assertions made by the sponsors was the importance of the quality, experience, and availability of the site's clinical research coordinator, and equally as important on the sponsor's side — or contract research organization — the quality, experience, and availability of the site monitor/clinical research administrator. This effectiveness of site monitor and study coordinator was together as important as the effectiveness of the principal investigator.
Was the relationship between site monitor and/or study coordinator effective, and was study completion statistically significant? Unfortunately, we were unable to determine this. The dilemma is that these types of site data are rarely collected or retained by most sponsors during clinical trial operations. Our research did, however, provide inference to support these assertions. For example, we were able to show statistically that the higher the number of prior studies the site had conducted for the sponsor or the greater number of prior studies (1572's) the principal investigator had participated in correlated with greater site performance. Another site specific variable was country: non-US sites typically had slightly higher number of patients enrolled per site than their US counterparts.
CTA: How can clinical trial sites help eliminate the barriers to a faster completion time, and what are the actions that would have to be taken by the sponsor or another organization?
DiFrancesco: I like to answer this question operationally in two ways: first from a site planning perspective and then as an in-study site management perspective. For site planning, our research indicates sponsors need to structure site study grants to incorporate milestone-based payments, and further, structure site study enrollment goals to be competitive. These results are supportive intuitively, that market and competitive-based incentives are effective in influencing site level behavior and their resulting performance.
A counter intuitive result pertained to grant payments. Our research revealed that higher grant payments are not related to faster study completion times, although some sites may enroll faster when non-refundable start-up payments are incorporated. However, there is no clear evidence that higher payments alone insure higher quality.
Pertaining to in-study site management, our research shows simply that higher performing sites will typically have grant agreements executed earlier in the study than other sites, and or once a grant agreement is executed early, the time to first patient first visit is typically less. When you think about it, this makes sense. The earlier a site is initiated, the longer time they will have to enroll patients, provided the study's last patient first visit milestone remains fixed: more enrollment time, more patients enrolled, more site performance and faster study completion! Furthermore, these results infers that the time to grant agreement execution, as well as grant agreement to first patient first visit are statistically good early in-study indicators of site performance.
CTA: What are the results and data about phase 3 studies that you made public at the DIA conference, and is there anything new to report?
DiFrancesco: Many of the preceding conclusions are new; however, most importantly we have shown statistically significant the relative explanatory power among site's different dependent variables as compared to the site's resulting performance. We will be publishing our quantitative conclusions and the results of further analysis later this year and next year in peer reviewed journals.
CTA: As the research industry becomes more fully outsourced to international sites, do you see the clinical trial completion speed improving in the short term or long term?
DiFrancesco: Our research did not incorporate this question explicitly other than, as earlier stated, non-US sites seem to show a slightly higher number of patients enrolled per site than their US counterparts.
Subjectively, my only comment is we have to stop seeing outsourcing or internationalization of sites as the panacea for speeding study completion. I suggest that we need to think in supply-chain terms: strategically, countries and sites represent a global supplier network; tactically, patient enrollment is controlled by actively adjusting number of sites and countries; and operationally, near real-time feedback of actual performance along with the willingness for early intervention "continuous course correction" enables meeting or exceeding planned patient enrollment deadlines. It's called management!
CTA: Clinical trial experts say there appear to be no immediate solutions to the problems of inexperienced investigators and sites and their common difficulty in recruiting subjects. Do you know of any long-term solutions to this problem?
DiFrancesco: Research studies have indicated that typically 30% of sites participating in a study will enroll 70% of the eligible patients. The reasons for this high variability in site performance are numerous. Because of this essentially irreducible variability in individual site performance, we need to look at participating sites as a portfolio, and manage them as a portfolio using statistical methods.
Our research, as discussed earlier, shows that the number of prior studies a sponsor has with a site is a determinate factor, as is the experience of the principal investigator. However, when we looked at the relative weight of these types of site selection variables relative to others, the site "inexperience" is not as prominent as you might expect. For instance, site planning and site in-study management variables (as discussed previously) have a much more important role and impact, than site experience.
Pertaining to subject or patient recruitment, the short-term as well as long-term solution is to incorporate the patient recruitment strategy early into the protocol and study design. This will result in a more deterministic enrollment plans, a more targeted site selection process, and a set of effective indicators to actively manage site and country enrollment during study conduct.
CTA: Are there any other problems or trends that you would be interested in commenting on?
DiFrancesco: Numerous research studies indicate the vast majority of studies fail to achieve their planned study deadlines. Furthermore, other studies indicate over the past decades there has been little improvement in study cycle-times. I assert we have to focus more on site planning and in-study site management than site selection alone. Furthermore, earlier in site planning process we have need to be more collaborative among sponsors, clinical research organization, site management organizations, patient recruitment firms, and prospective sites for site planning and in-study site management. We have great rigor in how we design clinical trials and their endpoints, but lack this same rigor in our operational planning and in-study management of studies. For instance, we need to look for more effective means to plan and manage the statistical variance inherit when conducting studies. Site performance will always be limited in its ability to significantly improve study completion speed if the planning of sites and the in-study management of them are systemically inadequate.
CTA: Would you please provide background information about your research?You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content