Are CR data accurate? Variables may confound
March 1, 2010 4 minutes read
Are CR data accurate? Variables may confound
'We are still at the very early stages of reporting'
Beneath the general praise Consumer Reports (CR) received for publishing hospital infection rate data and bringing the importance of infection prevention to the forefront, there is a lingering question in the mind of many a health care epidemiologist.
"A lot of people are wondering, how good are the data on which these ratings are based?" says William Schaffner, MD, chairman of the department of preventive medicine at the Vanderbilt University Medical Center in Nashville, TN.
The CR data are from hospitals that publicly report their central-line bloodstream infection rates as a result of state laws and hospitals that voluntarily report to the Leapfrog Group, a Washington, DC-based nonprofit that works with large employers and purchasers of health care to measure and publicly report on hospital quality. Since the risk of infection varies substantially across different types of ICUs, CR used a "standardized infection ratio," taking into account the unique mix of ICU types in a given hospital by comparing the hospital infection data for each ICU to the national averages for each such ICU type published by the Centers for Disease Control and Prevention.
For example, CR explained, the average infection rate for cardiac ICUs nationwide is 2 per 1,000 central-line days (the total number of days that patients are on central lines), while surgical ICUs average 2.3 infections per 1,000 central-line days. So, an infection rate 100% above average would be 4 per 1,000 days for a cardiac ICU, but 4.6 per 1,000 days for a surgical ICU.
"I don't know if the standardized infection ratio is going to be the measure that is going to be used [in the future], but it was a formula that we felt had some validity to use to compare across hospitals," explained Lisa McGiffert, director of Consumer Union's Safe Patient Project. "The way this information is typically reported in the states is pretty incomprehensible from the public's point of view. They report it by all these different kinds of ICUs, which means something to epidemiologists but is less meaningful to the public. We really needed to figure out how to bring this information together in a way that is understandable and the standardized infection ratio is the best thing we found out there."
Though conceding it is something of work in progress, McGiffert expects refinements and improvements as public infection rate reporting continues.
"We are still at the very early stages of reporting this information, and it's going to get better," she says. "Until you get it out there and try different ways to deliver the information, you're not going to learn better ways to do it."
One problem with the standardized ratio is that it relies on historical data that may not reflect recent interventions, notes William Jarvis, MD, a former leading CDC hospital outbreak investigator who is now a private epidemiology consultant for Jason and Jarvis Associates in Hilton Head, SC. In addition, any surveillance system is dependent on the rigor of its application and the parameters of infection definitions used, he observes.
"Obviously, as anyone in infection prevention knows, there are some cautions with these data," Jarvis says. "One, you've got different people applying definitions. I have been surprised that [infection definitions] I thought were clear cut were misinterpreted. So there are certainly variations between infection preventionists in applying the definitions."
Another factor to keep in mind is that a simple change in infection definition can greatly affect infection rates, which may otherwise appear to be rising and falling based on the efforts or lack thereof of the participating hospitals.
The CR report used 2008 data or the most recently available. However, future comparisons will have to acknowledge that the CDC changed central-line infection definitions beginning in January 2009, Jarvis explains.
"The most common category reported — one positive culture for coagulase-negative staph in a patient that had a central line and received antibiotics — is no longer a catheter-related BSI," he says. "So by definition, if you do absolutely nothing, your infection rate will probably decrease by at least a third."
More information about interventions and other variables such as the average duration of catheterization is needed to make meaningful comparisons, Jarvis adds.
"If your hospital has an average duration of catheterization of two days, and mine is 20 days, my patients are at much higher risk," he says. "What kind of needleless connectors are being used? There is no mention of that at all and we know from a variety of studies now [that it matters] which one of those you use, and whether you have an IV team. How many people are manipulating [the line], what's used for disinfection, and how long do you use it — all of these impact on BSI-rates."
One solution is to go beyond mere numbers and report process measures that are being used. The CR report moves in that direction by citing poor adherence to a highly regarded catheter insertion checklist, but in doing so conceded that the information is not reported in any systematic way. "We need to go beyond the absolute number [of infections] and find out more what is going on out there," Jarvis says.
Beneath the general praise Consumer Reports (CR) received for publishing hospital infection rate data and bringing the importance of infection prevention to the forefront, there is a lingering question in the mind of many a health care epidemiologist.You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content