Does ICU Benchmarking Really Work?
Abstract & Commentary
By Uday B. Nanavaty, MD, Pulmonary and Critical Care Medicine, Rockville, Maryland, is Associate Editor for Critical Care Alert
Dr. Nanavaty reports no financial relationship to this field of study.
Synopsis: ICU benchmarking is a process whereby the outcomes of different ICUs can be compared with one another. In this study from a French ICU, authors show higher mortality rates amongst patients transferred from another ICU or another hospital’s ward, an effect that cannot be explained by disease severity or case mix.
Source: Combes A, et al. Adverse effect on a referral intensive care unit’s performance of accepting patients transferred from another intensive care unit. Crit Care Med. 2005;33:705-710.
In this analysis of an administrative dataset, Combes and colleagues wanted to find out the effect of source of admission, especially for patients whose initial care was provided at another hospital, on the outcome. They analyzed the charts of 3416 patients admitted to their tertiary referral ICU and found that 597 (17%) patients were transferred to their ICU from a non-ICU setting of another hospital (hospital transfer) and an additional 408 (12%) patients had been transferred from another ICU (ICU transfer). The ICU transfer patients, who tended to be older, had higher SAPS II score, higher organ dysfunction score, and tended to have more underlying disorders and complex medical problems. Compared to the direct admissions from their own hospital, these patients also required more emergency surgery (32% vs 11%), had a higher incidence of the acute respiratory distress syndrome (ARDS, 23% vs 7%) and severe sepsis (22% vs 10%). These patients also experienced the highest mortality (34%) as compared to hospital-transferred (23%) or direct admissions (17%). The ICU transfer patients tended to have longer length of stay (3.6 fold longer), and they had the highest standardized mortality ratio (SMR) suggesting that the SAPS II score and case mix did not predict the higher mortality that was experienced in this group as compared to the hospital-transfer group or the direct admission group.
These ICU transfer patients had a higher mortality rate even after adjusting for case mix within the ICU or after adjustment for severity of illness using SAPS II. Using logistic regression analysis, Combes et al found that transfer from another hospital and another ICU was an independent predictor of mortality. Similarly, patients who were transferred to the ICU from a ward of another hospital experienced a 1.56-fold higher mortality compared to patients who were admitted to the ICU from the ward of their hospital. When Combes et al estimated the impact of these transferred patients, especially the ICU transfer patients, they estimated that the hospital would be deemed to have had 39 excess deaths per 1000 admissions as opposed to another hospital that had no such ICU transfer patients. Combes et al found that when comparing the performance of their ICU, adjusting for disease severity or case mix did not account for the difference in the mortality for patients who were transferred from another hospital. Combes et al therefore urge caution in using disease severity and case mix as the only adjustment variables in benchmarking the performance of ICU and suggest that source of admission be used as well. Unfortunately, there is no simple way of incorporating the effect of transfer, but Combes et al suggest that perhaps, benchmarking should be done with and without these transfer patients so as to highlight the factors beyond the control of the ICU that is being analyzed.
Commentary
This study clearly demonstrates that patients transferred from another hospital—especially from another hospital’s ICU—experience higher morbidity and mortality, in spite of having seemingly similar severity of illness. The problem is not how care is provided to these patients. I believe the problem is how we measure the care process. When patients have been treated at any hospital for a substantial length of time, some of the disease severity variables may be artificially normalized, making sick patients appear not so sick. For example, APACHE II scores give the same points for an arterial PO2 of 60 mm Hg no matter what level of positive end-expiratory pressure (PEEP) was used. Also, most mortality prediction models rely upon the values obtained within the first 24 hours of admission to the ICU, whereas patients who get transferred from another ICU often are beyond their first week of illness and the reliability of prediction models in those settings has not been evaluated. It is possible that there may be a delay in appropriate therapy (such as emergency surgery) prior to transfer. The fact that a patient needs a transfer to another ICU often suggests that they either did not respond to the therapy or an appropriate therapy was not available, hence there may be a higher mortality in this group.
Benchmarking is a process whereby the performance of an ICU can be compared to a standard ICU performance. The standard is devised by initially assessing the performance of large number of ICU patients in a large number of different ICUs and, perhaps by integrating a variety of disease severity scoring systems, to estimate the outcome for a given disease category. In the United States, we have often used the APACHE II and APACHE III databases to assess the disease severity and then predict the outcome. In European countries, the SAPS II scoring system is widely used to assess disease severity and then predict outcome. In European ICUs, benchmarking is done by using the SAPS II prediction model to predict outcome and then look at actual outcome. The ratio of observed mortality to predicted mortality is called standardized mortality ratio (SMR). An ideal ICU would have SMR of 1, suggesting performance that is expected.
Unfortunately, none of the disease scoring systems work very well when applied to a single individual patient. Most have a ceiling effect where once the patient is sick, the system cannot differentiate them very well. Although a group of patients with APACHE II scores of 30 can be predicted to have certain percentage mortality, which patients will live or die remains unpredictable. The other adjustment method, the case mix in this particular article, is also woefully inadequate. For example, respiratory failure classified in this study, could be an easily-reversible negative-pressure pulmonary edema or ARDS requiring 15 cm H2O of PEEP. Similarly, a neurological disorder can be a seizure or a fatal intracranial hemorrhage.
I believe the current trend of comparing ICU performance by a single yardstick is wrong. I think each ICU should maintain a prospective data collection system and look at their outcomes closely. I firmly believe that what needs to be monitored closely are our processes rather then our outcomes. Although outcomes research is in vogue, the outcomes of biological processes are not as much in our hands as we all are led to believe. Processes are what we can control and improve upon and processes are what should be closely monitored and reported.
In this analysis of an administrative dataset, Combes and colleagues wanted to find out the effect of source of admission, especially for patients whose initial care was provided at another hospital, on the outcome.You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content