The American Hospital Association in Washington, DC, is sending a stern message to the CMS: Your Hospital Quality Star Rating System isn’t working so well.
AHA Acting Senior Executive for Policy Ashley Thompson recently sent CMS a letter saying the Star Rating system must be improved to make the ratings more credible and relevant to improving the patient experience. She notes that hospitals were instrumental in the creation of Hospital Compare more than a decade ago, and she says they remain committed to sharing meaningful, accurate hospital quality information.
The Star Ratings don’t do that, Thompson claimed. “These ratings are intended to drive patients to wise choices about where they seek care. Thus, CMS has an obligation to ensure the differences in ratings it portrays are real, meaningful and important to patient outcomes,” she wrote. “Users and hospitals have a reasonable expectation that, if CMS is going to assign Star Ratings to hospitals for the purpose of identifying different levels of performance, CMS will be able to substantiate its assertion that a three star hospital is more likely to deliver care patients would find superior to that delivered at a one- or two-star hospital, and less likely to deliver superior care than a four- or five-star hospital.”
The AHA head noted that her group had warned CMS of the difficulty in devising a star rating that would “equip patients, families and communities with a meaningful, accurate picture of hospital quality that is relevant to their individual reasons for seeking care.” Lacking confidence that CMS could use the measures available on Hospital Compare to create a comprehensive star rating, AHA urged CMS to apply Star Ratings only to specific measure topics, like cardiac care, rather than one overall rating for each hospital.
As it turned out, CMS developed a star rating that does apply mostly to specific measures, but then touted the ratings as a measure of the hospital’s overall quality, Thompson explained.
Some validity to AHA concerns
It’s hard to argue with substantial sections of the AHA’s critique, says Frank Ingari, CEO of NaviNet, a Boston, MA-based healthcare collaboration network connecting more than 40 health plans and 60% of the nation’s physicians.
“Overall, the methodological analysis deserves a thoughtful response, since it suggests that CMS’ proposed approach may not be optimal scientifically, may promote confusing or even misleading impressions among consumers, and will appear to function as a ‘black box,’” Ingari says. “Most important is the AHA’s request for a more direct line of sight between measured performance and performance improvement actions. After all, this has been the principal power of the Medicare Advantage Star Rating system - payers have well defined paths (however challenging) that they can follow to improve their scores.”
The ratings process should have begun by identifying a small set of scientifically sound measures for critical aspects of care and which consumers find compelling, Thompson explained in the letter. Instead, CMS uses some measures — such as readmission measures — that were created to meet specific legislated program needs, and some created for research and registry purposes. The measures also focus on factors that affect only some patients, such as heart attack, stroke or pneumonia. Few of the CMS measures involve cross-cutting issues affecting many patients, Thompson noted.
“The measures themselves range from patients’ assessments of the cleanliness and quiet of the inpatient care facility, to measures of key outcomes that apply only to Medicare fee-for-service patients with particular conditions, to measures of how efficient hospitals were in providing certain imaging services in the outpatient setting,” she wrote. “It is not clear to what extent, if any, these are the measures that would be most relevant to patients or other users if they were to describe what they would want to be incorporated into a star rating system of hospitals.”
Thompson cited research showing that the readmissions measures CMS uses cannot be used to compare the performance of one hospital to another. Rather, the only valid comparison is between each hospital and a hypothetical average hospital, she noted.
“Thus, we believe it is not possible to accurately assign star ratings to hospitals based on these non-comparable performance measures. The measures are simply not constructed in a way to permit this kind of hospital-to-hospital comparison,” she wrote. “We urge CMS to reconsider using readmission and mortality measures in the star rating system since they will likely lead to the misclassification of hospitals, resulting in misinformation for patients.”
AHA also isn’t happy with CMS plans to group existing measures into seven categories — mortality outcomes, safety outcomes, readmissions outcomes, patient experience, timeliness of care, effectiveness of care and imaging efficiency — and apply a latent variable model to each group. The latent variable model holds that a single common factor is present for the measured performance of a hospital on each of the measures in a group, and that factor could be described as the hospital’s influence on the measured performance. CMS would calculate the value of the “latent variable” and then use that as the measure of how a hospital performed on each category, which dictate how many stars are awarded to the hospital.
That proposal may delight statisticians, Thompson wrote, but it won’t yield accurate quality ratings. “We urge CMS to take a step back and consider something simpler and with a more direct ‘line of sight’ between measured performance and performance improvement actions,” she said.
Even if the program can be improved, Ingari says Medicare Advantage Star Ratings are the most potent proven reform driver in the healthcare system, Ingari says. Using well-defined metrics widely approved by the clinical and payer communities, and powered by significant financial rewards and punishments, the Star Ratings have given CMS a vehicle to reshape U.S. healthcare by effectively “moving the goalposts” a few yards every year.
“The AHA letter can be read, in some ways, as a plea to see that hospital star performers have the same characteristics, including the more gradual introduction that CMS used with the MA Stars program years ago,” he says. “I would like to see the MA Stars program used as the centerpiece of value-based measurement programs in Managed Medicaid, Exchanges, and Hospital Compare. It can always be improved, but is proven, flexible, and effective.”
The AHA letter is available online at http://tinyurl.com/neucjlz.
- Frank Ingari, CEO, NaviNet, Boston, MA. Telephone: (617) 715-6000.
