A new analysis from the American Hospital Association (AHA) concludes that there are fundamental design flaws in the CMS star ratings.
The flaws may make the star ratings invalid for comparing hospital quality, the report says. The analysis was conducted by Francis Vella, PhD, chair of the Department of Economics at Georgetown University in Washington, DC. (Vella’s analysis is available online at http://bit.ly/2aJi5kA.)
Rick Pollack, AHA president and CEO, issued a statement saying the analysis was troubling.
“As currently designed, CMS’s star hospital ratings program is not up to the task of providing the public with meaningful and accurate assessments of hospital performance,” Pollack said. “Patients need reliable information to make important choices regarding their healthcare. And hospitals and health systems need reliable information so that they can continue to improve the quality of the care delivered. CMS star ratings misses the mark on both accounts.”
Vella’s analysis specified the following problems with the star ratings:
-
There is a wide variation in the number of measures and categories used to identify the star ratings across hospitals. Different hospitals are rated using a number of different measures, which can bias the results, irrespective of their actual performance.
-
The methodology is not well constructed. The estimation aspect gives the impression of being rigorous and objective, but actually is highly dependent on choice of measures and the weighting scheme is entirely subjective and highly determinant of the final outcomes.
-
Ignoring other social determinants of quality outcomes (such as location of hospital, race, income, and patient composition) potentially biases the results. Two or more identical hospitals could have very different outcomes depending on the type of patient they have, where they are located, the type of health issues they typically face, and multiple other factors.
-
The use of a star system implies that substantial differences in quality may exist across hospitals when they do not.
AHA’s statement accompanying the analysis urges CMS to make substantial changes in the star ratings system before making it public.
