Leapfrog safety scores spur debate: How reliable are they?
Some low-scoring hospitals say results are invalid — Leapfrog unperturbed
Low patient safety scores from The Leapfrog Group have some hospitals crying foul and claiming that the group’s data is old and the scoring methodology flawed. If you didn’t get the score you were hoping for, should that be cause for alarm, or can you dismiss it?
The answer depends on who you talk to, but by and large most hospital administrators and quality experts consulted by Healthcare Risk Management say the Leapfrog scores are considered reliable. That doesn’t mean they are an exact assessment of your patient safety experience, but they are the closest anyone outside your organization is going to get, says Leah Binder, president and CEO of The Leapfrog Group.
“I would send my family to a hospital that got an A, and I would be cautious at a hospital that got less than an A,” Binder says. “I put my own family’s health at stake when assessing the validity of this score. That’s a strong statement, but a true one.”
The Hospital Safety Score was first launched in June 2012, measuring 2,652 hospitals on 26 factors that the group says indicate the level of overall patient safety. Hospitals are given a score of A, B, C, D, or F, and more than 1,200 hospitals earned a C or below. The 26 factors are measured through a voluntary survey to hospitals and also from the Hospital Compare data compiled by the Centers for Medicare & Medicare Services (CMS).
Over the summer of 2012, the Blue Ribbon Expert Panel convened to review the methodology in the light of the first launch, considered commentary from experts and associations including the American Hospital Association (AHA), conducted data analyses, and looked at additional evidence. The Blue Ribbon Expert Panel recommended maintaining the original scoring methodology, with the exception of two alterations that give certain hospitals more credit on two of the measures. The Leapfrog Board approved the recommended methodology.
As soon as the first batch of scores was released, complaints were heard from some of the hospitals that did not fare well — including some respected providers. Mount Sinai Medical Center in New York City criticized the safety scores and called the grade “an incomplete and imperfect snapshot” in a statement to The Wall Street Journal’s Heath Blog. The hospital added that “much of the analysis is based on outdated information from disparate sources.” (See the full comments at http://tinyurl.com/wsjstatement.)
The Cleveland Clinic in Ohio received a C and rejected that assessment for the same reason. The clinic said The Leapfrog Group used old data. Some hospitals, including Cleveland Clinic, have opted not to participate with the Leapfrog surveys on which the scores are based because they don’t have confidence in the system. But administrators from those hospitals also have claimed that the participating hospitals have an advantage because they are scored on more elements.
Completing the survey is labor and time-intensive, without yielding much useful information for the hospital, explains Shannon Phillips, MD, MPH, quality and patient safety officer at the Cleveland Clinic. “We decided we wanted those same talented people to be doing something that made a difference in patient safety rather than spending that time on the survey,” Phillips says. “The Leapfrog score is one of many scores that are generated with different data and methodologies, and anything that suggests we don’t provide safe care isn’t going to make anybody happy. But we’re staying very focused on our safety and quality goals, and these scorecards are not part of our institutional strategy.”
The AHA also criticized the Leapfrog scores in June 2012 and said they were “neither fair nor accurate” and that “no one should use it to guide their choice of hospitals.” (See the full letter at http://tinyurl.com/letterfromAHA.) AHA President and CEO Richard J. Umbdenstock even went so far as to accuse The Leapfrog Group of manipulating data to give some hospitals poor scores.
In response to criticism of the scoring methodology, Binder acknowledges that “we have excellence, but not perfection.” The most frequent complaint is that the data does not reflect the most current status of a hospital’s safety, but Binder says that always will be impossible.
“When we hear that the data is too old, it’s from 2011, and I don’t think that’s very old,” Binder says. “Sure, I’d like to have a ticker tape in my office that gives me up-to-the-minute data on every hospital, but we don’t have that. We use the latest data available to us, and I trust these scores with my family’s health.”
Binder also offers to work with any hospital association to urge CMS to release data on a more timely basis. “We do support the idea that we should get the data faster, but having said that, this can’t wait,” she says. “Consumers need the information.”
The complaints probably are driven in part by the fact that hospitals are not accustomed to transparency with their safety data by having it available to the public and assessed for an overall score, Binder says. Risk managers and other hospitals administrators also might be acting on the fact that their own internal data is different from that used by Leapfrog and might sometimes portray a more favorable picture, notes Patrick Romano, MD, MPH, professor of general medicine and pediatrics at the University of California, Davis, and a member of the panel that assessed the Leapfrog methodology.
It is understandable that a hospital’s internal data could paint a different picture if it is more current and complete than what was available to Leapfrog, Romano says. That does not invalidate the Leapfrog scores, he says, but it should be impetus to provide Leapfrog with that information.
“Realize that The Leapfrog Group is going to assess the data and score your hospital whether you participate or not, so it is to your advantage to participate and help us gather the most complete, accurate picture of your patient safety experience,” he says.
Sources
- Leah Binder, President and CEO, The Leapfrog Group, Washington, DC. Telephone: (202) 292-6713.
- Shannon Phillips, MD, MPH, Quality and Patient Safety Officer, Cleveland Clinic, Cleveland, OH. Telephone: (216) 444-4998.
- Patrick Romano, MD, MPH, Professor of General Medicine and Pediatrics, University of California, Davis. Telephone: (916) 734-7237. E-mail: [email protected].
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content