Do performance measures help healthcare?
Not as much as some experts had hoped
It has been more than 30 years since the Centers for Medicare & Medicaid Services (CMS) moved from using a chart review process to implementation of standardized measures as a way to determine the quality of care patients receive. In those years, there have been periodic reports of vast improvements, huge disappointments, and finding that piece of the puzzle that will make it all come together.
Meanwhile, the reality for health care organizations has been an increasing data collection and reporting burden, and a responsibility to completely change the way large organizations respond to the tales that quality measures tell. They also have to figure out which of the competing performance measures — from a variety of organizations, associations, and government bodies that demand data — they are required to collect, which they might want to gather if they have the resources, and which are just plain useless to their enterprise. Talk about the difficulty of turning a battleship around: Try changing healthcare in America in a meaningful way.
A report released in May and sponsored by the Robert Wood Johnson Foundation and the Urban Institute looks at the future of performance measures1, and it includes seven recommendations that could be the next big thing in helping to improve quality in health care organizations. Or the next medium thing. But those recommendations come with caveats.
We are potentially awash in data. The problem is how to make sure what we collect is meaningful and relates to the problems we want to solve, the authors note.
And the last 30 years of data collection — has that improved anything? “We know that we are improving our performance over time on process measures, but we don’t know if that translates to better outcomes,” says Peter J. Pronovost, MD, PhD, FCCM, senior vice president for patient safety and quality and director for the Armstrong Institute for Patient Safety and Quality at Johns Hopkins University in Baltimore, MD.
Providers must believe in measures
There are instances in which the outcomes for patients have improved. If you look at central line-associated bloodstream infections in the intensive care unit, he says, the numbers have declined sharply in the last decade. That means fewer patients getting sicker in the hospital and, in all likelihood, fewer deaths.
Pronovost, one of the report authors, says there is a question about whether we are, even now, measuring the right things if we want to improve patient outcomes and the quality of care. “If you look at the top causes of preventable death in the hospital, we have things like hospital-acquired infections, decubitous ulcers, and deep-vein thrombosis. But we don’t have a scorecard that measures those things accurately on a national scale.”
Indeed, he says that if you ask doctors about the things that are being measured, many of the metrics are of little use to physicians and lack validity in their opinion. “If you look at CMS and how they measure complications for patients with hospital-acquired conditions, we don’t know how accurate those measures are. If you show me a report of that, I would dismiss it as irrelevant to quality. And that’s not just me, but the vast majority of docs. There are things we measure that we wouldn’t give the time of day to except that we have to measure it; we get paid to do so.”
If the providers don’t believe in the measures, don’t think they say anything that will help them help patients, then the battle is nearly lost. “We measure something like 280 things here at Hopkins. I’d like to see patients and physicians vote on whether each of those were valid and useful measurements that say something about their care and outcomes. And if they didn’t say it was, I wouldn’t do it.” It could be easy to organize such a vote through social media.
“Then if patients say the measures are important, and clinicians say they have scientific validity, we would do it. And there wouldn’t be 300 things to measure. There would be 10,” he says.
Public consensus has been missing from the discussions about this. A few token lay people in committee meetings isn’t enough. And you aren’t going to get the public agreement and acceptance of reported data unless they see the data as clear, meaningful, transparent, and unbiased. That hasn’t happened yet either, Pronovost explains. One idea is to create an overarching body that would be in charge of creating all performance measures, determining their import, looking at whether they are meaningful to stakeholders, and deciding whether and when they should be used for any pay-for-performance program. The body would operate something like the Securities and Exchange Commission does on Wall Street.
It’s not how much you do
There are healthcare organizations out there that collect tons of metrics but don’t provide as high quality care as an organization that measures half as many data points, he continues. “They may have the potential to be better, but measurement isn’t enough. You can’t lose weight just by standing on a scale. But you can’t track it unless you do. Measuring all this stuff? Most of it we do because we are required to. And the physicians who need to see it to improve aren’t seeing that data. So most of what those organizations that measure a lot of things are doing has little correlation with improvement.”
Meanwhile, an organization that doesn’t measure as much but has a leadership that has stated its interest in improving, has managers that create clear goals and programs and processes to meet them, and that measure their progress toward those goals — Pronovost says that hospital “absolutely will move the needle.”
Teamwork, a culture of safety — those are other requirements for improving. But they can be ephemeral and it can be hard to determine if you have really created those things, he says. “We have to do some things that are hard to measure — or that are impossible to measure. But just because we can’t measure it doesn’t mean we shouldn’t do it. And it doesn’t mean that we pretend to measure it, and even if the number we get is junk, we report it and pretend it’s fine. The cost of that kind of measure is more than the benefit you can get out of it.”
The discussion that government, health leaders, and other stakeholders are having now about performance measures is healthy, he says. That people are arguing about whether a penalty for 30-day readmission rates is fair or if it penalizes the hospitals with the sickest patients is all part of what will move us toward a performance measurement system in this country that is much more meaningful and useful than what we have now.
Until the powers that be lay down whatever the new law is, though, there are things that any quality manager can do right now to make the most of the data being collected.
First, Pronovost says to create a good quality management infrastructure. That means making sure that there is accountability. “There is this notion that clinicians have profound individual accountability, yet all this preventable harm is happening,” he says. “I think it’s explainable by the bystander effect: If no one is assigned responsibility, no one steps up.”
Ask at a meeting who is responsible on a particular unit for quality and no one raises a hand or voice. Maybe the assumption is that it is everyone’s responsibility. But it needs to be an actual person’s stated responsibility. There can’t just be a system- or hospital-wide quality director, but someone in each unit and department. “Think of a fractal — there are horizontal and vertical linkages. You have the vertical links for accountability with someone with overall responsibility. But you have horizontal links where various units, departments, or whatever, can share learning with peers.”
Think fractals
At Hopkins, they are piloting a new quality paradigm where there is a system-level quality committee that includes each of the six hospital CEOs; at each hospital level, there is a committee that includes every department head; at the department level, every unit is represented; and at the unit level, it’s the clinicians who show up. At each level, there is an individual with accountability, and at each level both physicians and nurses participate. In some of the higher echelons, a tech person may also be a mandatory participant. The people who have accountability train for a patient safety certificate with 40 hours of coursework that includes learning about issues like teamwork, patient-centered care, Lean and Six Sigma management, and costs. The upper levels also receive training in staff evaluations.
So far, they figured that if every department supplies a nurse and a physician half time, and every unit 10% of a nurse and a physician, it will cost 0.8% of the hospital’s revenue. Given that hospital margins are running about 1%, that’s significant. But Pronovost says they are fairly certain this new structure will save 4-5% in hospital revenues. The pilot, at Johns Hopkins Hospital, is starting in just three departments and should continue through the end of the year. If it is as successful as Pronovost thinks it will be, it will expand to the rest of Johns Hopkins Hospital and the other five facilities in the system.
That kind of makeover may be too big a bite for some. But there are other things you can do right now, Pronovost says. First, if you are collecting data that no one uses or looks at, stop it. To figure that out, ask every department and unit to list every performance measure it collects. Talk to the users of that data about its usefulness. Have them rate it on a five-point scale for importance and validity. Determine what you can stop collecting. “This can lead to some great conversations about what might make a particular piece of data useful,” he says. “Maybe you are collecting something at some point in the patient experience, and if you change it, maybe you can create some data that will be of use to providers and patients.”
At the very least, you may be able to reduce your data collection burden. If you’re lucky, you might find a way to make the performance measures you collect do real good for your patients.
For more on this story, contact Peter J. Pronovost, MD, PhD, FCCM, Sr. Vice President for Patient Safety and Quality, Director of the Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Baltimore, MD. Telephone: (410) 502-3231.
Reference
1. Berenson RA, Pronovost PJ, Krumholz HM. Achieving the potential of health care performance measures. May 2013. http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-healthcare-performance-measures.html
It has been more than 30 years since the Centers for Medicare & Medicaid Services (CMS) moved from using a chart review process to implementation of standardized measures as a way to determine the quality of care patients receive.You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content