Report cards don’t make the grade with physicians or consumers
The good news: Organizations use info for quality improvement
For years now, health care organizations have been creating and touting programs that allow consumers to see how they do when it comes to a variety of quality measures — from preventive care and well-baby visits, to caring for those with chronic illnesses. Sometimes the data relate to physician groups, at other times to health plans. But regardless of the kind of data presented, consumers aren’t using them to make health care choices, and physicians don’t use them to determine to which specialists they should refer patients. Those are the findings of a new study published in February by RAND, a Santa Monica, CA-based research and development organization.
The good news from the study, however, is that health care organizations and hospitals seem to use the data as a stepping stone for quality improvement programs, says Paul Shekelle, MD, PhD, lead author of the report, a staff physician at the VA Greater Los Angeles Health Care System, and a RAND consultant.
"There was this idea that report cards were going to foster a consumerist approach to health care the [same] way there is a consumerist approach to buying a television or refrigerator," Shekelle says. "That hasn’t panned out."
The reasons are manifold, he says. Topping the list is that those who create the report cards don’t make them in a readily understandable format. "Another reason is that you often think about buying something like a television prospectively," he says. "But with health care, you don’t have time to study the issue. It’s not like you can postpone a heart attack until you have researched doctors and hospitals."
Also, says Shekelle, at the health plan level the unit of delivery of the information isn’t as meaningful to patients as information about their own physicians would be. "Someone might say Cedars-Sinai is good for bypass surgery, but that message isn’t nearly as powerful as someone saying that Dr. Jones is great for bypass surgery."
It isn’t just potential patients who find health care report cards less useful than their proponents would have liked. Physicians, also, are not likely to use publicly available information when making decisions about where to refer patients. "Although physicians tend to consider the information accurate," the RAND report states, "they seldom share it with patients. Moreover, many physicians believe that public disclosure of performance information encourages other physicians to refuse to treat those patients in the poorest health." The report goes on to say that there is no evidence that those beliefs have a basis in fact, but they still are a factor in physicians’ decisions not to use publicly available data.
Positive response from plans
But there are groups that take report card data and make use of it: health plans and hospitals. The report notes one particular study that proves comparative data can have a positive impact. In the early 1990s, New York State began a reporting system for physician and hospital performance on bypass surgery. The data included mortality, choice of physicians, and hospitals. During the study period, risk-adjusted mortality rates improved dramatically.
"Plans and hospitals perceive this data as a big deal for one of two reasons," Shekelle notes. "Either they are worried about prestige, or they are worried about market share. But their public reaction is the same. If they score high, they tout their performance as a top hospital. If they score low, they say the data collection is flawed. Then they go back to their facilities and do whatever they can to make sure the numbers look better the next time around. That there is an agent of change at work here is clear."
There is a fear that in order to make the information more accessible and understandable to the consumer at large, those collecting it and creating report cards from it may dumb it down too much. Shekelle thinks the future may lie in a multilayered version of a health care report card. Consumers could find out a simple score — based on stars or a number rating system — for their hospital or plan, but they also might be able to dig down deeper to find out more detailed information.
"I don’t think [this report] is going to spell the end of report cards," he says. "They aren’t going to go away. I think that it will continue to evolve down to smaller units of measurement — down to the individual doctor level that is of interest to patients."
What is less clear, Shekelle continues, is whether the data will become more comprehensive or continue to focus only on a few measures. "We would ideally like to see a much more comprehensive quality assessment that uses fewer people in the sample but more items."
Currently, most report cards are based on one of two data sets, the Health Plan Employer Data and Information Set (HEDIS) collected by the National Committee for Quality Assurance based in Washington, DC, and the Consumer Assessments of Health Plans Survey collected by third-party vendors. HEDIS, says Shekelle, focuses on just 14 items. Shekelle and his colleagues would like to see upwards of 200 different items collected.
"The biggest concern that consumers and organizations should have is whether the measures are giving a fair picture of quality," he says. "The individual items are just small samples of quality. Ask yourself: Are they reflective of the whole organism?"
Not that Shekelle thinks anyone should stop creating or using the report cards because they aren’t perfect. "I think that if we continue to wait for more perfect systems, that is tantamount to endorsing the status quo. And everything we know about the status quo is that there are gaps between what happens and what should happen on the most basic of measures. We don’t have to wait for a perfect cure. We use what we have even if it has faults and flaws. Then we work on making it better."
The RAND study lists three things that can make the report cards more useable. First, make sure the data are presented in a readable manner. There are several ways this can be done, including the judicious use of clear headings, presenting information in several ways — such as graphically and in text — and making sure the information consumers will be most interested in is available to them without having to read through an entire report.
Second, educate the various audiences — the public, health care professionals, and the media — on the data and their importance.
Lastly, assess the impact of the public release of information. Find out who uses it and how. That, the report concludes, "will allow the creators of health care report cards to tailor the messages they present more effectively and to reach a much larger segment of the potential users."
Shekelle hopes that data will continue to be available publicly, although there are some in the industry who would like to keep data distribution private. "There isn’t any proof of it, but I think there is an intuition that internally released data won’t have the same impact. Public release of quality information does provide an impetus for positive change."
To see a complete copy of the report, visit the RAND web site at www.rand.org/publications/RB/RB4544.
[For more information, contact:
- Paul Shekelle, MD, PhD, Staff Physician, VA Greater Los Angeles Health Care System and consultant to RAND, 1700 Main St., Santa Monica CA 90407. Telephone: (310) 393-0411, extension 6669.]
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content