Mix of national, local benchmarks are critical for perspective
However, there are times when regional, local data are preferred
Have you ever done a benchmarking study and thought, "There’s no way those numbers can be right?" Ever been tempted to scrap the whole thing when the data don’t fall in your favor? Ever figured your community has too much managed care penetration or too many diabetics or too many elderly people to make national benchmarking helpful?
If so, you’re not alone. In fact, one consultant calls the following four words the deadly disease of benchmarking: "Yes, but we’re different." But just because you’re not the only one who feels that way doesn’t mean you’re right, says Sharon Lau of Medical Management Planning in Los Angeles.
"Of course we’re all different, but somebody’s doing it better, and we may be able to learn something from that," Lau says. "We always hear some hospitals say that the national numbers don’t apply in their situation, but we try to discourage that type of thinking. There is always something to learn from the national benchmarks."
The editors of Healthcare Benchmarks tried to get you off the hook with the argument that national benchmarking might not apply in certain situations, but we couldn’t find anyone who agrees with that statement. What we heard, instead, is that you need a range of national and local information if you want your benchmarking efforts to be successful. You want the national numbers so you can keep your finger on the pulse of what’s hap pening around the country, and you need the local ones to help you differentiate your service.
Even if you’re a cancer research hospital, for example, and you can’t find many other institutions that can relate to your clinical profile, you still can use national benchmarks in situations that everyone faces, like admission wait times, Lau says. Anything that’s process-oriented, such as billing, can be measured no matter what the other circumstances are. In fact, you would learn more if you looked for someone who isn’t like you at all.
"Don’t use your local situation as an excuse. You want to find the process that’s the very best regardless of where it is and what industry it’s in," she says. "I can’t think of anyone who gets a bill out faster than American Express. That’s the example I would use if I wanted to improve my billing process."
On the clinical side, people will say they couldn’t possibly use a clinical pathway that proved successful for asthma treatment because it’s cookbook medicine, and their physicians would never go for it. "But I say there’s always something you can take away from a best practice, even if it’s just pieces that you can adapt to your organization," Lau says. "We hide behind the excuse that our population is unique. I think there’s a lot of comparability in populations. If we were at least open-minded enough to look at all of the data out there and consider using pieces of it, I think we could improve our processes."
Benchmark for perspective
Bob Gift, principal of Systems Management Associates Inc. of Council Bluffs, IA, says any benchmarking effort with any organization, no matter how different, provides needed perspective. "You’ll never find another organization that is exactly like yours, and even if you could, you wouldn’t learn anything from them because they’re the same as you," Gift says. "The way you learn is by looking at people who are different. Too often what happens is that people find out they’re not in the ballpark, and then they say the comparative data is no good anyway because it’s comparing apples to oranges."
It’s true that the national benchmark might not wholly apply in your situation, but that doesn’t matter, Gift says. "What you really should be doing is looking at the actual work that produces the benchmark. So it doesn’t matter if you have a higher incidence of diabetes in your region than the facility you’re looking at. The first argument people will give is that their patients are sicker. But we still have a wide variation in practice patterns across the country, and there has to be a reason for that."
You need the national benchmark to provide perspective and incentive for improvement, but it won’t do you any good unless you ask the right question: Are we improving our own performance? It’s not enough to ask if you’re meeting the national benchmark. Gift says you should use the national data to find out if you’re in the ballpark and then take the benchmarking information you gather and use it to stimulate change in your organization.
The only way to stimulate real change through benchmarking is to look at the better performers and understand what makes them successful, says Eleanor Anderson-Miles, director of corporate communications for MECON Associates in San Ramon, CA. "Too frequently, people think benchmarking is comparing data, but it’s not. The data is the starting point that helps you pick your benchmarking partners. Then you look at their processes. If you don’t change the process, you won’t change the data."
That means you should never look at the national average, Anderson-Miles says. The average is helpful for getting a sense of the trend, but it doesn’t give you the opportunity to find out where that number came from. You have to look at the specific institution that is performing well and find out how they’re doing it.
"If you have a high population of diabetics, pull peers from other places in the country that also have similar percentages and look at what they’re doing," she says. "If you have a Level I trauma unit, benchmark with other Level I’s. Figure out what your pertinent influences are and use those as your key criteria for finding your peers."
The other reason for national benchmarking is to get a broader base of knowledge, Anderson-Miles says. "People who go to medical school often end up in hospitals near that school, and ideas become entrenched. If you break the mold by going outside that area, it will result in a better job. If you benchmark with a small local group, you might do better, but then again, you might be the least rotten apple in the bunch. There’s no way to tell unless you go outside."
And once you find out those national numbers, don’t be afraid of them, Anderson-Miles says. Many people are threatened by benchmarking. But that defeats the purpose, which is, after all, finding out how people are doing things better than you are.
"You have to change the culture to the point where it’s OK to find out you don’t have good productivity," Anderson-Miles says. "Then you go out and find the best ideas for improving. Too frequently, people use data as a means of punishment. I’ve seen hospitals collect data and find out they’re not doing well and then destroy the data."
Anderson-Miles could only think of one situation in which national benchmarking would not be useful: if you’re going through a merger and are doing a comparative study to see which site is best at a particular function. "But if your goal is operations improvement and saving money, it always benefits you to go far and wide," she says.
Which benchmarks count most?
Dennis Dunn, senior scientist for the Sachs Group in Evanston, IL, agrees that you need both the national and local numbers but says there are circumstances when one set should weigh more heavily. "The national benchmarks really give you different information than local ones," he says. "Sometimes you need that broader view of the cost of a disease or general staffing requirements, and you need that standard to measure yourself against. But if you’re planning a disease management program, you need a forecast that really represents what will happen in your local population."
Because of wide variations in practice patterns, you always should be aware of your local statistics when doing any type of planning or forecasting, Dunn says. "If you have an enlarged prostate, there’s going to be a huge variation in how you’re treated in Colorado vs. New York. The probability is many times higher in some places that you will be treated with surgery, and in those areas, you’ll need different resources."
Or if you have a particularly high incidence of a certain disease in your area — like diabetes in areas with high concentrations of certain Indian tribes — then you need your numbers as local as you can get them. "That’s important for any kind of a planner looking at a diabetes treatment center or any preventive services," Dunn says. "In certain tribes you have a greater than 50% incidence of diabetes, so you’ll have a tremendous demand, but in another population, you may not have a fraction of that. Local area information can change your notion of what staffing is required and can pinpoint areas with special needs for intense preventive medicine."
Site selection and construction is another case in which local data is paramount, he says, since physician practice patterns will determine how many rooms you need in a satellite surgery center, for example.
"We give our clients both national and local data," Dunn says. "We try to give benchmarks for bellwether communities such as Minneapolis and the San Francisco Bay Area where managed care has been an institution for decades. As people start seeing managed care increase in their area, they want to know what the extreme case can be."
Then there’s political reality
Philip Newbold, chief executive officer of Memorial Hospital in South Bend, IN, says his hospital always looks at both national and local benchmarks. But he admitted that reality sometimes pushes aside the national results. "It might be interesting to see how another medical staff does something. Maybe they employ advanced nurse practitioners, but if that doesn’t fit how your medical staff practices, it doesn’t really matter," Newbold says. "It pays to be aware of how other people are doing things, but the political reality is that if your medical staff is not going to credential advanced nurse practitioners or do a certain procedure on an outpatient basis, you won’t be able to implement it."
What isn’t acceptable, Newbold says, is knowing about an improvement in quality that you can implement and not doing it. "There are hundreds and hundreds of quality improvements that could be made in hospitals tomorrow morning that have been in the literature, and all people have to do is read it and implement it and yet we don’t do it," he says. "We don’t deal with it because of time pressures or the pace of change, but there’s no excuse."
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content