Myths that trip you up: Are you guilty of these benchmarking mistakes?
January 1, 1998 7 minutes read
Myths that trip you up: Are you guilty of these benchmarking mistakes?
Pursuit of numbers can work against you
Because challenging the status quo of an entrenched hospital culture is at the very core of benchmarking, professionals must counter common assumptions about the process or else accept mediocrity and perhaps even failure, say seven benchmarking experts surveyed by Healthcare Benchmarks. What are the top benchmarking myths they’ve encountered in their careers? The answers may surprise you:
1. We must compare apples to apples. While this oft-repeated phrase certainly contains a basic benchmarking truth, it may disguise the real opportunity for improvement, the experts say. The problem with this assumption is that it represents not only benchmarking at its simplest level, but a "compliance mindset typical of organizations not yet mature in their quality process," says Ned Barber, PhD, president of Barber Associates Inc., a quality assessment and improvement consultant for health care systems in Park Ridge, NJ. "The tendency is to only look to see if your performance is comparable to an equivalent organization. But this is only covering the essentials. After all, shouldn’t you be doing at least as well as they are?" he says.
That elementary approach ultimately may cause your facility to be less competitive, adds Eleanor Anderson-Miles, director of corporate communications at MECON, a benchmarking firm in San Ramon, CA. "Members of an alliance benchmark against each other, but it’s not each other they are competing against," she says. "For example, if you’re a teaching hospital, you automatically think you must compare yourself to another teaching hospital. But it may be the community hospital down the street that is your biggest competitor."
For instance, suppose a freestanding ambulatory care center offers quick and efficient cataract surgery. Should your large university medical center even try to benchmark the freestanding center’s best practices?
Sure, says Barber. "Examine competing organizations who are doing it better and strive for that level. You may not be able to perform there because of the economics of volume, but at least you’re raising the bar."
Ron Webb agrees. "By spending too much time worrying about how your apples compare to other apples, you may miss the fact that people are going somewhere else because of the oranges," says the senior benchmarking specialist at Interna-tional Benchmark Clearinghouse in Houston.
Insisting on only one type of tool in your benchmarking kit leads to the second biggest benchmarking myth, says Sharon Lau, consultant for Medical Management Planning, the facilitator and vendor based in Bainbridge Island, WA.
2. But we are different. "Even when benchmarking at the departmental level, no two will ever be 100% alike," says Lau, who refers to this assumption as the "terminally unique disease."
By concentrating on quantitative differences between the two institutions instead of their qualitative processes, benchmarkers often end up using the difference to justify any change.
"This disease is closely related to the yes-but’ syndrome that explains’ why your model of efficiency won’t work at my institution," Lau adds.
Those who have succumbed to either infirmity won’t dare benchmark outside health care. Yet Carla O’Dell, PhD, president of the American Productivity and Quality Center, says she "constantly sees evidence of the power of out-of-the-box benchmarking" among the organization’s 250 members.
A leading medical center, for example, may study Marriott’s hotel guest registration process to improve their own admissions process. "To reach new, world-class levels of performance, you have to break out of your industry paradigm and ask, Who does this process better than we do?" she says. "Remember, health care patients judge you not only on the quality of care, but also on how much time, hassle, and paperwork is involved." (For more information on cross-industry benchmarking, see story, p. 4.)
If the first set of misconceptions hasn’t set you up for failure, the next will.
3. Benchmarking will change performance.
For those expecting instant results from a set of numbers, benchmarking without identifying best practices is bound to produce disappointing results.
"Benchmarking itself is not a magic bullet," Lau says. "It gives you tools and information and knowledge, but you must act on them. If you’re not willing to change, all you have is a notebook of data on a shelf."
For those who are convinced that nothing can help their sad state of affairs, benchmarking without acknowledging the underlying morale issues may only further ingrain stagnation.
"If an organization tends to use reprimand as its method of inspiration for improvement, any initiative will be threatening," stresses Anderson-Miles. "Employees will be afraid they will be penalized for what the data show and shut down or not fully cooperate."
Or, staff may fear they will be held accountable for change based on unreliable data. "Until this decade, we didn’t have good comparative information. Data were aggregated and poorly collected. People still remember that," she says.
Yet the tendency to depend on numbers runs rampant in a data-driven industry like health care. This characteristic leads to the fourth, and often fatal, mistake:
4. Patient satisfaction scores tell all. By relying solely on what Barber calls "external data" such as patient satisfaction scores, you actually may miss the underlying cause of the patient complaint. He offers the example of food temperature. "Patient satisfaction scores for food may be misleading because many factors can influence it. The most accurate measure would be to stick a thermometer in the food to gather information firsthand on how hot or cold it is by the time it gets to the patient’s room," he says.
Those specific data then can allow you to look at the process of how the food is prepared and delivered. "That’s the real issue," he says.
Likewise, the customer satisfaction trap can produce a misleading sense of complacency when comparing yourself to your peers, says Richard A. Wargo, a former benchmarking specialist with the International Benchmarking Clearinghouse. Wargo now is the total quality specialist at Honey-well’s Home and Building Control. "By merely looking at numeric "benchmarks" of customer satisfaction, you ignore two things: the information you will need to achieve superior levels of performance, and the differences between your organization and the benchmark organization," he says.
5. Benchmarking is just data gathering.
All experts agree those top four ailments are but a symptom of the real problem: equating benchmarking with data gathering.
"Many organizations still think benchmarking is about comparing numbers," Webb says. "The attitude is, We’re performing here; hospital X is performing there.’ And that is where they stop. But the real power of benchmarking is to find out how or why they do certain things. That’s what will allow you to make the largest degree of change."
But how do benchmarking professionals get administrators to look beyond the numbers?
"Explain that, while accurate data are important, they won’t give you the true picture," Webb says. "Tell them that the largest return for their benchmarking investment will be in learning the process behind the numbers."
Mark Czarneck, MBA, CPA, says the effect of data on a benchmarking project’s success can be a double-edged sword. "The good news is that your physicians become statisticians. And the bad news is that your physicians become statisticians."
While physicians are by nature interested in sound research data and, more importantly, often convinced to change because of it, they also can get so involved in gathering and analysis that the real purpose is forgotten. This excessive measuring and monitoring actually can impede efficiency instead of promoting it, Czarneck says.
All the experts agree that a balance of metrics and best practices is the key to true process improvement. "On the one hand, you don’t want to get so caught up in data that you see it as an end instead of a means," Czarneck says. "On the other, you don’t want to delve into improving processes without having sound data to back them up. You may be doing thing twice as fast, but entirely wrong."
How can you create that careful equilibrium of benchmarking and best practices? Experts say it’s all in how you establish measures and ask questions. "Measurement should be used to speed things up, not to slow things down," says Thomas W. Nolan, PhD, principal of Associates In Process Improvement in Silver Springs, MD. "Instead of measuring all of something, measure a sample every 10th patient, or the next 10 bills."
For example, if you want to benchmark delays in the operating room, measure them at prime activity times such as 11 a.m. and 2 p.m. Instead of measuring delays every day, measure each of your operating rooms one day per week.
Keep it simple
"By computing the median of those times and then plotting them on a chart, we had an ongoing data collection that was accurate but not cumbersome," Nolan explains. "It’s important to keep data collection simple, so you can put energies into test of change rather than data management."
He reassures those who worry that small-scale data lack "scientific" validity. "Randomized clinical trials are needed to establish standards of practice but not to test best methods for putting those standards into practice," he says.
Finally, know what questions you need to ask before you begin making benchmarking contacts and especially before going on site visits. "If you have only a vague idea of what you want to know, then you are not conducting a site visit, you are participating in industrial tourism," Barber warns. "You’re likely to come with random ideas you won’t be able to apply."
Webb suggests searching the literature for background and ideas that can help clarify the questions you want to ask. "Then formulate a list of 10 questions, starting with the No. 1 thing you want to know about their process in order to improve yours. And resist the urge to ask about the numbers. Instead, ask, How did you do it?’"
For more information, contact Carla O’Dell, President, American Productivity Quality Center, and Ron Webb, Senior Benchmarking Specialist, International Benchmarking Clearinghouse, 123 North Post Oak Lane, Houston, TX 77024. Telephone: (800) 776-9676. Web site: www.apqc.org. Sharon Lau, Medical Management Planning, 2049 Balmer Drive, Los Angeles, CA 90039. Telephone: (213) 644-0056. E-mail: [email protected]. Web site: www.mmpcorp.com. Ned Barber, President, Barber Associates Inc., 10675 Queen Court, Park Ridge, NJ 07656. E-mail: [email protected]. Web site: qserve.com/ ba-inc. Thomas W. Nolan, Associates in Process Improvement, 1110 Bonifant St., Suite 420, Silver Springs, MD 20910. Telephone: (301) 589-7981. Mark Czarneck, President, Benchmarking Network Inc., 4606 FM 1960 West, Suite 555, Houston, TX 77069-9949. Telephone: (281) 440-5044.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content