With disparity among systems, it’s hard to compare apples to apples
As the experts will attest, benchmarking is part science, part art. The science of data capture is becoming more sophisticated all the time, although a relatively small number of institutions currently are taking advantage of state-of-the-art, fully integrated systems.
"It’s all going electronic," says Stephen Lawless, MD, MBA, chief knowledge officer of Wilmington, DE-based Nemours, one of the largest pediatric subspecialty practices in the country. "The next step, although only a few have delved into it, is an integrated system. Lots of people have electronic documentation systems of encounters, billing, pharmacy, or orders; but less than 1% or 2% of the country have systems that are integrated, which means those systems actually talk to each other."
"Benchmarking has evolved to be much more sophisticated," adds Tania Bridgeman, RN, PhD, director of clinical path development for the University of California at Irvine Medical Center, (UCIMC), a tertiary, full-service facility that has a fully integrated system. "There’s more input, the sophistication of the data is better, and you have a higher confidence level when you go out there [to benchmark with other institutions]."
The advantages of system integration are many, Lawless says. "If you have complete data capture, and if, for example, you’ve admitted a child to your hospital, you have a record that the child has been seen, all orders are electronic, and most of the documentation is electronic. Then, when he is discharged, electronic summaries are sent out. Also, when he is seen for a follow-up in the doctor’s office, the electronic system there is integrated with the hospital. That is the ideal, although surprisingly few people have such a totally integrated system," he observes.
What does complete integration mean to your benchmarking efforts? "In terms of benchmarking, it means you have all the elements," Lawless says.
One of the early benefits of electronic systems, he notes, was to take care of legibility problems, which were at the root of about 50% of all errors in prescription writing. "The other 50% were due to people prescribing duplicate drugs, the wrong dosage, drugs that were not appropriate for integration with other drugs, labs ordering in duplicate, or people making decisions in care based on a lack of information," he observes.
"Did the doctor know, for example, whether his patient had previously seen, say, a lung doctor?" he poses. "You often had to rely totally on word of mouth or on a phone call. Now integration is seamless. If the doc down the road has already seen my patient for a routine exam and has done a cholesterol screening, why would I want to duplicate that? From a benchmarking standpoint, completeness of record is no longer defined by whether you have a history. Now the key benchmarking question is, do you have all the elements of patient care in a continuum? The change in the benchmarking process is dynamic."
For example, Lawless says, assume your outcomes measure expresses how few school days your asthmatic patients miss compared to someone else’s patients. "Now, with all the data elements captured, you can really compare and see if your management style is different from that of the doctor down the street; you’re not just comparing patient age, sex, and race. Integration will be able to help us benchmark outcomes really nicely," he explains.
"It’s not good when your systems can’t talk with each other," Bridgeman says. "We’ve had a high degree of success with clinical path development because we’re automated."
Part of her clinical path development focuses on benchmarking, she explains. "You have to benchmark. Let’s say you want to address joint replacement; it doesn’t behoove you to just sit down with everyone at your facility who does them and create a pathway; you have to go out and benchmark," Bridgeman says.
With a number of vendors in the field, there is more than one way to achieve integration. At UCIMC, they use two major systems — TDF, by Atlanta-based Eclypsis, a clinical documentation system, which interfaces with a decision support system from a company called Transition Systems I. "They manufacture the software for a hospitalwide database with everything in it — docs, discharge, diagnosis codes, complication rates, age, sex, and so on," she adds.
UCIMC has had TDF since Bridgeman joined the staff in 1999, and had the decision support software a couple of years before that. The two systems can talk to each other. "Decision support is resource-driven — everything that’s charged in the hospital will show up," she says. "You can even tease it into getting clinical information by creating a dummy charge code."
Decision support provides the internal data, such as the physician’s name, DRG codes (for broad definitions) or ICD-9 codes (which, Bridgeman says, are superior), cost, length of stay, age, sex, financial category, charges, variable direct costs, total cost, and payer mix.
"Externally, we go out into the university health system consortium — UHC," she says. "This includes universities with affiliated hospitals, which input data from all over the country. You get morbidity, mortality, complication rates, race, age, sex, unexplained deaths, costs; you can even hone down into utilization in an OR or how many days critical care was used." It’s the UHC, she says, that does the risk-adjusting.
Catholic Health Initiatives, a Minneapolis-based health care system with 60 hospitals across the country, works with a vendor called Soluciant, says Mike Stoecklein, senior operations consultant, who provides internal consulting resources for performance improvement, working with the hospitals’ performance management teams.
"Our system works with Soluciant to provide benchmarking data under a program they call Action OI," he says. He adds that the system currently is installing "a more robust" version of the software.
"It looks at a variety of functions, both at the hospital level, like revenues, expenses, productivity, and it can also let you compare yourself against similar hospitals," Stoecklein says. "You can choose certain attributes. At a department level, you can pick and choose the comparison groups that make the most sense for you. You can also perform normalization of the data to look at individual differences and nuances of different facilities."
Scientific advances in benchmarking, and even complete integration, often are accompanied by a new set of challenges, says Lawless.
"It can create problems," he concedes. "Today, we presented our quality dashboard to our major board of managers at Nemours. The main issue they had was with benchmarking — looking at medical errors and rates."
The board asked if Lawless had any benchmark data to compare with other facilities. "I explained that we have electronic orders, electronic capture of data, and so on," Lawless says. "The trouble is, most people out there who publish data on medical errors use a paper system. Our data look 10 times better, but in the true spirit of benchmarking, we have to find out how well we do compared to those facilities with other integrated electronic systems."
Most groups now use the same processes, he says, but electronic systems create brand new processes. "Once you have [an integrated system], data becomes so much more accessible; but once you have it, what do you do with it?" he asks.
With integration, defining terms becomes more critical, says Lawless. "For example, what is a patient encounter or visit? If someone asks, How many medical errors do you have per patient day,’ what do you mean by medical error? Now that we can share the data so easily, those missed definitions are very crucial." The good news, he says, is that "if we define carefully, we can really compare one institution to another." Right now, however, there is no universal format for medical record numbers, he concedes. "So, how do you match up one [facility] to another?"
Bridgeman also sees definition challenges presented by integration. "We don’t know if we’re comparing the same elements. We don’t really know what the other facilities are doing. When UHC, for example, applies what they call a ratio of cost to charge, you might not be sure what the other hospitals submitted — variable direct costs or actual costs. Also, while most of the participating institutions have colleges of medicine and research, we don’t always know if they do. And if you do, your costs are higher."
Vendors also eventually must learn to speak the same language, Lawless says. "Vendors are crucial. I’d say that 25% of the large organizations are implementing electronic systems, but you have some vendors that are best in breed’ for practice management, and others for patient care documentation," he asserts.
"What ends up happening is you create lots of interfaces. The next step is to make sure all the elements are defined the same way. If vendors could talk to each other or agree to certain nomenclature people would be comfortable with, you could buy different products and have them all interface," Lawless continues.
Bridgeman cites a real-world example of problems that also arise with external databases. "We use the Office of Statewide Health Planning Department, based in Washington, DC, and [like UHC] they also depend on what the other hospitals submit. They provide data on ethnicity, sex, age, comorbid conditions, mortality, payer mix. They also quote median income per capita per region, which makes the data useful for marketing, by getting patient populations within certain rings. The issue with them, however, is that their data are normally around 18 months old."
Still, she insists, "Both are fabulous databases." As for her software vendors, both of them have hotlines. Here, too, however, definitions are critical. "Both are dependent on having a very accurate query. Otherwise, with the wrong ICD-9 code, for example, you’re likely to get flawed data."
Bridgeman predicts the move to integration will grow in the future. "I think it’s the way to go. The huge thrust today is for safe health care, and as the Joint Commission [on Accreditation of Healthcare Organizations] says, the more driven by informatics you are, the safer you will be."
Despite the challenges, systems integration "will create a new bar, and that will only benefit patients," Lawless adds.
However, Stoecklein cautions, it probably will be impossible to ever be sure that you are completely comparing apples and apples. "I don’t know if you’ll ever have all apples, but you will have red fruit and red fruit. People want to find somebody exactly like them, and I don’t think you’ll ever see two hospitals that are exactly the same. But the real value in benchmarking is finding somebody you can learn from — which may be somebody who is not like you at all." n
Need More Information?
For more information, contact:
• Tania Bridgeman, RN, PhD, Director of Clinical Path Development, University of California at Irvine Medical Center. Telephone: (714) 456-3697. E-mail: [email protected].
• Stephen Lawless, MD, MBA, Chief Knowledge Officer, Nemours. E-mail: [email protected].
• Mike Stoecklein, Senior Operations Consultant, Catholic Health Initiatives, 7650 Edinburgh Way, Suite 200, Minneapolis, MN 55435. Mobile Phone: (952) 334-3578.
As the experts will attest, benchmarking is part science, part art. The science of data capture is becoming more sophisticated all the time, although a relatively small number of institutions currently are taking advantage of state-of-the-art, fully integrated systems.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content