Outcomes assessment seen as better indicator of care
Scores given by the joint commission for Accreditation of Healthcare organizations (JCAHO) are a poor indicator of the quality of health care patients actually receive, according to John Griffith, professor of health management and policy at the University of Michigan School of Public Health.
Griffith and fellow researchers recently compared JCAHO-assigned hospital scores to Medicare-based measures of mortality and complications and found little relationship between the two.
"We expected to see good’ JCAHO hospitals get good’ performance scores, be safe, well financed, efficient and progressive," Griffith says. "The data showed the hospitals with the worst JCAHO scores have as good performance as the group with the best. A hospital’s JCAHO score doesn’t predict its outcomes success."
JCAHO, which rates 95% of acute care hospitals in the United States, by scoring four dozen separate activities in hospitals and calculating a weighted overall score before deciding to accredit, was established in 1950. Griffith says the group still bases its criteria on descriptions of processes believed to contribute to effective care written by the College of Surgeons in 1920.
Though the process and structure measures used in accreditation are often acquired cheaper and faster than outcomes information, their usefulness and credibility depend on whether or not they can predict actual outcomes. And JCAHO ratings can be affected by conditions that change once the accreditation inspection is over.
Griffith says that two problems arise between the investigative and rating poles of "do they do the processes" and "what was their score?" The problems are that some institutions spend enormous amounts of energy securing a good score but appear to resume sloppy habits one the JCAHO inspector leaves, and some inspectors are not as thorough as they should be. "It’s easy to get misled," Griffith says. "An institution that wants to hide the truth has more weapons than the inspector does, and an inspector’s job is not really an appealing one."
Outcomes a Better Quality Measure
Outcomes are a far more reliable quality-of-care measure, Griffith believes, and thus should be used to develop a dependable rating system. Both outcomes assessment and a balanced score card assessment are needed. "It’s nice to look at the processes used in surgery to keep patients from developing infections," Griffith notes. "It’s even smarter to look at the actual infection rate."
He adds that it would be smarter yet to see if the institution has the proper financial support to survive, a program in place to develop effective personnel and a sufficient volume of activity to assure its staff remain fully skilled and practiced.
In gathering data, Griffith used seven measures selected from Medicare reports by Solucient LLC of Evanston, IL, which were widely recognized as reflecting important attributes of performance and developed using broadly accepted definitions and adjustments. The measures were also essentially independent of one another, reflected a wide range of actual performance and had substantially lower year-to-year variation for the same hospital than cross-sectional variation across hospitals.
Griffith concluded that a disjuncture exists between the outcomes measures used and Joint Commission hospital performance evaluations. He notes that the Solucient measures of quality are conceptually similar to the aims of the Institute of Medicine, and adds that his data indicate that hospital and community characteristics do not substantially alter mortality and complication scores.
Griffith says the first step in successfully revising the accreditation process may well be studying successful practices, adding that it’s now possible to differentiate hospitals with consistently good outcomes from those with improving outcomes and those with recurring weak performance. He points to work done by the National Quality Forum (NQF) and LeapFrog Group as the best direction for developing a new system.
He particularly likes the NQF’s consensus report, A National Framework for Healthcare Quality Measurement and Reporting, which identifies 17 NQF-endorsed principles and policy statements that comprise the beginning of a national framework for improving the quality of U.S. healthcare through measurement and reporting. The draft report is available for review at http://www.qualityforum.org/hospreport.pdf.
(For more information contact John Griffith at (734) 936-1304.)
Suggested Reading
1. Griffith J, Knutzen S, Alexander J. Structural versus Outcomes measures in hospitals: A comparison of joint commission and medicare outcomes scores in hospitals. Quality Management in Health Care. 2002;10 (20):29-38.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content