Doubt cast on numbers quoted in IOM report
Organization did not follow scientific standards’
When the Institute of Medicine (IOM) talks, people listen. That’s why when the organization released its report on medical errors, which stated that between 44,000 and 98,000 hospitalized American die each year as a result of preventable medical errors, quality managers sat up and took notice. Being so widely respected places a heavy responsibility on organizations such as the IOM to prepare their reports in a detailed, scientific manner. In this case, say some observers, the IOM fell short.
"It was ironic that an article on errors itself contained errors," notes Harold C. Sox Jr., MD, chairman of the department of medicine at Dartmouth- Hitchcock Medical Center in Lebanon, NH, and co-author of an article in the November/ December 2000 issue of Effective Clinical Practice.1 In the article, Sox and his co-author state, "The methods used to estimate the upper bound of the estimate [98,000 preventable deaths] were highly subjective, and their reliability and reproducibility are unknown, as are the methods used to estimate the lower bound [44,000 deaths].2 "Due to the potential impact of this number on policy, it is unfortunate that the IOM’s estimate is not well substantiated," they continued.
Uncharacteristic of the IOM
Sox is quick to note that the report did not reflect the kind of work the IOM typically does. "This did not measure up to its usual standards. I have chaired several IOM committees, and I know the high standards the organization holds itself to." Nevertheless, he says, "The IOM committee didn’t follow the expected scientific standards of documentation of its findings. In my opinion, [the committee] should have published a precise description of how it calculated those numbers, where they came from, and if it made assumptions, a description of those assumptions."
Sox’s second concern was about the scientific standards used by the authors of two major IOM-cited articles on the number of errors. "They pointed out that in trying to make the case about the number of errors, they used excellent methods to make sure those numbers were as accurate as possible, but when it came time to deciding on the number of deaths that would not have occurred were it not for medical errors, they did not apply the same rigorous standards. It seemed more like educated guesses," he notes.
An argument among academics’
Sox says he is concerned that his article could overshadow the basic validity of the IOM report. "This was an argument among academics. Our focus was the process that was used to come up with the numbers of deaths that would not have occurred except for errors. My article was about a failure of scientific process, and my biggest concern was that my paper might distract from the main message of the IOM report. Whether the correct number is 44,000 or 4,400, it’s too high, and as long as there are opportunities to reduce the number of medical errors, the profession has an obligation to improve those systems that cause the errors."
The major accomplishment of the IOM report, he says, was to present a convincing case that errors in health care usually are the result of failures of systems rather than the failure of an individual. Therefore, efforts to reduce errors have to focus on improving the systems that allow health care professionals to provide health care. "In pointing out what I have about the report, I do hope the people don’t misinterpret my agreement with the general thrust of the IOM report," says Sox.
Nevertheless, the errors in a report from such a prestigious organization do present a cautionary tale for quality professionals as they read the literature: Make sure the authors did their homework. "If an organization is issuing a report that presents some numbers of events that by their nature are going to have an inflammatory effect, it has an absolute obligation to document how it got its numbers," Sox concludes.
Reference
1. Sox HC, Woloshin S. How Many Deaths are Due to Medical Error? Getting the Number Right. Effective Clinical Practice November/December 2000; 277-282.
Need More Information?
For more information, contact:
• Harold C. Sox, Jr., MD, Department of Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, NH 03756. Telephone: (603) 650-7684. E-mail: [email protected].
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content