Does your data answer the 'So what' question?
Make sure your info means something to providers
By Patrice Spath, ART
Consultant in Health Care Quality
and Resource Management
Forest Grove, OR
(Editor's note: This is the first article in a two-part series on helping providers with information that means something to them. In this issue, Patrice points out the flaws with some performance measures and shows how to put meat on them. In the February issue, she explains how inferential tests can be used to compare outcomes of a particular patient population to reference values and subgroups in the population.)
Hospital and health networks are data-rich, but caregivers in those settings may be information-poor. It is common to see committees reviewing quality measurement reports packed with data. Yet, the data may add little to clinicians' understanding of the patient care process. Caregivers leave the committee meetings without knowing whether current practices are resulting in desirable outcomes. The reason for this phenomenon is simple: The data do not answer the "So what" question.
Measures of performance are data that assist the health care evaluator in assessing the quality of the services. Those measures should provide information about the structure of health care services, the process of health care delivery, the appropriateness of the health care decision, and the outcome that resulted from the health care intervention. Each of those components of health care must be evaluated to obtain a clear picture of quality. The presentation of performance measurement data traditionally has been limited to monthly reports of measure results. The data displays may have included a table of numbers or merely the raw numbers for the month, followed by a comparison to any or all of the following:
* previous month's measure results;
* results from the same month, last year;
* an average value of the previous year;
* a national average;
* a threshold arbitrarily defined by the organization.
Such presentations and comparisons have limited interpretive value. They may, in fact, cause flawed interpretations. Listed below are some common performance measures of a hospital's obstetrical service:
Total cesarean rate = No. of cesareans total deliveries
Primary cesarean rate = No. of primary cesareans total deliveries
Repeat cesarean rate = No. of repeat cesareans total deliveries
Vaginal birth after cesarean (VBAC) rate = No. of vaginal deliveries for patients with previous cesarean
total No. of deliveries for patients with previous cesarean
Performance measures such as these are traditionally displayed as percentages in a table or graph such as the data displayed in the chart below. Comparisons with the organization's historical average or other "thresholds" may accompany the raw numbers.
Performance measurement reports, like the one shown in the traditional performance measurement report, provide caregivers with incidence data (for example, the number of patients undergoing VBACs) but doesn't answer the question: "So what if our VBAC rate was 34% in November?" Several issues must be considered when producing performance measurement reports that provide an answer to this question.
First, the performance measure should tell caregivers something about an important aspect of care that affects patient outcomes. Practitioners are not interested in knowing about insignificant issues that have little or no influence on the overall quality of patient care services. Listed below are some commonly found performance measures for various hospital and medical staff departments. While the data for these measures may be easy to collect, the results do not answer important questions. Included with each measure are the quality questions that remain unanswered because the measurement, when reported by itself, wrongly focuses on less important aspects of health care services.
Performance measure: Percent of rehabilitation discharge summaries completed within 24 hours of patients' hospital discharge.
Unanswered questions: Did the patient meet the treatment goals defined by the therapist? If no, was the treatment plan changed to reflect the lack of progress?
Performance measure: Number of STAT X-ray orders completed within two hours of receipt of order.
Unanswered questions: Was the STAT order appropriate or could the X-ray have been completed in a less timely (and less costly) fashion with the same treatment outcome?
Performance measure: Mortality rate for patients in the medicine department.
Unanswered questions: How many unexpected deaths occurred? What percentage of the deaths represent suboptimal patient care practices as determined by peer review?
Performance measure: Number of unexpected returns to the intensive care unit.
Unanswered question: How many patients should have returned to the intensive care unit but were not transferred back to the unit?
Performance measure: Number of patient care plans completed within 24 hours of the patient's admission.
Unanswered questions: How many of the care plans were individualized for the patient's particular problems? What percentage of the care plans were updated as necessary and followed throughout the entire patient stay?
Performance measure: Number of breaks in sterile technique in the operating room.
Unanswered questions: How many sterile-technique problems were preventable? What was the impact of improper technique on the outcome of patient care?
Performance measure: Number of patients transferred from the emergency department to another hospital.
Unanswered questions: Were all patients stable at the time of transfer? Did the physician communicate necessary information to the receiving facility? Did the receiving facility accept responsibility for the patient? Was the mode of transportation appropriate?
Performance measurement data should address the more important quality questions; otherwise caregivers may find it of little value. Report information that answers the more important question, "How are we doing in those areas that ultimately have a positive or negative impact on the outcomes of patient care?"
Incidence counts of compliance/ noncompliance with an important aspect of care are not as motivating as data showing the impact of compliance/noncompliance. In the chart on p. 15, the top graph reports the percentage of patients who received preoperative, in-home education prior to their total hip replacement. Compliance with this aspect of care is shown to range from 30% to 55%.
The incidence data, while useful, do not answer the "So what" question. That is, "What difference does it make if patients do not receive this preoperative education?" The bottom graph reveals the impact of noncompliance. As more patients are educated preoperatively, a larger percentage are able to ambulate within 24 hours post-surgery. *
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content