Ready to answer ORYX data questions? Here’s what Joint Commission will ask
Fail to report for 2 quarters, receive automatic type I recommendation
In the near future, you’ll be answering surveyors’ questions about ORYX, so you need to prepare by collecting data, analyzing the results, and using them to compare your ED with those in other facilities, says Stuart Shikora, MD, FACEP, a surveyor with the Joint Commission on Accreditation of Health care Organizations in Oakbrook Terrace, IL, and an ED physician at Mount Diablo Medical Center in Concord, CA.
Your hospital already should have chosen six measures to report for the ORYX program, which integrates performance measurement into the Joint Commission’s accreditation process. If the indicator involves the ED, you may be asked in-depth questions about the collected data.
"[The measures] are supposed to reflect the scope of practice and important issues at that institution," says Shikora. Many of the measures affect the ED, he adds. Those measures include intrahospital mortality of trauma patients with systolic blood pressure of less than 70 within two hours of ED arrival who did not undergo a laparotomy or thoracotomy, explains Kathleen Catalano, RN, JD, senior consultant to the Greeley Co., a health care professional consulting firm in Marblehead, MA, specializing in regulatory compliance. Another indicator is trauma patients undergoing selected neurosurgical procedures: time from ED arrival to procedure, she says.
If your hospital chooses indicators that affect the ED, it can be a boon for your department. Being involved with ORYX measures can demonstrate the ED’s value to the hospital, Shikora emphasizes.
"Not all EDs are liked by the other departments," he says. "With ORYX, you can do something that might lead to better patient care and demonstrate improvement for the hospital."
The Joint Commission now has more than a year’s worth of data on ORYX measures for hospitals, so surveyors might ask you to explain how those data have been collected, analyzed, and used for comparison, Catalano notes.
The data that have been sent to the Joint Commis sion are reviewed by surveyors and discussed during the performance improvement and leadership interviews during surveys, explains Catalano. "Also, if the data were truly bad, the Joint Commission could send surveyors to the facility, unannounced, to find out what was being done to rectify the identified problem," she says.
If any of the performance measures chosen by your hospital involve the ED, you’ll need to answer questions about them during your next Joint Commission survey, says Shikora. (For a list of ORYX indicators and performance measures, see pp. 41-42.)
The following are some questions Joint Commis sion surveyors will ask:
• Did you collect data in a systematic way over 12 or more months? Are you continuing to collect data on the same measures, or have you switched to other issues?
At this time, surveyors are not scoring the quality of the data, and they won’t make judgments that you did the right or wrong thing, says Shikora. "At this point, the goal of the surveyors is to see that there is some data collection, and if data quality is an issue, that the health care organization is doing something to correct the problem, as well as how the data is being analyzed and used by the organization," he says.
• Was appropriate statistical methodology used?
Surveyors will want to see that you are using statistical methods and that your decisions regarding quality improvement are data-driven, says Shikora. "They will want to know if you recognized a trend, and were there performance outliers? Did you adjust the way you collect data, and did you change the way you care for patients?"
• Do you have comparison charts?
Your selected performance measurement system for ORYX is required to provide you with comparison feedback information quarterly. Surveyors will want to see how you used those comparison feedback data charts and any supporting data, says Shikora. "Com par ison charts allow an organization to compare itself to other institutions looking at similar measures," he explains. "For example, do you have more patients who left without being seen than a comparable hospital?" (See sample of a Joint Commission comparison chart, p. 39.)
The facilities also need to be comparable in size and scope of services. "A 1,000-bed hospital with broad spectrum of services may not accurately be compared with a 100-bed hospital with a narrow scope of services," says Shikora.
The ED should provide data that have been compiled by the quality improvement department or the contracted ORYX analysis service, he notes.
• Do you have control charts?
These are charts representing internally acquired data. (See sample of control chart, p. 39.) Surveyors will get much of this ORYX information before they arrive on the scene, he says.
"They will have a list of measures, the data your organization has submitted, and the Joint Commis sion’s statistical analysis," he says. "They will know if there has been a lapse in reporting, which you should be able to explain."
However, surveyors won’t know what decisions you’ve made based on the data and analysis. "That’s for your teams to present at the survey," Shikora explains.
The reporting is a new development for the Joint Commission, he says. "It used to be that surveyors had no idea what the facility was doing until they showed up and listened to your performance improvement presentation."
But as of this year, surveyors will be receiving the ORYX data several weeks in advance, Shikora says. "This will give the surveyors an idea of what the hospital is looking at, what its measures are, and whether data has been reported regularly."
• Have you reported data according to schedule?
An automatic type I recommendation results if an institution fails to report data for any measure for two quarters in a row, Shikora warns.
• Can you explain the data reported?
If an ED measure was chosen by the hospital, you probably will be asked to explain the data that were reported, he says, noting that the ED may be asked following questions about reported data:
— Why was the measure chosen?
— What did you learn by studying it?
— Can you explain the data that were generated?
— Who analyzed the data, and how were they trained?
— What were the results?
— What trends were identified?
— To whom were the data reported?
— What improvements were made in the system of patient care based on the data and its analysis? (See list of additional possible surveyor questions, p. 41.)
At the physician leadership meeting, ED physicians might be asked to describe their involvement in performance improvement activities, notes Shikora, "so they need to be familiar with the concept of control and comparison charts and ORYX data. They also need to demonstrate they were involved in the choice of the measures from the hospital. If an ED measure was chosen, explain why it was chosen and how it was analyzed."
• Do you understand the difference between clinical significance and statistical significance?
"There are many findings on control charts that are statistically significant, but not clinically significant," he explains. "You may look at a control chart and say, the ED data was beyond the expected range for a month,’ but it might not be clinically significant."
For example, patient returns to the ED within 48 hours might be much higher one month, but there might be extenuating factors. "Those patients may have been asked to come back to the ED because they had wound checks and the outpatient clinic was closed for renovations," Shikora says. However, a good measure likely would exclude patients returning to the ED for a scheduled visit, he adds.
Surveyors are attuned to this, he stresses. "They are looking for quality improvement activities just as before, only now they will be looking for a systematic, statistical approach to data collection, analysis, and decision making. They know there is a difference between statistical and clinical significance, so do not be worried about showing data to surveyors."
Still, ED staff who compile ORYX data need to interpret them and explain whether items are clinically significant, he notes. Everyone is afraid to be a performance outlier, he points out: "The ED needs to know that having an outlier performance is not necessarily bad. It may mean you did exceptionally good work."
However, when core measures are being used (as early as 2002 or 2003), the information gleaned from performance measures may be considered more rigorously by surveyors in scoring standards, says Catalano. "In the future with core measures, when data is recovered and analyzed by the Joint Commission, if something is noted that may be detrimental to patient care, the Joint Commission may perform an impromptu visit to the facility and analyze their process."
• Kathleen Catalano, RN, JD, 200 Hoods Lane, Marblehead, MA 01945. Telephone: (781) 639-8030. Fax: (781) 639-0085. E-mail: kathijoe@worldnet. att.net.
• Stuart Shikora, MD, FACEP, Mount Diablo Medical Center, Department of Emergency Medicine, 2540 East St., Concord, CA 94520. Telephone: (925) 258-0013 or (800) 838-6140. Fax: (925) 258-0014. E-mail: [email protected].
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content