More robust evaluation required by CMS’ Seventh Scope of Work
The latest requirements from the Centers for Medicare & Medicaid Services (CMS) call for more detailed evaluation of processes than ever before, causing one health care system to look to industry for the tools needed to respond appropriately.
Overlook Hospital in Summit, NJ, part of Atlantic Health System, is using computer simulation as an integral part of its process improvement in community-acquired pneumonia (CAP) and cardiac care.
"Within the past few months, CAP antibiotic cycle time has developed a narrower window for the CMS Seventh Scope of work than for the sixth — from eight hours down to four," notes James Espinosa, MD, FACEP, FAAFP, chairman of Overlook’s emergency department (ED).
"But more than that, the Seventh Scope of Work talks about 90% of the patients receiving antibiotics within four hours. This is no longer a statement about central tendency; it really states something about variation and distribution. You could, for example, achieve an average or a median of less than four hours but still not be compliant with the 90%; it’s a much harder test," he says.
The key question for Espinosa then became — how could he gain a sense of assurance that an improved process would behave not only in an average time of fewer than four hours but 90% of the patients would be treated in less than four hours?
"The traditional way would be to brainstorm best practices; look at what we have done in the past; make an intervention; and if the tendency is below four hours [but you don’t have the 90%], that’s good, but you still may need to retool. The tough thing in those situations if you do not perform at 90%, is the question becomes whether the design is not robust enough, or you simply have not implemented it enough — i.e., your people have not adapted to the new process. This takes several more months, so it can be four or five months before you have a decision as to whether the design is good enough," he continues.
The bottom line was that Overlook concluded it was unable, using traditional methods, to estimate the capability of the improved process in terms of the fraction of patients who would receive the first dose within four hours of admission. "For that reason, we decided to work with computer simulation," Espinosa says.
For help with the project, he turned to Twin Peaks Group LLC of Sherborn, MA, an operations management consulting firm with expertise in product development process and operations flow management, supply chain management, and financial impacts.
"We brought our experience from the industrial world to the health care field," says Shashi Sathaye, PhD, a Twin Peaks principal, who began working with Overlook in February 2003.
"In discussions, we thought this may be an opportunity to apply computer simulation, which we have done for product design in industry," he says. "In terms of treating pneumonia, it’s similar to process design in that you’re coming up with a new cycle time." But the CMS shift from a mere target time from ED admission to first dose of antibiotics to a percentage of patients falling within that target range "was a significant change from our perspective," Sathaye adds. "Now, the tail of the distribution — the outliers — could not be more than 10%."
This called for a tool to determine what is referred to in industry as process capability.
"When people design and improve processes, we just don’t know how much they will improve," explains Dan Krupka, PhD, managing principal of Twin Peaks. "Process capability answers the question, What’s the fraction of people that get the right meds within a specific amount of time?’"
To create the simulation, the process was broken down into major steps, such as, "CXR performed" and "MD confirms pneumonia, evaluates patient, and orders meds," and detailed steps, such as, "Wait for triage nurse," and "Wait for X-ray technician." Time distributions for the new process steps were based on ED nursing staff estimates.
"We knew from the data we had what our current process was yielding; it was in the low 60s," Krupka notes. "If we broke down the process into several steps, we could ask the people who live’ each of these steps to estimate the time distribution of each so we could get a triangular distribution. In other words, what’s the least amount of time you’ve seen this happen? What’s the longest? What’s the most likely? It’s not perfect, but it’s a good way to go. If you ask people about each step and use the input for simulation, you can get the process capability."
"This type of data is much more reliable — much better than just moving through from the time of admission to the time the patient leaves," Sathaye says. "What the computer simulation does is take [the minimum, maximum, and most likely] small subprocess times and throws a dart to come up with a time."
That process is repeated for thousands of simulated patients for each of the subprocesses. "Then you get a histogram for patients to go through the entire process," he continues. "You then compare that to the distribution you got with the historical data."
Based on the simulation, the new process was far superior to the old process — a 75% process capability vs. 62%. However, there was a nagging question: Just how accurate were the nurses’ estimates? To ascertain just how good the nurses were at estimating, the Twin Peaks team had them estimate distributions for the old process steps and compared that to the historical data; their success in these estimates lent further credence to their estimates of time distributions for the new process steps.
This approach has several advantages over more traditional PI methods, Sathaye adds. "Speed is, of course, one advantage. Second, you can actually visualize the flow; you can have a moving picture, have patients go through the process, depict interactions of doctors and nurses. You can convey to all the stakeholders what it means to them."
"One of the good things about simulation software today is there are better graphics," Krupka explains. "You can show people on the screen and how things are moving." Different icons can be designed for beds, X-ray machines, physicians, nurses, patients, and so on.
"You can see all of them move around — even see queues forming because something did or did not happen," he points out. "You see the patient come in, see the queue growing, and visualize the average wait time. Then, for example, you can put in a second X-ray machine and see what happens."
The final advantage, according to Sathaye, is reliability. "You can take the model and actually set up measurements of each of the individual processes. By comparing the nurses’ best estimates with what actually happened historically, you get a certain comfort level. Without it, it’s hard to take a leap of faith."
In this particular case, he notes, they got a 90% to 95% confidence level from the simulation.
The computer simulation efforts have had a significant impact on Overlook’s PI efforts, notes Tina Maund, MS, RN, CPHQ, director, performance improvement.
"The most important impact from my viewpoint is that it allows very detailed review of process substeps and precise analysis of related time intervals," she observes. "This has allowed highly objective and detailed study of process substeps and has supported cycles of work based on clearly defined priorities for action — i.e., we make some process changes, examine the impact, and if improvement in that area is at an acceptable level, we move on to another priority focus area for changes within the process."
In addition, Maund explains, the simulation model building requires that her staff estimate the time impact of proposed process changes and then evaluate the actual time impact of the change.
"This provides interesting insights into the expected process function vs. the reality," she points out. "In some instances, this has revealed greater impact than expected and in others, the impact has fallen short — leading us to restudy the process substeps involved and to strengthen the redesign."
Another interesting aspect of the computer simulation relates to potential application to failure mode and effect analysis (FMEA), Maund explains.
"That process now requires an estimate of frequency re: process failures," she says. "We plan to experiment with linking the simulation estimates of frequency of process failures — re: events that exceed target times for specific actions — with FMEA analysis and see how this impacts the risk priority numbers for that process. We expect that this linking will give us a more accurate risk priority number," Maund notes.
"This seemed custom-fit to what we needed; it’s been very energizing and helpful," Espinosa adds.
"If nothing else, it sends the staff a very strong signal that something very special is in the air, and that we must care to be doing this." In the future, he adds, Overlook will apply computer simulation to admission-cycle time as well as to some aspects of door-to-balloon time. "We’re doing pretty well, but I’d like to see if we can trim a little more time off the process."
Need More Information?
For more information, contact:
• James Espinosa, MD, FACEP, FAAFP, Chairman, Emergency Department, Overlook Hospital, 99 Beauvoir Ave., Summit, NJ 07902-0229. Phone: (908) 522-5310.
• Dan Krupka, PhD, Managing Principal, Twin Peaks Group LLC, 60 Whitney St., Sherborn, MA 02770. Phone: (508) 647-9303. Fax: (413) 403-5374. E-mail: Dan-Krupka@twin-peaks group.com.
• Tina Maund, MS, RN, CPHQ, Director, Performance Improvement, AHS/Overlook Hospital. Phone: (908) 522-4912. Fax: (908) 522-5315.
The latest requirements from the Centers for Medicare & Medicaid Services (CMS) call for more detailed evaluation of processes than ever before, causing one health care system to look to industry for the tools needed to respond appropriately.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content