Where research and reality collide
Measurement and science's role in QI
Checklists have become a ubiquitous term for the patient safety movement, which most recognize as being born with the Institute of Medicine's 1999 report "To Err is Human." With books like Atul Gawande's "The Checklist Manifesto" hitting bookstores' prime shelves, public attention on health care and errors has become heightened. And with health care reform on the political horizon, health care from the inside and out is being scrutinized like never before.
Checklists illustrate the need for standardization. But according to patient safety guru Peter Pronovost, MD, PhD, who has published a new book "Safe Patient, Smart Hospitals: How One Doctor's Checklist Can Help Us Change Health Care from the Inside Out," the concept is not as simple as it sounds and represents a broader need for measurement, data, and standardization in health care. He says safety and real culture change in health care is not just about checklists
Pronovost is an intensive care specialist physician at Johns Hopkins Hospital; professor at the Johns Hopkins University School of Medicine in the departments of anesthesiology and critical care medicine, and surgery; and medical director for the Center for Innovation in Quality Patient Care. He created a five-step checklist for central line insertion that helped to curb infections not only at Hopkins and as part of the Michigan Keystone initiative but internationally with other hospitals picking up on his work.
"The story of a simple checklist has eminent appeal," Pronovost says. "I really want to show that it's much more than that." He likens the checklist to the New Year's resolutions we all make, every year lose weight, exercise more, make more money. How many of those well-intentioned goals move to the wayside as more pressing issues come to the forefront? He wants the health care field to understand that a checklist means nothing if you don't measure the success of the intervention at hand. "We're 10 years after 'To Err is Human,' and we don't have a lot of empiric data, virtually no empiric data, that patient outcomes are better. I think that's quite sobering, and if I think about what we've done, our efforts and there have been a lot of them have been more competitive than cooperative, more independent than interdependent, and far too focused on efforts than on results."
In addition to measuring quality improvement interventions, Pronovost says the health care industry must tackle the tendency to put "ego ahead of safe practices" and empower staff to hold each other accountable for following protocols and doing the right thing.
A checklist, he says, is not Harry Potter's wand. It's not a magical tool guaranteed for quick fixes. Yes, it stands for standardization, something the health care industry needs, but the necessary cultural changes are far harder to achieve. The checklist informs you of what to do, but Pronovost looks at the barriers to complying. What is most important to changing outcomes is measurement and empowering staff to speak up when things are wrong, he says.
He says he often hears the phrase "these data are for quality improvement, not research." But he says "do the data know any difference? You're making a public statement that care is better. There's certain rules of measurement that you have to adhere to, and I think the quality improvement field has been too loose."
But the Institute for Healthcare Improvement's director Fran Griffin, MPH, says Pronovost is a researcher and that is a difference between being a quality improvement director in the hospital. What IHI is doing "is not research. We're not trying to create new knowledge," she says. "So for example, if you take the central line bundle, nothing in the central line bundle is new. The five elements of the central line bundle, four of them have to do with insertion and have evidence to back them up. So we're not out there trying to figure out what's the new technique for central lines that needs to be studied. We're just saying we want to learn how we're going to make that process work here. So the what has already been established. The focus is on the how."
Pronovost says, "To be fair, many of my academic colleagues are too rigorous. They would never get off the dime if we did every study the way they would want to do it because health care is often practiced in the messy real world where I can't control everything. So part of what I'm trying to do in this book and in my work is bridge the research world that often needs to loosen up a little bit and the quality improvement world that needs to tighten up a little bit and end up at some place in what I call the 'sweet spot,' the middle, where the measurement and science is good enough that I can look at a patient or you and say with confidence, 'Hey, things are better.'"
The problem he sees is that often national policies are made without a scientific understanding of their validity. He wrote a piece in the December 2009 issue of Academic Medicine "Perspective: Physician Leadership in Quality" showing that most physicians who become quality leaders "went into it because they had nothing else to do. It wasn't a destined career track where they had the formal training and the skills necessary to do this well. It was, 'Well, this is a nice guy or a senior clinician, let's give this to them.'" That's something health care quality needs to move away from, he says. Certain skills are required to be an effective quality leader, he says measurement skills, leadership skills, and human factors engineering skills.
While Griffin agrees that you shouldn't say a measure is working without measuring compliance, she says quality improvement directors can obtain those skills, either by getting their organization to fund additional training or by going out and doing it on their own. She says she's met QI directors who were great in their role and QI directors who weren't, and she's sat in on quality meetings where the measurements were not in line with what the team was trying to measure. They recognized the need to measure, she says, but didn't know which variables to choose to get the answers they wanted.
Griffin, who worked in quality before joining the IHI, says QI directors "don't need to be statisticians. This is an area where I see people sometimes making it harder on themselves than it needs to be... [For quality improvement directors, they're] not designing a randomized control trial that you're trying to get published in the New England Journal of Medicine. It doesn't need that level of rigor."
She says there has been much debate about the data of quality improvement data vs. the data of the randomized control trial and acknowledges that it's tough for quality improvement directors to speak data with physicians who are schooled in the scientific speak of the randomized control trial. "If you put a run chart showing data over time up on the wall, you don't need to be a statistician to interpret whether it's going up or down or jumping all over the place... With data for learning, the thing [IHI] often tells people is you just need enough [data] to know. You don't need a year's worth of data statistically analyzed to know whether or not something is getting better. Go ask the frontline nurses. They'll tell you after a week. So we emphasize that."
Robert M. Wachter, MD, is professor and associate chairman of the department of medicine at the University of California, San Francisco; chief of the division of hospital medicine and chief of the medical service at UCSF Medical Center; and health care blogger. He says measuring a process before you fix it and then after you fix it is "fundamental to quality improvement."
"Organizations that don't build that in," he says, "who just do things because they seem like they may help, usually end up getting burned. Changes don't stick."
But he says quality improvement today is not "your father's quality improvement. It's a much more nuanced, interdisciplinary, iterative process where you begin by identifying all aspects of the problem in a new way and you focus on changing systems of care, not just shaming people for not doing something."
Though everyone, he stresses, is trying to do the right thing, "IHI has tended to have a philosophy in which anecdotal evidence may be enough to catalyze a recommendation to change a given process." This may move things along faster but means you may be doing things that make sense but don't work and you've committed resources to it, he says. He, like Pronovost, supports balance between the scientific world of data and the quality improvement realm. Wachter says if a clinical study comes out saying tight insulin control is a good idea, it can take years for clinicians to implement it. "Whereas in safety and quality, because of the regulatory and accreditation environment, there often is a very, very short path between an organization like IHI or others coming out and saying, 'We think this is the right thing to do...' So the pressure to do it grows very quickly."
The checklist concept, he says, has become "useful shorthand" and often misunderstood. It's not just a list with boxes on it. It implies:
- collaborative and evidence-based efforts to codify and settle on certain best practices;
- standardization of those practices;
- thinking in terms of human factors engineering and deciding what elements should not and should be on the checklist;
- roles and accountability.
"Not only did I not learn anything about checklists, systems, process change, change management, human factors engineering in medical school, I learned things that actually set me back in these efforts," of looking at patient safety in a team environment, Wachter says. "I learned about being really smart and clever, an individual free agent who's practicing like House."
But in becoming involved with patient safety and quality improvement, he finds himself asking: "What is the role of standardization in a field that has resisted that? What is this thing called human factors engineering? What is the role of teamwork? What is the role of leadership and expertise?" The patient safety field is "still relatively new," he says.
Checklists involve "a level of teamwork and role allocation that's critical. It involves embedding a checklist in a culture that allows the checklist to do what you need it to do and sometimes conversely, the checklist is a culture-changing activity," he says.
Checklists have become a ubiquitous term for the patient safety movement, which most recognize as being born with the Institute of Medicine's 1999 report "To Err is Human."You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content