Does your organization have a culture of safety?’ Here’s how to find out
JCAHO will want to know your findings
Would you like some compelling evidence to present to administrators about a chronic safety problem? Do you want a foolproof way to achieve staff buy-in when a new patient safety initiative is implemented? How about finding out what staff really think about reporting safety concerns?
Doing a patient safety culture assessment can do all of these things for your organization, says Joann Sorra, PhD, of the Rockville, MD-based Westat Group, which helped to develop and test the Agency for Healthcare Research and Quality (AHRQ)’s Hospital Survey on Patient Safety Culture tool.
Although conducting a self-assessment of safety culture currently is optional, the Joint Commission is expected soon to require organizations to do this. "This is part of the draft standards, subject to approval, for the new leadership standards, targeted for implementation in 2007," says Richard J. Croteau, MD, JCAHO’s executive director for strategic initiatives.
"It’s clear, across the full spectrum of health care in this country, that organizational culture is not consistently supportive of patient safety," adds Croteau. "If we can accept that statement, then we need to somehow change the culture to be more supportive of safety. If you are going to change something, you have to measure it and know if you are improving."
The results of a patient safety culture survey will help you identify and prioritize areas to focus improvement efforts based on the input of hospital staff, Sorra says.
"A staff perspective is important, because it reveals what both clinical and non-clinical staff are concerned about in terms of patient safety," she explains.
Although other types of activities can be used to identify where to focus patient safety improvement resources — such as root cause analysis, patient safety rounds, patient safety indicators, and event reports — the self-assessment method is unique because it is based directly on input from staff, Sorra notes.
Until recently, there was no evidence-based way to assess culture, but now several validated tools are available. In addition to the AHRQ tool, organizations are using the Safety Attitudes Questionnaire and Safety Climate Survey developed by researchers at the University of Texas Center of Excellence for Patient Safety Research and Practice, the Patient Safety Assessment Tool developed by the VHA’s National Center for Patient Safety, and the Patient Safety Climate in Healthcare Organizations (PSCHO) survey tool, developed by Stanford University’s Consortium for Patient Safety.
At the Johns Hopkins Hospital & Health System in Baltimore, safety culture is measured throughout the entire organization using the Safety Attitudes Questionnaire.1 "Each department and nursing unit receives their results benchmarked to other hospitals," says Peter J. Pronovost, MD, PhD, medical director at the Center for Innovations in Quality Patient Care at Johns Hopkins University School of Medicine. "We use these measures as part of our safety scorecard."
The goal is to have 80% of staff in all units report positive safety and teamwork climate, says Pronovost. "For nursing units less than 80%, we implement a comprehensive unit-based safety program," he adds.
To be effective, the survey should be given organizationwide, stresses Croteau. "One of the things we learned is that there isn’t a single culture in any organization," he says. "There is considerable variation from one unit to another. So you need to implement the assessment process at the unit level, since a problem may exist in a single unit."
While some organizations already are doing self-assessments to identify problems, others are far behind in this process, Croteau says. "Some are right out in front of the curve, but others just feel it’s one more onerous burden being put on them by external forces. That’s the range of organizations we are dealing with."
There is growing evidence that self-assessments can affect patient safety, with one study finding that safety climate improved in specific survey areas for 26 hospitals in a California hospital consortium. However, researchers also discovered wide variation in culture among hospitals.2
"The main implication is that while most employees view safety culture in hospitals as pretty good, there remains significant room for improvement," says Sara J. Singer, the study’s lead author and a senior research scholar at Stanford University. When the researchers compared the California hospitals to naval aviation, they found that on average, safety culture among naval aviators was three times stronger, Singer notes.
"The key to the effectiveness of our survey method is that we provided benchmark data for our hospitals," says Singer. "They could compare themselves not only to an acknowledged high reliability organization — navy aviation — but also to the other hospitals in our study. This enabled them to identify areas of relative strength and also specific areas to target for improvement."
"We’ve got to attack the culture of the entire institution, rather than a piece here and a piece there," says Kevin Tabb, MD, chief quality and medical information officer for Stanford Hospital & Clinics. "We’re doing that in a number of ways, allowing us to look at every nook and cranny of the hospital to find out how we are doing in relation to where we should be. It’s been a very good process for us."
Asking everyone for input
Completing the self-assessment required by the Joint Commission’s Periodic Performance Review (PPR) is one way of doing this. "But you are then only speaking to a certain segment of your employee population," says Tabb. "You never know where you are going to get a good idea from. We’ve gotten important tips from transporters about patient safety."
The organization considers the results of its PPR findings, along with a broader random sample of employee responses, gathered annually using the PSCHO tool. "We have had a very good response rate. What I’ve found is that internally people are very honest and open to coming to the table with where we need improvement," says Tabb. "I am constantly surprised by the good suggestions that we get from all over the organization."
The culture survey is "another tool in our arsenal," in addition to the PPR, use of electronic patient safety software, and participation in national benchmarking programs, says Tabb. "In some ways, this is similar to other industries, such as automobile companies going down the line and asking people what they think, instead of just management," he says. "When you talk about culture, it has to permeate every aspect of the organization."
At Northwestern Memorial Hospital in Chicago, a survey was developed and administered in 2002 to assess patient safety culture. In December 2004, the hospital was going to repeat the administration of that survey, but right around that time got word that the AHRQ-validated survey was available. "So we changed our plans and decided to use the AHRQ survey instead," says Marilyn K. Szekendi, RN, MSN, APRN, BC, research nurse coordinator for the patient safety team, part of the organization’s division of quality and operations.
The tool was pilot-tested to see if staff thought it was too long or if any of the questions were confusing, but no such problems were identified, so the organization decided to use it.
Upon administering the survey, it was found that survey items referring to one’s "supervisor" didn’t apply well to physicians, notes Szekendi.
"That is not a meaningful term to most physicians in the context of their professional responsibilities," she explains. "We realized that the survey is really meant mostly for care providers, but we decided to send it to all employees and did get responses from non-direct care providers. It was good to see what the general culture is, if non-clinical staff feel they can speak up about improvements in their own departments," she says.
In December 2004, the hospital’s information systems (IS) department prepared a web-based version of the survey so it could be sent out electronically to more than 6,000 employees, plus nearly 2,000 medical staff and residents.
"We got back a tremendous response of 28%, with over 1,600 usable surveys returned," says Szekendi. "We had it online for about six weeks, and it was e-mailed to everyone. We sent out two reminders, and with each reminder we got a nice surge in responses."
Hand-off communication
At Indian River Memorial Hospital in Vero Beach, FL, staff completed the AHRQ survey in October 2004, with a total of 617 responses and a 57% response rate.
"One of our IS folks built us a program so we could give the test online, and we scheduled time for departments to use the computer training room to complete the survey," says Barbara Horne, RN, vice president and chief nursing officer. After the survey was completed, a coupon for a free meal in the cafeteria was printed out as an incentive.
The system gave real-time results, so that Horne could see how many staff had responded at any given time. "I was able to call a particular manager to say, Only two of your staff members out of 50 have responded to the survey. Please encourage people to take part in this,’" she says.
The biggest concern identified involved handoff communication — staff were worried that information was falling through the cracks as patients were moved from one area to another.
"We have addressed that in a number of different ways," says Horne. For example, surgical floor nurses receiving patients from the post-anesthesia recovery area were uncomfortable with the handoff process. "We had those two groups sit down together to outline the conditions that a transporter can be used as opposed to a nurse, and what exactly needs to be included in the report, which they both agreed on," says Horne.
Other changes involved the process for patients being moved from the ED to inpatient units, and an abbreviated form was developed for patients being transported to a floor for treatment.
The goal is to ensure that the receiving caregivers have all the information they need to care for the patient, such as the patient’s "do not resuscitate" status, and facts that can impact the safety of both staff and patients, such as informing nurses that a pneumonia patient also happens to be a psychiatric patient.
"I’m eager to see if staff responses change when we compare the next survey results with our baseline data, as a result of these interventions," says Horne, adding that the organization plans to repeat the survey later this year.
Pinpoint problem areas
At Northwestern, the results were broken down by units and disciplines so they could be compared easily. The survey results were used to target three key areas for improvement: handoffs, communication between units, and feedback after reporting an incident. "One thing that surprised us was that people indicated more discomfort filling out incident reports than we expected," says Szekendi.
The data are useful to help with staff buy-in for improvement projects. "Now we can go to the department and instead of just taking data from the literature, we can tell them, This is what you told us was a problem,’" says Szekendi.
The following was done to address each area:
- A new web-based incident reporting system was implemented.
- Patient safety and nursing morbidity and mortality meetings are held monthly, taking a case study from a reported incident and asking staff for ideas on how to prevent future incidents.
- Training programs on handoffs and communication have been initiated.
Now the organization has a baseline and is planning to repeat the survey to assess the impact of the changes made over time, says Szekendi.
At Chesterfield, MO-based Sisters of Mercy Health System, the AHRQ tool was used as a pilot study in one hospital in fall 2005, and will be done across the complete health system during March 2006. "We will be working with each hospital to improve in areas that we find opportunities," says Charles Gasper, senior analyst for the organization.
Each question was evaluated for the mean responses, based upon whether the survey respondent was a clinician, worked directly with patients, and what shift he or she worked. These scores were further aggregated into 10 composite groups, such as "teamwork within units."
"We found the culture scores to be positive across the hospital," says Gasper. "However, in some cases, these sub-groups had statistically significantly lower scores for various safety areas."
A report was generated for hospital leaders with recommendations as to which composite areas should be addressed. The report gives information about the overall survey results and also is broken down by shifts, clinicians and non-clinicians, and whether staff work with patients, physicians, and nurses.
The summary report was presented to the hospital’s CEO and senior leaders, with portions presented to targeted groups and individuals. In addition, posters reporting the ratings for each of the composite scores are being put up throughout the hospital to communicate the results to front-line coworkers.
"We view this first analysis as baseline data from which we will compare results going forward," says Gasper. "However, a few areas of opportunity presented themselves during the analysis. These will be addressed during this year, with improvement evaluated over the course of future surveys."
Results will be tracked for each hospital over time, with the health system also utilizing comparison data provided by AHRQ, says Gasper.
"This will give us a better idea of where we stand, and who within our health system might be able to give us better insight as to how to improve our culture of safety," says Gasper.
Some organizations struggle with determining what result constitutes an "area for improvement." "We have suggested that if 50% of staff are "neutral" or "negative" on an item, that would be an area for improvement," Sorra says. "An area of strength could be defined as 75% or more of respondents answering positively on an item."
However, some hospitals find these cutoffs don’t work for them — either they end up with too many areas for improvement or not enough — so they’ll have to select a different cutoff, says Sorra. "These criteria can be raised or lowered with the overall goal of identifying a subset of areas for improvement where attention can be focused," she explains.
Once areas for improvement are identified, the next challenge is how to prioritize them. "The other problem is not knowing what actions to take or what initiatives to implement to improve a particular area," says Sorra. "This is something any hospital doing a patient safety culture survey is struggling with."
Free materials available from AHRQ include a survey user’s guide that provides guidance for selecting a sample, collecting data, analyzing the data and reporting results, and a PowerPoint feedback report template that hospitals can use to populate their survey results data to make presentations.
(See www.ahrq.gov/qual/hospculture.)
"The PowerPoint template displays the percentages of responses on the survey items, grouped according to the safety culture dimensions the items are intended to measure," she says.
Another free, downloadable resource is available on Premier’s web site (http://www.premierinc.com/all/safety/culture/index.jsp) — an Excel data entry, analysis, and reporting tool that allows hospitals with 2,500 surveys or less to import survey data and automatically generate charts of the results.
Comparison survey results are available on the AHRQ web site, so that you can see how your organization compares with other organizations, says Sorra. Overall composite scores are posted from 20 participating hospitals, with guidance on how to evaluate results compared to the benchmarks.
Westat currently is under contract with AHRQ to develop a national benchmarking database. "We expect to have a call for data submission some time in late spring 2006 requesting hospitals that have administered the AHRQ survey, and that meet several eligibility criteria, to submit their data to the database," she says.
"Announcements from AHRQ and other sources will be forthcoming once the database is underway and criteria for submission have been developed."
References
- Sexton JB, Thomas EJ, Helmreich RL, et al. Frontline assessments of healthcare culture: Safety Attitudes Questionnaire norms and psychometric properties. The University of Texas Center of Excellence for Patient Safety Research and Practice. Available at http://www.utpatientsafety.org
- Singer SJ, Gaba DM, Geppert JJ, et al. The culture of safety: results of an organization-wide survey in 15 California hospitals. Qual Saf Health Care 2003;12:112-118/
[For more information, contact:
Charles Gasper, Senior Analyst, Sisters of Mercy Health System, 14528 S. Outer Forty, Suite 100, Chesterfield, MO 63017. Telephone: (314) 628-3677. Fax: (314) 628-3473. E-mail: [email protected].
Barbara Horne, RN, Vice President and Chief Nursing Office, Indian River Memorial Hospital 1000 36th Street, Vero Beach, FL 32960. Telephone: (772) 567-4311. E-mail: [email protected].
Peter J. Pronovost, MD, PhD, Medical Director, Center for Innovations in Quality Patient Care, The Johns Hopkins University School of Medicine, 600 N. Wolfe Street, Meyer 295, Baltimore, MD 21287-7294. Telephone: (410) 502-3231. E-mail: [email protected].
Joann Sorra, PhD, Westat, 1650 Research Blvd., Rockville, MD 20850. Telephone: (301) 294-3933. Fax: (301) 315-5912. E-mail: [email protected]. Web: www.westat.com.
Sara J. Singer, Center for Health Policy, 117 Encina Commons, Stanford, CA 94305-6019. E-mail: [email protected].
Marilyn K. Szekendi, RN, MSN, APRN, BC,Division of Quality and Operations, Northwestern Memorial Hospital, 676 North St. Clair, Suite 700, Chicago, IL 60611. Telephone: (312) 926-9186. E-mail: [email protected].]
Would you like some compelling evidence to present to administrators about a chronic safety problem? Do you want a foolproof way to achieve staff buy-in when a new patient safety initiative is implemented? How about finding out what staff really think about reporting safety concerns?You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content