Educate teams to develop the best review criteria
Practice guidelines require planning
By Patrice Spath, ART
Brown-Spath Associates
Forest Grove, OR
With the proliferation of nationally developed clinical practice guidelines, it is no longer necessary for individual medical staffs to start from scratch when defining quality patient care. However, success at using guidelines for performance measurement activities starts with well-trained committees that include the right members. Organizations translating clinical practice guidelines into medical review criteria and performance measures should retain knowledgeable clinicians and health information management professionals to assist in the process.
The committee charged with developing guideline-based medical review criteria should have the same multidisciplinary composition that has been suggested for guideline development panels. For instance, the committee developing performance measures for pain management should include at least one representative of each of the following disciplines:
· surgeon;
· primary care physician;
· nurse;
· anesthesiologist;
· internist;
· pharmacist;
· physical therapist.
Other disciplines, such as psychologists, physiatrists, neurologists, and medical ethicists should be added if they are available. When selecting members for the committee, consider the task group recommendations of the Washington, DC-based Center for Clinical Quality Evaluation (formerly the American Medical Review Research Center) when developing guideline-based quality and utilization review criteria:
· Criteria panel members are practicing clinicians with experience in the relevant specialties.
· Panel members have academic experience and/or a research-related background.
· Panel members have experience with quality assurance and the application of quality review criteria.
Those expected to comply with the guidelines also should be afforded an opportunity to approve the medical review criteria that will be used to judge their practices. Their input should be obtained before the criteria are finalized.
Health care professionals chosen for the purpose of translating clinical practice guidelines into medical review criteria and performance measures should be familiar with the clinical aspects of the patient care they hope to evaluate. Their knowledge should encompass both clinical training and experience in the health care process undergoing review. When groups lack clinical familiarity with the area they are evaluating, they are more likely to misinterpret guideline statements and develop faulty medical review criteria and performance measures.
In addition to clinicians, this group should include as technical assistants professionals with experience in health care quality measurement, particularly the information management components of criteria development, study design, data collection, and display of results. Ideally, the task force also will include support from an individual with epidemiologic training who has knowledge of research methods and statistics. The contributions of epidemiologic and health service research techniques to the measurement of health care quality cannot be denied, but the typical hospital may find such support is unavailable or too costly to procure. At the current time, experienced health information management professionals, nurses, or other professionals with formal quality management training can adequately oversee the data components of the average hospital-based performance measurement project. When outcomes measurement extends beyond a patient's hospitalization, such as long-term functional outcome studies, the services of a master's-prepared researcher become more important.
Prior to embarking on the guideline translation process, the group should receive education in the following areas:
· attributes of good clinical practice guidelines;
· attributes of good medical review criteria;
· differing purposes of clinical practice guidelines and medical review criteria;
· methods for deriving medical review criteria and performance measures.
The two publications available from the Agency for Health Care Policy and Research listed at the end of this article are excellent learning resources.
How have other groups faced these issues?
Ideally, the committee's educational process also will include an analysis of how other groups have struggled with and resolved issues involved in translating clinical practice guidelines into medical review criteria. For example, the hospital group charged with developing medical review criteria is likely to face conflicting recommendations that may be found in guidelines on the same topic developed by different organizations. Conformance to guidelines is a fundamental definition of quality, but the task group will need to deal with the question of "Whose guidelines for what condition?" The guidelines being developed by professional medical societies vary considerably in length and format, focus, who was involved, and rigor applied to the development process. The experience of groups who have already dealt with this issue can be useful for educational purposes.
For example, when faced with several research recommendations and/or conflicting guidelines in their Clinical Efficacy Assessment Projects, the American College of Physicians (ACP) relies on a combination of clinical epidemiology, statistics, clinical expertise, and decision analysis to finalize their recommendations. The governing body of the ACP, with input from the membership and advisory bodies, the Council of Subspecialty Societies and Council of Medical Societies, develops guidelines for clinicians specifying the circumstances in which the test or procedure is indicated, not indicated, or contraindicated. Areas where data are inadequate to permit any authoritative recommendation are clearly identified. The hospital committee involved in translating guidelines into medical review criteria can use a similar process, indicating which criteria are evidence-based and which criteria merely represent the consensus of those working in the institution.
Resolving differences that arise when there are multiple guidelines on the same topic will require the task group to employ various group decision-making or consensus techniques. Education on using certain tools for making team decisions can be particularly useful in these situations. Brain storming, multivoting, and nominal group process are three of the techniques commonly used for arriving at a group consensus. When the group cannot reach agreement on a particular medical review criteria, it's important they not use the criteria to make quality judgements.
Look at your data sources
The medical review criteria development group should understand the limitations that might be imposed by inadequacies or lack of clarity in patient care documentation. When the cost of acquiring the information exceeds the value of the information obtained, consider excluding the data element from the criteria set. It is not uncommon for groups to limit their criteria sets and performance measures to only those data elements that are readily accessible in the medical record. Determining the extent to which data source limitations will influence the group's criteria set and performance measures requires the group to be familiar with various data access and collection alternatives.
The patient record, or a computerized database of multiple patient records, will likely serve as the primary data source for evaluating the quality of health care services. However, the patient record may not contain important information about patient histories and physician prescribing patterns. When using computerized clinical or financial information as a data source, several considerations are important:
· Correctly coded diagnosis data may be subject to misinterpretation. For example, the ICD-9 code for urinary tract infection is 599.0. The same code is used for urinary tract infection if the patient is admitted to the hospital with the infection or develops the infection after admission. Analysts wishing to separate community-acquired infections from nosocomial infections cannot do so based on the coded information alone.
· Computerized financial data sources commonly do not differentiate between such events as "planned" and "unexpected" hospital readmissions. To determine the incidence of either type of readmission, manual record review would be necessary.
· There may be a lack of information pertaining to the performance question under study. For example, substance abuse often is underdiagnosed in hospitalized patients with other health problems. If management of patients with substance abuse is being reviewed, hospital claims data may be a poor source of information. The variables of patient preference or patient refusal in accepting treatment recommendations made by clinicians are other important data elements that will be missing in most computerized data sets.
Another issue affecting data collection from computerized databases relates to the current state of the art in computer systems. Unfortunately, existing software packages have been developed for widely diverse computing environments and an even wider range of computer languages. Consequently, available data systems tend to exist as isolated components with limited utility, flexibility, systems compatibility, or interactive potential. Unless the developers of medical review criteria and performance measures can be assured of the availability of a powerful relational database management system for rapidly integrating fragmented records and file systems, they may be limited to manual retrieval of information from handwritten patient records.
The criteria development group should be educated in the cost-effectiveness of various data- collection alternatives prior to their development of medical review criteria and performance measures. The data needed to measure compliance with one or more criteria may be difficult to gather, and therefore the group may choose to exclude the criterion from their criteria set. For example, a criterion concerning patient education might be valid. However, when patient education takes place but is not documented in the record, the criterion cannot be used in records-based review because the required information is not present. A more costly data collection effort, such as patient interviews, might be selected if the task group decides the need for the information outweighs the additional financial burden.
Examine your criteria language
The task group also should understand the difference between implicit and explicit medical review criteria. The use of implicit criteria means the assessor of the quality of care, usually an expert, asks the question, "What would I do in a similar situation?" Explicit criteria are stated in advance. For example, "The patient's pain should be assessed every two hours during the first postoperative day after major surgery." The use of explicit criteria minimizes personal judgement errors made by data abstractors. Consequently, the comparability and reliability of the assessments are much better than is true with implicit criteria.
To define explicit criteria, the task group must develop objective definitions for all data elements used in the criteria set and clearly identify the target population, as well as minimizing the use of ambiguous terms. For example, it is unacceptable to use words such as "high" or "abnormal" in relation to laboratory findings without providing quantitative definitions that are not subject to interpretation by data abstractors. Ultimately, the group should oversee the development of a dictionary of data element definitions to ensure uniform collection of information pertaining to criteria compliance.
The group should learn about the concept of data validity. The validity of a criterion or a performance rate is the extent to which it actually measures the characteristic or property one wants it to measure. Measuring the completion of a particular test or treatment during a patient care episode is valid in that it indicates that the event occurred. As a measure of quality, the presence of the test or treatment is valid only to the extent that the test or treatment impacts the quality of the patient care outcome. If measuring the quality of patient care is the intent of the group, this is best accomplished by only developing criteria and performance measures that are derived from clinical guidelines where evidence or professional agreement is very strong. Criteria based on guidelines that are controversial or where professional agreement is weak are unlikely to be valid measures of quality.
Good clinical practice guidelines don't necessarily translate into good medical review criteria and performance measures. It's important that the multidisciplinary teams involved in measuring quality include the right members. In addition, they must be adequately educated about the pitfalls of the translation process and how other groups are overcoming commonly faced barriers. Quality management professionals can play an important role in these educational efforts.
Resource
U.S. Department of Public Health and Human Services, Public Health Service, Agency for Health Care Policy and Research. Using Clinical Practice Guidelines to Evaluate Quality of Care - Volume 1: Issues; Volume 2: Methods. Rockville, MD: March 1995. (AHCPR Pubs. No. 95-0045 and 95-0046). See the AHCPR Web site - www.ahcpr.gov - or call (202) 512-1800 for ordering information.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content