Question Authority!
Abstract & Commentary
By Barbara A. Phillips, MD, MSPH, Professor of Medicine, University of Kentucky; Director, Sleep Disorders Center, Samaritan Hospital, Lexington. Dr. Phillips is a retained consultant for Cephalon and Ventus, and serves on the speakers bureau of Cephalon and Boehringer Ingelheim.
Synopsis: The joint cardiovascular practice guidelines of the American College of Cardiology (ACC) and the American Heart Association (AHA) are largely based on expert opinion, case studies, or "standard of care," rather than on scientific evidence.
Source: Tricoci P, et al. Scientific evidence underlying the ACC/AHA Clinical Practice Guidelines. JAMA 2009;301: 831-841.
This paper is a collaborative effort from investigators at Duke, the University of North Carolina, and the American College of Cardiology (ACC). The authors set out to understand the underpinnings of the joint practice guidelines of the American College of Cardiology (ACC) and the American Heart Association (AHA). Specifically, they wanted to learn how much of the recommendations are based in scientific evidence, as opposed to being expert opinion. They examined ACC/AHA practice guidelines issued from 1984 to September 2008. They found 53 guidelines on 22 topics, including a total of 7196 recommendations. Each recommendation is designated by class of recommendation and by the level of evidence. The authors evaluated each of the recommendations according to its class of recommendation and level of supporting evidence.
The process of formulating guidelines can be found at www.acc.org and www.aha.org. In brief, the ACC/AHA joint guidelines classify the evidence upon which recommendations are based into 3 categories:
1. Level of evidence A: Evidence from multiple randomized trials or meta-analyses;
2. Level of evidence B: Evidence from a single randomized trial or nonrandomized studies;
3. Level of evidence C: Expert opinion, case studies, or standards of care.
Recommendations are made and classified based both on the strengths of the study data (e.g., level of evidence) and on the relative importance of the risks and benefits identified by the evidence. Recommendations are classified as follows:
Class I: Conditions for which there is evidence and/or general agreement that a given procedure or treatment is useful and effective;
Class II: Conditions for which there is conflicting evidence and/or a divergence of opinion about the usefulness/efficacy of a procedure or treatment;
Class IIa: Weight of evidence/opinion is in favor of usefulness/efficacy;
Class IIb: Usefulness/efficacy is less well established by evidence/opinion;
Class III: Conditions for which there is evidence and/or general agreement that the procedure/treatment is not useful/effective and in some cases may be harmful.
As can be inferred from this scheme, level of evidence C and recommendations of class II or higher indicate that there is a lack of definitive supporting evidence and some uncertainty about the appropriate medical decision. Guidelines are also divided into the following categories: 1) disease-based guidelines; 2) interventional procedure-based guidelines; and 3) diagnostic procedure-based guidelines.
Among the 53 guidelines evaluated in this study, 24 were diseased-based, 15 interventional procedure-based, and 14 diagnostic procedure-based. As of September 2008, 17 of the 53 guidelines were listed as the current guidelines on the ACC web site.
With regard to the class of recommendations, 1124 of the 3044 total recommendations were class II, with a median of 41% of recommendations in class II across the guidelines. In general, the guidelines shifted to more class II recommendations and fewer class III recommendations, while the use of class I recommendations remained fairly constant over time. Among disease-based and interventional guidelines, there was a trend toward more class II recommendations, and the proportion of class I recommendations decreased. In diagnostic guidelines, there was an increase in class I recommendations and a decrease in class II recommendations. The proportion of class III recommendations decreased among all guidelines, especially for interventional guidelines.
With regard to the level of evidence on which the guidelines were based, the 16 current guidelines that reported levels of evidence included a total of 2711 recommendations. Of these, only 314 recommendations were from level of evidence A (12%), whereas 1246 were supported by level of evidence C (46%).
Commentary
This is a disheartening paper, for several reasons. First, we appear to still be mostly practicing medicine by the seat of our pants. Secondly, everyone from insurance companies to malpractice lawyers looks to guidelines and standards to determine reimbursement or malfeasance. Thirdly, it is likely that guidelines based on expert opinion are frequently funded by and/or influenced by industry.1,2 Finally, the overall picture is probably bleaker than this article suggests, since cardiology may well be the best-funded and most rigorously studied field of medicine.
As the authors somewhat understatedly note in their abstract, "The proportion of recommendations for which there is no conclusive evidence is ... growing." These findings highlight the need to improve the process of writing guidelines and to expand the evidence base from which clinical practice guidelines are derived." No kidding. But in the meantime, what do we do? In the accompanying editorial (which I highly recommend), Shaneyfelt and Centor say this about guidelines, "If all that can be produced are biased, minimally applicable consensus statements, perhaps guidelines should be avoided completely. Unless there is evidence of appropriate changes in the guideline process, clinicians and policy makers must reject calls for adherence to guidelines. Physicians would be better off making clinical decisions based on valid primary data."3 I tend to agree!
References
1. DeAngelis CD, Fontanarosa PB. Impugning the integrity of medical science: The adverse effects of industry influence. JAMA 2008;299:1833-1835.
2. Al-Khatib SM, et al. Preventing tomorrow's sudden cardiac death today: Dissemination of effective therapies for sudden cardiac death prevention. Am Heart J 2008;156: 613-622.
3. Shaneyfelt TM, Centor RM. Reassessment of clinical practice guidelines. Go gently into that good night. JAMA 2009;301:868-869.
The joint cardiovascular practice guidelines of the American College of Cardiology (ACC) and the American Heart Association (AHA) are largely based on expert opinion, case studies, or "standard of care," rather than on scientific evidence.You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content