Annals of Emergency Medicine
Volume 36, Issue 5 , Pages 520-523, November 2000

One is the only number that you’ll ever need!☆

  • Brian H. Rowe, MD, MSc

      Affiliations

    • Division of Emergency Medicine
    • ,
  • Terry Klassen, MD, MSc

      Affiliations

    • Department of Pediatrics
    • ,
  • Peter C. Wyer, MD

      Affiliations

    • Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta; and Columbia University, College of Physicians and Surgeons, New York, NY.

Address for correspondence: Brian H. Rowe, MD, MSc, Division of Emergency Medicine, University of Alberta, 1G1.63 WMC, 8440-112 Street, Edmonton, Alberta, Canada T6G 2B7.

Article Outline

Abstract 

[Rowe BH, Klassen T, Wyer PC. One is the only number that you’ll ever need. Ann Emerg Med. November 2000;36:520-523.]

 

See related article, p. 517.

Back to Article Outline

Introduction 

Surprisingly, the mention of evidence-based medicine (EBM) conjures up mixed emotions among many physicians.1 Some appear to view EBM as a belief, and behave as if disbelief is heresy. Others project similarly strong beliefs in the opposite direction. As with many issues in medicine, we believe the best approach to clinical decisionmaking lies somewhere in between these 2 extremes. In the companion piece to this commentary, Schriger1 has demonstrated the courage to ask some important questions in the midst of an increasing interest in, and acclaim for, the promise of EBM in this and other fields of medicine. We acknowledge the need to identify these issues and appreciate the opportunity to openly debate them to clarify these for our clinician readers.

Back to Article Outline

Evidence-based medicine 

To start the discussion, it is critical to emphasize that, no matter how strong the evidence from clinical research, it remains just one of the components of evidence-based decisionmaking (Figure).

Patient values and preferences and the practitioner’s clinical experience and knowledge of disease play equally important roles in deciding what to do for the individual patient.2 To illustrate this concept, suppose a 26-year-old woman presents with acute asthma and is treated by you in the emergency department. During her stay, her peak flow rates improve from 38% to 68% predicted. She awaits discharge 3 hours after presentation, and you inform her that the best available evidence (derived from a strong systematic review) suggests she would be less likely to have to return to the ED if she received a short course of oral corticosteroids.3 In addition, your experience tells you she could have considerable difficulties if her airway inflammation is not treated aggressively. She informs you she is being married in less than a week and refuses prednisone, because in the past she has experienced hypomania, acne, and fluid retention while taking it. You are left with a need for a final, yet suboptimal, decision regarding treatment.

After discussing the risks, benefits, and possible tradeoffs, you and your patient arrive at a plan that addresses her needs and priorities and is consistent with your own clinical experience: single-dose prednisone in the ED, high-dose inhaled corticosteroids for the next 3 weeks,4, 5 and close follow-up. This is a clear example of EBM in real practice, and is neither an oversimplification nor reductionist thinking.1 In the absence of strong evidence supporting administration of systemic corticosteroids for her asthma exacerbation, the decision reached by you and your patient would likely have been the same as if there were no evidence: no systemic steroids.

Back to Article Outline

Searching strategies 

Having considered the components of evidence-based decisionmaking, we are ready to address the issue of evidence—how we find and select it. After the formulation of an appropriate question, one of the key elements of EBM is a rigorous attempt to search for and find as much of the evidence as possible. One of the many accomplishments of the EBM movement has been the recognition that simple searches of MEDLINE are woefully inadequate and may reduce the yield of all randomized clinical trials (RCTs) in a topic area below 50%.6 Consequently, thousands of volunteers in the Cochrane Collaboration are hand-searching for RCTs, and their efforts have produced a registry of more than a quarter million trials in The Cochrane Controlled Trials Register (CCTR), which is now considered the most definitive source of RCTs.7 EBM has been instrumental in addressing the issue of publication bias (the selective publication of small, positive studies) and other issues in systematic reviews.6 A key consequence of the EBM movement, where methodologically sound searching techniques identify more of the evidence (eg, published, unpublished, English, non-English articles, and so on), has been a reduction in situations whereby a single article (or none, for that matter) is used to justify action. In reality, single articles may still serve as the best evidence, especially in emergency medicine where large trials have not been completed.

Back to Article Outline

Levels of evidence 

We will adhere to the courtroom analogy introduced by Schriger1 for the purpose of comparing the methods used by evidence-based authors and clinicians with those used in a context less committed by its nature to adherence to scientific objectivity (Table).

Table. Comparison of decisions based on EBM and courtroom approach.
CharacteristicEBMCourtroom
All evidenceSoughtSought
Admissible evidenceActively controls against biasInherently biased pre-established criteria
Literature search
Preestablished selection criteriaSingle assessor (judge)
±Multiple assessors
Grading levels of evidenceA priori levels of evidenceDebate
Empirical support for criteriaNo empirical support for criteria
DecisionBased on evidence, experience, and patient preferenceNonvalidated criteria for decision
“Chance”
To begin, we would suggest that both EBM and legal proceedings start with a search for “truth” in evidence. “Admissible” evidence is selected in both settings using a priori criteria (selection criteria versus preestablished rules of evidence). However, EBM has the advantage that it may use more than one “judge” to determine which evidence is admissible. For example, in a sound systematic review, 2 or more authors frame the selection criteria in a fashion that takes into account the specific issue at hand. They then independently review citations to determine which studies meet the criteria for inclusion. Although the authors may well have biases and preconceptions, they are usually not explicitly seeking to prove opposing conclusions.

On the other hand, in a judicial proceeding, a single judge must rule on which “evidence” is to be allowed from among that offered by 2 adversarial advocates. Moreover, the livelihoods of these adversaries depend on the success with which they bring about a predetermined decision favored by a passionately interested and biased client. Under these circumstances, no matter how impartial the judge, it is much less plausible that all of the relevant evidence has been considered.

Perhaps the main difference between EBM and the legal setting is that the criteria for evaluating evidence are firmly established in medical decisionmaking. For example, in the matter of study design pertaining to a question on therapy, the RCT, or a systematic review of RCTs, constitutes the highest level of evidence.9 Empirical evidence from research shows the observational studies overestimate the clinical effect of therapy by as much as 40%.10 In the case of study designs pertaining to questions regarding the predictive power of diagnostic tests, there is strong evidence that methodology is critical in the development of firm conclusions from diagnosis studies.11 Finally, methodologic studies involving RCTs contained in systematic reviews have shown that low-quality trials result in a 37% overestimation of the effect of therapy compared with high-quality studies.10 These findings are all related to the fact that the study designs corresponding to higher levels of evidence involve more rigorous methods aimed at the avoidance of systematic biases in the research design. These efforts to reduce bias result in more valid estimates of what standard statistical methods assume to be “the truth.” Again, there are empirical studies to support these choices in medicine.10, 12 Wouldn’t it be wonderful if law was practiced this way?

Finally, once the evidence has been selected and evaluated, evidence-based reviewers and clinicians seek orderly and structured approaches to the process of combining evidence with patients’ values and practical concerns and experience in the clinical context in which the decision(s) must occur.9, 13, 14, 15, 16 In contrast, in a court proceeding, after a general “charge” to the jury by the judge, the decisionmaking process remains largely unstructured and the decisions relatively prone to influence by bias of individual jurors, no matter how strong the evidence presented by either side.

Back to Article Outline

Single sources of evidence 

Arguments against using results from single publications include concerns about fraudulent research, failure to recognize other levels of “evidence,” and the nihilistic approaches used in EBM/critical appraisal.1 Fraud and other kinds of misconduct in research are serious, and their importance should not be underestimated. However, we believe that systematic research approaches and the elaborate system currently in place (eg, ethics submission, peer review, standards of reporting) reduce its impact. Second, although the EBM approach may select a single article for elaboration, this is not the case when multiple similarly designed, and equally applicable, studies are available. In such cases, a systematic review would highlight the aberrant results, attempt mathematical pooling, and in the setting of heterogeneity, attempt to explain it. EBM has been instrumental in examining the discordant results found between different RCTs, between RCTs and systematic reviews, and between different systematic reviews dealing with the same question. Examining heterogeneity on the basis of design, population, intervention, outcomes, or chance provides an opportunity to explore reasons for differences. We believe such an approach is most appropriate and informative.

Given that appropriate selection criteria have been used and the best evidence is clearly available from only one article, what is the clinician to do? If that publication is a systematic review, then such evidence may provide definitive direction for the clinician.17 When a single randomized trial is all that is available to support a therapeutic or preventive intervention, then we would argue this remains the “best available evidence.” Furthermore, if this is still inadequate, any inclination to resort to lesser evidence should be replaced by a call to produce more randomized trials, a systematic review, or both. When the single study is one of the less rigorous pieces of evidence, such as anecdotal reports or preliminary observational studies, then this call should become a shout. Overall, EBM provides an estimation of the level of uncertainty on which decisions are based.

Back to Article Outline

EBM For the clinician 

EBM methodology cannot provide a guarantee to the clinician that there is a definitive answer to any question. Why then should clinicians embrace EBM in practice, spend time searching and evaluating evidence, and go through this often difficult process? There are many reasons why EBM should be used in the emergency medicine setting. First, when available and sound, evidence from clinical research helps to make physicians’ practice more confident, enjoyable, and easier.18 Second, some preliminary evidence suggests that EBM may in fact keep physicians free of litigation.19 Finally, evidence-based clinical practice provides the practitioner with a systematic approach to lifelong learning in a fashion that can only help the patient.

In the end, both patients and clinicians are searching for a reasonable estimate of the “truth” with respect to the diagnosis and treatment of problems presenting to the ED. In some cases, evidence will be strong and irrefutable and can be expected to represent the standard of care. When evidence is less conclusive, we may need to base some decisions on a single study.17 Although this may feel “lonely,” it is often better than expert opinion, “chance,” or guesswork. In cases where insufficient research has been performed, we should encourage our research colleagues to persevere and add valid research results pertinent to emergency practice to the literature. Finally, in the 50% or greater of cases when we find that evidence is lacking,18 we must rely on our clinical experience and knowledge of disease combined with a thoughtful consideration of our patient’s values and preferences. Caution is clearly required in all of these settings. In all cases, we owe it to our patients to be able to inform them about the evidence, its strengths and limitations, and how applicable it is to their own treatment.

Back to Article Outline

References 

  1. Schriger DL. One is the loneliest number: be skeptical of evidence summaries based on literature reviews. Ann Emerg Med. 2000;36:517–519
  2. Haynes RB, Sackett DL, Gray JMA, et al.  Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions. Evidence-Based Med. 1996;1:196–198
  3. Rowe BH, Spooner CH, Ducharme FM, et al.  The effectiveness of corticosteroids in the treatment of acute exacerbations of asthma: a meta-analysis of their effect on relapse following acute assessment (Cochrane Review). Oxford, UK: : Update Software; 1998; Cochrane Library (Issue 3)
  4. Edmonds ML, Camargo CA, Brenner B, et al.  Early use of inhaled corticosteroids in the emergency department treatment of acute asthma (Cochrane Review). Oxford, UK: : Update Software; 2000; Cochrane Library (Issue 1)
  5. Levy ML, Stevenson C, Maslen T. Comparison of short courses of oral prednisolone and fluticasone propionate in the treatment of adults with acute exacerbations of asthma in primary care. Thorax. 1996;51:1087–1092
  6. Dickersin K, Manheimer E. The Cochrane Collaboration: evaluation of health care and services using systematic reviews of the results of randomized controlled trials. Clin Obstet Gynecol. 1998;41:315–331
  7. Rowe BH, Alderson P. The Cochrane Collaboration: a resource for clinical problem solving in emergency medicine. Ann Emerg Med. 1999;34:86–90
  8. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309:1286–1291
  9. Guyatt GH, Sackett DL, Sinclair JC, et al.  User’s guide to the medical literature: IX. A method of grading health care recommendations. JAMA. 1995;274:1800–1804
  10. Moher D, Pham B, Jones A, et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet. 1998;352:609–613
  11. Lijmer JG, Mol BM, Heisterkamp S, et al.  Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282:1061–1066
  12. Schulz KF, Chalmers I, Hayes RJ, et al.  Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–412
  13. Guyatt GH, Sinclair J, Cook DJ, et al.  Users’ guides to the medical literature: XVI. How to use a treatment recommendation. JAMA. 1999;281:1836–1843
  14. Hayward RSA, Wilson MC, Tunis SR, et al.  Users’ guides to the medical literature: VIII. How to use clinical practice guidelines: A. Are the recommendations valid?. JAMA. 1995;274:570–574
  15. Dans AL, Dans LF, Guyatt GH, et al.  Users’ guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. JAMA. 1998;279:545–549
  16. Wilson MC, Hayward RSA, Tunis SR, et al.  Users’ guides to the medical literature: VIII. How to use clinical practice guidelines: B. What are the recommendations and will they help me in caring for my patients?. JAMA. 1995;274:1630–1632
  17. Wyer PC, Rowe BH, Guyatt GH, et al.  The clinician and the medical literature: when can we take a shortcut?. Ann Emerg Med. 2000;36:149–155
  18. Ellis J, Mulligan I, Rowe J, et al.  Inpatient general medicine is evidence based. Lancet. 1995;346:407–410
  19. Beerworth EE, Tiller JW. Liability is prescribing choice: the example of the antidepressants. Aust N Z J Psychiatry. 1998;32:560–566

☆ Reprints not available from the authors.

PII: S0196-0644(00)60294-9

doi:10.1067/mem.2000.110996

Refers to article:

  • One is the loneliest number: Be skeptical of evidence summaries based on limited literature reviews

    David L. Schriger
    Annals of Emergency Medicine November 2000 (Vol. 36, Issue 5, Pages 517-519)

Annals of Emergency Medicine
Volume 36, Issue 5 , Pages 520-523, November 2000

Article Tools

* Image Usage & Resolution