Annals of Emergency Medicine
Volume 49, Issue 3 , Pages 355-363, March 2007

Knowledge Translation: Closing the Evidence-to-Practice Gap

  • Eddy S. Lang, MDCM, CCFP(EM), CSPQ

      Affiliations

    • Department of Emergency Medicine, McGill University and Sir Mortimer B. Davis Jewish General Hospital, Montreal, Quebec, Canada (Lang)
    • Corresponding Author InformationAddress for correspondence: Eddy S. Lang, MDCM, CCFP(EM), CSPQ, Emergency Department, SMBD Jewish General Hospital, 3755 Cote Ste Catherine, Montreal, Quebec, Canada H3T 1E2; 514-340-8222 ext 5568, fax 514-340-8138
    • ,
  • Peter C. Wyer, MD

      Affiliations

    • Columbia University College of Physicians and Surgeons, New York, NY (Wyer)
    • ,
  • R. Brian Haynes, MD, PhD, FRCPC

      Affiliations

    • the Department of Clinical Epidemiology and Biostatistics and Department of Medicine, McMaster University, Hamilton, Ontario, Canada (Haynes)

Received 11 July 2006; received in revised form 14 August 2006; accepted 25 August 2006. published online 10 November 2006.

Article Outline

Knowledge translation describes any activity or process that facilitates the transfer of high-quality evidence from research into effective changes in health policy, clinical practice, or products. This increasingly important discipline attempts to conceptually combine elements of research, education, quality improvement, and electronic systems development to create a seamless linkage between interventions that improve patient care and their routine implementation in daily clinical practice. We outline the gap between research and practice and present a case study of an emergency medicine example of validated evidence that has failed to achieve widespread implementation. The authors describe a model of organization of evidence and its relationship with the process that links research from the scientific endeavor to changes in practice that affect patient outcomes. Obstacles to evidence uptake are explored, as well as the limitations of current educational strategies. Innovative strategies in realms such as computerized decision support systems designed to enhance evidence uptake are also described. The potential interface between knowledge translation and continuous quality improvement, as well as the role for bedside tools, is also presented. Research in knowledge translation includes studies that attempt to quantify and understand the discrepancies between what is known and what is done, as well as those that examine the impact and acceptability of interventions designed to narrow or close these gaps. Sentinel examples in this line of research conducted in the emergency department setting are described.

 

Back to Article Outline

Introduction 

Knowledge translation is a relatively new term that has rapidly gained prominence in many health care disciplines, including medicine, public health, and health care policy development and administration. However, the notions underlying knowledge translation are not recent and might be recognized by a number of fairly synonymous terms and phrases, including “translating research into practice,” getting research into practice, knowledge use, knowledge dissemination, knowledge transfer and evidence translation, research uptake, evidence uptake, and others. The Canadian Institutes of Health Research defines knowledge translation as “the exchange, synthesis and ethically sound application of knowledge—within a complex system of interactions among researchers and users—to accelerate the capture of the benefits of research for patients through improved health, more effective services and products, and a strengthened health care system.”1

Although knowledge translation may be unfamiliar terminology to most emergency physicians, the existing gap between current best evidence and evidence-based practice is a concern that most clinicians can relate to and has been at issue for decades. The members of the Clinical Research Roundtable at the US Institute of Medicine have suggested that the failure to translate new knowledge into clinical practice and decisionmaking in health care is a major barrier preventing human benefit from advances in biomedical sciences.2 Whether it falls into the realm of acute care, the management of chronic medical conditions, or prevention care, it appears that patients often fail to receive recommended standards of care or are receiving potentially harmful or unproven treatments. McGlynn et al3 examined more than 400 quality indicators in some 6,700 patients drawn from a dozen metropolitan areas and suggested that 45% are not receiving recommended care. This gap in care is also highlighted in the landmark 2001 Institute of Medicine report titled “Crossing the Quality Chasm: A New Health System for the 21st Century.”4 The Institute of Medicine report described the “chasm” that exists between medical advances (what we know) and medical care currently in place in the United States (what we do). The report emphasized 3 aspects of quality underachievement: misuse (medical error), underuse (of proven therapies), and overuse (of inappropriate treatments). The report covers a large body of literature that outlines the nature of these deficiencies and outlines how a large proportion of patients in a variety of clinical contexts fail to receive standard care. The recommendations contained within the report outline a number of systems-based changes in the structure and functioning of the health care system designed to close the “quality chasm” in 6 domains, ie, care that is safe, effective, patient centered, timely, efficient, and equitable. The Institute of Medicine report has played a pivotal role in advancing the knowledge translation agenda in health care policy development, as well as through research initiatives through organizations such as the Agency for Healthcare Research and Quality.

The purpose of this article is to introduce emergency practitioners and researchers to knowledge translation as a frontier concept and to identify and explore ways in which it potentially intersects emergency medicine education, practice, and research. We will start by defining the problem in some detail.

Back to Article Outline

Ottawa Ankle Rules: A Case Study of Knowledge Translation in Emergency Care 

There is ample evidence within the emergency medicine literature that uptake of research evidence into clinical practice is inconsistent even when implementation strategies are undertaken. One of the best-studied examples of this phenomenon is the Ottawa Ankle Rules, in which some implementation strategies have failed to affect rates of ordering for radiographs.5, 6 The Ottawa Ankle Rules, first derived in the early 1990s, were designed as a highly sensitive bedside instrument for determining which emergency patients with ankle injuries may forgo radiography without risk of missing a clinically important fracture, thereby reducing unnecessary radiographs.7 The clinical accuracy of the Ottawa Ankle Rules underwent a number of prospective validation studies in subsequent years, the results of which were synthesized in a systematic review of all conducted trials.8 This systematic review supported the diagnostic accuracy of the Ottawa Ankle Rules, reporting a pooled sensitivity of 98% (95% confidence interval [CI] 96% to 99%) and a specificity of 32% (95% CI 24% to 44%). Impact analyses of the Ottawa Ankle Rules, including a multicenter randomized trial, confirmed its diagnostic performance and demonstrated its potential acceptability to physicians and ability to substantially reduce the ordering of radiographs.9, 10

Despite these demonstrations, published surveys indicate that the degree of clinical uptake of the Ottawa Ankle Rules in the “real world” is inconsistent. A rigorously done survey suggested that 90% of United States and Canadian respondent emergency physicians were aware of the rules. However, only 35% of US respondents reported using it in daily practice compared with more than 80% of their Canadian counterparts.11 A more recent survey of Canadian physicians performed by many of the same investigators reported that, despite high familiarity and reported “use” only 31% could correctly remember all of the components of the rule, and only 42% based their decisionmaking primarily on the Ottawa Ankle Rules.12 Both of these studies concluded that the mere publication of an abundance of confirmatory studies is insufficient to result in widespread implementation and changes in practice, even in centers originally involved in developing and validating the Ottawa Ankle Rules.6, 11, 12

The experience with the Ottawa Ankle Rules highlights the challenges facing clinical implementation within emergency care of even the most highly validated diagnostic interventions. Similar challenges are evident on the therapeutics front. Examples include suboptimal management of acute otitis media13, 14 and the significant underuse of established therapies such as aspirin and β-blockers in the setting of acute coronary syndromes.15 Obstacles to the application of existing best evidence and the development and study of innovative approaches to overcoming these obstacles fall squarely into what knowledge translation is intended to accomplish.

Back to Article Outline

A Model for Understanding Knowledge Translation 

The factors that impede the efficient transfer of well-substantiated clinical research into clinical emergency medicine care are myriad and complex. Strategies designed to address them have been categorized by Glasziou and Haynes.16 They view the research-practice continuum as being divided into 2 major categories: the first, involving the task of “getting the evidence straight,” and the second, related to “getting the evidence used.” These concepts are represented as interconnected domains within Figure 1. The pyramid component of this figure was initially proposed as a hierarchic scheme for understanding the different forms in which published research evidence may be presented.17 The category of “getting the evidence used” is illustrated in the figure by the evidence-to-practice pipeline. The following sections of this review will elaborate on and clarify the specific elements of this evidence-to-practice model.

  • View full-size image.
  • Figure 1. 

    A model for closing the evidence-to-practice gap. This schematic demonstrates 4 stages of moving from research to practice-altering outcomes. The first stage involves getting the evidence straight, illustrated by increasingly more applicable forms of information drawn from valid and important clinical research and represented in the 4S pyramid. The evidence-to-practice pipeline, also shown, reveals the dissipation of useful conclusions from clinical research, thus failing to make it into practice. The 3 remaining disciplines of knowledge translation can facilitate evidence uptake and help close the gap between research and practice. Adapted from Glasziou and Haynes.16 EBM, Evidence-based medicine; CQI, continuous quality improvement.

Back to Article Outline

Getting the Evidence Straight: The Pyramidal (4S Levels) Organization of Research Evidence 

Systems 

Located at the peak of the evidence pyramid, “systems” refer to clinical information support instruments such as those that could be seamlessly integrated into a patient's electronic health record. The ideal clinical information system would remind physicians of key relevant actions in both the diagnostic and therapeutic realms that are supported by an existing evidence base. The optimal evidence delivery system would offer recommendations in concise and easily manageable bits of information and would be constantly updated to incorporate recent developments on the research front. Furthermore, this type of system would be automatically activated in the appropriate clinical setting and become seamlessly integrated into workflow for the clinician. Health information systems meeting all of these specifications are still in preliminary stages of development. Early work with expert charting and real-time clinical guidance systems have to date met with only a modicum of success.18

Synopses and Critically Appraised Topics 

Short of fully integrated information systems, clinicians might best be served by readily searchable databases that can provide evidence-based synopses of best evidence specifically designed for bedside use. Perhaps best known of the journals that systematically peruse the biomedical literature for research that is of relevance to the field of internal medicine is the American College of Physicians' ACP Journal Club. This bimonthly periodical provides readers with succinct analyses and critical appraisals of published research reports that meet a minimum methodologic standard.

The 1-page synopses provide information on the citation, the research question, and study design and include a standardized reporting of results. They also include a relevance and newsworthiness score provided by a network of clinicians, including emergency physicians, who rate each article in advance of the synopses' publication.19 Unfortunately, the content of ACP Journal Club, and even of Evidence-Based Medicine, a related publication with a broader clinical focus, overlaps only modestly with the topic interests of an emergency medicine audience. At present, no systematically organized forum for research synopses exists within our specialty.

Some formats for evidence synopses exist within the emergency medicine literature and in online databases. These consist of critically appraised topics, or “CATS,” that provide shortcut reviews of clinical questions based largely on the findings of 1 or more studies on a given subject. Best evidence topics, or “BETS,” are a variation on this theme and are summaries of best available research evidence designed to address specific clinical questions of relevance to emergency medicine (available at http://www.bestbets.org). Annals of Emergency Medicine also provides a venue for evidence-based synopses of key questions that are relevant to emergency medicine (available at http://www.annemergmed.com/content/sectII). The other major objective of synopses and critically appraised topics is to provide clinicians with a bottom-line evidence summary in a format that is accessible and readily incorporated into decisionmaking and clinical care. Practice guidelines are rarely able to provide this degree of user friendliness.

Although in its early stages, a growing body of research suggests that ready access to synopses can have various degrees of impact on physician practice and patient outcomes. A systematic review of clinical information retrieval technology suggests that approximately one third of information retrieval efforts by physicians influence their decisionmaking.20

Systematic Reviews and Syntheses 

With hundreds of randomized controlled trials being published each week, it is nearly impossible for emergency physicians to stay abreast of important developments in clinical research by engaging in a filtering process for determining which studies are relevant and could have an influence on their practice. The process of getting the evidence straight is particularly daunting in emergency medicine, which as a broad horizontal specialty might be appropriately influenced by developments from many other fields. Thus, a key component of assimilating the evidence would ideally involve the search, critical appraisal, and synthesis of best evidence, using systematic review methodology, to address fundamental questions of diagnosis and management in clinical emergency medicine.

The most important feature of systematic reviews is that they use a predetermined methodology for assessing the scope and validity of research evidence and provide an analysis for the application of important findings in clinical practice. Unfortunately, systematic reviews published in the emergency medicine literature during the past several years have been found deficient in quality, with gaps evident in comprehensiveness of search strategies and in the methodology used for evaluating the quality of incorporated studies.21

The Cochrane Collaboration is recognized for its well-established record in systematic review methodology, with a database of more than 2,600 published systematic reviews to date, covering the gamut of health care interventions and hundreds more protocols for other systematic reviews that are in development. Encouraging from an emergency medicine perspective is the newly created emergency medicine/emergency medical services field within the Cochrane Collaboration, which will encourage the development of systematic reviews of particular relevance to the specialty. Systematic reviews not prepared under the auspices of the Cochrane Collaboration constitute more than half of all reviews, and both Cochrane and non-Cochrane reviews of relevance to emergency care are included as alerts and for searching through BMJ Updates+ (available at http://bmjupdates.com), a free service sponsored by the BMJ Publishing Group.22 PubMed Clinical Queries also provides a search strategy for reviews (available at http://www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml).

Getting the Evidence to Patients: The Evidence-to-Practice Pipeline 

In the schema adapted from Glasziou and Haynes16 (Figure 1), the evidence pipeline represents the trajectory that research evidence (represented as water) must take to be incorporated into clinical practice. The inability of clinical research to influence patient care and affect outcomes is represented as the evaporation of water (research evidence) as it tracks down an evidence pipeline, where a number of factors contribute to its evaporation and dissipation. Knowledge translation is viewed as encompassing 4 major disciplines (resource development and access, bedside evidence-based medicine, clinical quality improvement, and the use of decision aids), all of which improve the path from awareness to adherence. Having considered the relevant categories of resource, the pyramid in Figure 1, we are ready to elaborate on the remaining 3 steps in the translation process.

Back to Article Outline

Bedside Evidence-Based Medicine 

Bedside evidence-based medicine responds to what is commonly a problematic divergence between the questions that arise during clinical encounters and a readily accessible pathway to finding usable answers. Easily searchable databases of clinician-friendly evidence summaries that can be called up quickly would meet this need effectively. Handheld computers offer the potential to store and make readily available the type of resources that could meet this need. Emergency medicine examples of bedside evidence-based medicine are emerging in the literature on the clinical practice and educational frontiers. Bullard et al23 reported a randomized trial comparing a wirelessly networked mobile computer to a traditional desktop version, with both systems providing ready access to a number of decision rules and locally agreed-on clinical practice guidelines. Physicians randomized to the mobile version of the decision support system were more likely to make use of these resources.

Bedside evidence-based medicine also encompasses the large body of evidence that informs the clinical assessment of patients and decisionmaking and guides the ordering and interpretation of any indicated diagnostic tests. Validated instruments that define a patient's probability of a suspected disease (pretest probability), before undergoing specific diagnostic tests, provide an example of this aspect of bedside evidence-based medicine. An example of such an instrument is the Wells criteria for stratifying risk of deep venous thrombosis before performance of a test such as compression ultrasonography.24

Back to Article Outline

Continuous Quality Improvement 

Continuous quality improvement refers to initiatives that include designing, implementing, and monitoring adherence to system-wide changes that facilitate the incorporation of best evidence into patient care. As an example, a quality improvement initiative designed to promote adherence to an evidence-supported management algorithm for suspected pulmonary embolism25 and a program to monitor compliance by physicians can be viewed as important steps toward ensuring uniformly high-quality care for patients suspected of having this condition. Quality improvement programs designed to improve evidence uptake are new to the emergency medicine literature. Lindsay et al26 used a modified Delphi process to develop a series of quality indicators for use in the emergency setting, many of which, such as aspirin use in acute coronary syndromes and corticosteroid use in asthmatic patients being discharged from the emergency department (ED), address the evidence practice gap.

Emergency care contexts can incorporate quality improvement methodology into evidence-based initiatives. Essential components include planning a strategy to improve adherence to locally agreed-on evidence-based management and monitoring adherence through an “evidence uptake” indicator.27 For example, such an initiative might use an integrated charting/computerized physician order entry system to require verification of the Ottawa Ankle Rule criteria in the context of radiographic orders for all patients presenting with ankle injuries. Application of the rule occurs within this electronic interface or through selection from a limited number of reasons for bypassing the recommended course of action. Such a system could readily be designed to track quality indicators reflecting uptake of the Ottawa Ankle Rule, impact on the number of radiographs ordered, and specific issues related to noncompliance by members of the clinical team.

Back to Article Outline

Decision Aids, Patient Education, and Compliance Aids 

Computer-based clinical decision support systems would seem to offer important opportunities in the realm of knowledge translation, especially given the traditionally hectic and fast-paced environment of the ED. A review of the effect of these interventions on physician behavior and patient outcomes28 in 97 eligible studies revealed a significant effect of improving physician compliance, especially in reminder-type systems in which indices of patient care were increased in 76% of the relevant trials. Disconcerting within this review is the relative paucity of examples of high-quality studies that are of relevance to emergency medicine practice.

Increasingly, however, many of the therapeutic interventions that have been proven beneficial in the context of emergency medicine require an integrative and collaborative approach with other specialties. Examples such as early goal-directed therapy in septic shock, therapeutic hypothermia in survivors of cardiac arrest, or timely percutaneous coronary interventions in ST-segment elevation acute myocardial infarction require collaborative clinical pathways developed for local implementation among emergency physicians and other relevant acute care disciplines.29, 30 These interventions also lend themselves to standardized protocols that ensure uniform patient care.

Organizations such as the Agency for Health Quality and Research and the National Guideline Clearing House are avidly involved in developing methods for improving the implementation of clinical practice guidelines.31 This includes anticipation and planning for their implementation from early in their development by involving key stakeholders in the guideline selection and creation process.

Barriers to Evidence Uptake 

The barriers to evidence uptake have been the subject of extensive study and scholarly work elaborating a number of models describing the obstacles associated with incorporating change into professional practice and behavior.32, 33 Much of the work on barriers has involved implementation of practice guidelines but is transferable to all circumstances in which a convincing evidence base is inconsistently applied or ignored in everyday practice.

One of the most comprehensive schemes for considering the barriers to evidence uptake classifies groups of barriers into the 3 domains of knowledge, attitude, and behavior (Figure 2).33 The basic premise underlying this schema is that innovation and change in a physician's clinical practice are contingent on some combination of all 3 domains, each of which may encounter opposing forces. Opposing knowledge acquisition are the increasing volume of new literature relevant to clinical practice, the time investment required to achieve mastery of this information, and barriers to online access. Attitudes that encourage change may be hindered by skepticism and mistrust of clinical research, as well as by uncertainty about its applicability to practice. Behavioral changes that reflect new evidence and innovation may be impeded by external pressures that favor the inertia of the status quo. These include environmental factors such as the need to commit resources to the process of initiating a new hospital or interdepartmental treatment protocol, medicolegal concerns and patient expectations about the need for diagnostic tests that may also obstruct change or innovation, and institutional or regulatory issues. Physicians may also be wary of being the first among their peers to introduce significant changes in practice. Although not insurmountable, specific barriers to change have been addressed with educational approaches that have had less than their desired effect.34

  • View full-size image.
  • Figure 2. 

    Barriers to evidence uptake. Innovation and change in clinical practice at the individual, departmental, and institutional levels are contingent on 3 key sources of behavior change: knowledge, attitudes, and behavior. However, a series of complex barriers exists for each of these dimensions, some of which are illustrated here. Adapted from Cabana et al.33

Effectiveness of Guidelines and Continuing Medical Education 

Continuing professional development initiatives are relied on heavily in the health care system to inform and improve physician practice, making it more compliant with evidence-based strategies. A review of systematic reviews on this topic34 noted that the most widely used approaches consisting of didactic presentations and the dissemination of printed material are the least effective means for changing physician practice. A systematic review, consisting of randomized trials or well-designed quasiexperimental studies, examined the effect of continuing education meetings (including lectures, workshops, and courses) on the clinical practice of health professionals or health care outcomes.35 This review revealed little or no effect of didactic presentations, in contrast to moderate to large impact from programs that involved a workshop component.

Interactive approaches, such as audit/feedback, academic detailing/outreach, and reminders, seem most effective at simultaneously changing physician care and patient outcomes. Audit and feedback is among the best studied of the interactive approaches to changing professional behavior. It refers to analyzing a physician's practice pattern and providing feedback on inconsistencies with evidence-based practice. A systematic review of 72 studies36 suggested that these approaches can improve physician compliance by as much as 71% but can also have the unintended effect of reducing compliance by 10%. Negligible or paradoxic effects of audit and feedback were seen primarily when the intensity of the intervention was weak, ie, delayed or anonymous feedback on practice patterns, or when the degree of compliance with the target intervention was already high. Multifaceted interventions that rely on more than 1 method to remind physicians about following evidence are also generally more effective than simpler approaches but are also more costly.37

Guidelines are often deficient in the methodology used for their development and, even when touted as evidence-based, may not reflect a systematic methodology. When not developed with rigorous methodology, practice guidelines are vulnerable to the perception of bias from vested interests. Clinical practice guidelines have been greeted with some skepticism in the emergency medicine and wider medical community, with concerns related to the manner in which recommendations have been reached, the strength of evidence underlying recommendations, and the perspectives and values that have guided the process.38, 39, 40

These issues may partly explain why guideline implementation research has yielded such weak evidence of benefit.41 A review of 235 studies that examined guideline dissemination and implementation strategies found that these interventions result in no more than a 10% median improvement in health care provider behavior.41 Although modest, this benefit can have important implications at the level of population health.

Knowledge Translation Research 

Knowledge translation can be viewed as a clinical practice paradigm and a research agenda. Research is relevant to all 4 of the knowledge translation disciplines illustrated in Figure 1 (1, research synthesis, guidelines, EBM journals; 2, bedside EBM; 3, clinical continuous quality improvement; 4, decision aids, patient education, and compliance aids) and needs to address the exigencies and particularities of specific specialties and practice settings.

The common denominator of all knowledge translation research is a measure of changing of practitioner behavior in the direction of applying evidence-based interventions and strategies to patient care. A second and key objective should be the demonstration that patients benefit, but this is often missing from such studies.27 Secondary outcomes include tabulation of the financial and manpower resources consumed in the course of achieving change in the primary outcomes.

An aspect of knowledge translation that is largely understudied relates to the impact of evidence summaries and syntheses on patient care. There is no evidence in the realm of emergency medicine that such interventions would be effective. There is, however, evidence that handheld computers equipped with evidence-based summaries of key research in hypertension can translate into more rational prescribing practices for trainees in family medicine.42 Another example of practical bedside application of evidence syntheses affecting patient care was described in relation to an inpatient medical service and made use of an evidence cart consisting of textbook and preappraised resources.43 The authors noted that of nearly 100 attempted searches, 90% were effective in uncovering relevant information that informed decisionmaking, and a significant number of these resulted in new or altered management decisions for patients. When the cart was removed from the ward, searches and use of information from research returned to the previous baseline.

Much of knowledge translation research is presented as quality improvement research initiatives. One of the most ambitious examples in emergency medicine is a 32-site cluster randomized trial of the emergency care of pneumonia based on a validated prediction rule for severity.44, 45, 46 This study by Fine et al44 and Yealy et al45, 46 compared 3 implementation strategies for a clinical practice guideline designed to identify which patients with pneumonia require hospitalization. The guidelines were also meant to ensure that key processes of care, such as antibiotic administration within 4 hours of presentation, were followed.

The participating centers were randomly allocated to evidence uptake initiatives according to low-, medium-, and high-intensity approaches. The high-intensity approach distinguished itself from the other strategies by incorporating real time, paper reminders that facilitated scoring and calculation of the severity score, with data provided about predicted mortality by strata. High-intensity centers also incorporated guideline adherence into their center's quality-of-care program, which included audits and feedback on performance. Using discharge of low-risk patients as the primary outcome and adherence to key processes of care as secondary measures, the authors were able to demonstrate that the high-intensity and, in some instances, the moderate-intensity approaches were superior to the competing strategies, often achieving a doubling in the outcomes of interest. For example, for discharging low-risk patients, the moderate- and high-intensity sites achieved rates of 61.0% and 61.9%, compared with 37.5% for the low-intensity group (P=.004).

Although examples of short-term adherence to evidence uptake initiatives are readily available, it is necessary to determine whether these will result in more than brief periods of compliance driven by enthusiasm and novelty. Chouaid et al47 describe a 2-year study that examined the implementation of an emergency asthma management guideline. A longitudinal educational and audit/feedback program was effective in achieving a prolonged change in practice.

Design Issues in Knowledge Translation Research 

Similar to the standard that is held for therapeutic interventions in individual patient care, randomized controlled trials are the criterion standard for studying the efficacy of interventions designed to increase evidence uptake. However, there are challenges associated with the study of complex interventions and system changes designed to influence practice. Consider a hypothetical study of the impact of a computerized physician order entry mechanism for requesting ankle radiographs designed to encourage compliance with the Ottawa Ankle Rules. Imagine that, in the intervention arm of this trial, a radiograph of the ankle can be requested only after the Ottawa Ankle Rules assessment has been entered into a computer-based ordering interface, whereas the control arm involves leaving clinicians on their own (no computer system) and free to apply or ignore the Ottawa Ankle Rules as they see fit. If the effectiveness of this strategy were being studied at only 1 site, it would be problematic to have physicians treat patients entered into both the intervention and control arms of the trial because the computerized physician order entry might have a blatant or unperceived influence on patient care in the control arm, thus diluting the effect, a phenomenon known as contamination. More appropriate would be to randomize physicians or, better yet, distinct ED sites to a computerized physician order entry or “current practice” strategy. As a result of these unique aspects related to the study of evidence uptake, cluster randomized designs and cluster-driven analyses figure prominently in this line of research.

This notion of randomizing groups or clusters of patients tied by a common bond (say, ED or treating MD) engenders its own difficulties.48 There is the possibility that outcomes can be influenced by cluster-specific patient or health care provider characteristics that are unrelated to the intervention under investigation. For example, if the ED physicians at a particular site in the above scenario are particularly computer savvy, this cluster-associated phenomenon could influence the results of a study examining an electronic radiograph ordering system. Consideration of the cluster phenomena has been inconsistent in the existing medical literature,49 and only recently have there been published recommendations on the manner in which cluster randomized trials should be reported.50

Although inferior in their strength of inference and more susceptible to bias, pre-/postdesigns can also be used in studying the effectiveness of knowledge translation interventions. Secular trends and observer bias are 2 sources of erroneous inference in such trials. For example, if a pre-/postdesign is used that shows that computerized physician order entry is effective in safely ensuring adherence to the Ottawa Ankle Rules, it could always be argued that factors other than the computerized physician order entry were responsible for an observed trend in the direction of decreased use of ankle radiographs. Interrupted time-series designs, in which the intervention is alternated with usual care for repeated periods, can help reduce some of the biases inherent in pre-/postdesigns when cluster designs are not feasible.

Attention bias, the effect of changing performance (only) while being observed (sometimes referred to as the “Hawthorne effect”)51 is a potential confounder in observational studies and randomized trials and is particularly a concern when the outcome is physician behavior. Dean et al52 studied admission rates of patients with community-acquired pneumonia before and after implementation of an unvalidated admission guideline in a network of outpatient facilities in Utah, simultaneously classifying patient severity with the validated patient outcome research team prediction rule. Admission rates decreased in all patient outcome research team severity classes despite the fact that practitioners did not actually adhere to the guideline. The process of alerting practitioners that their admission rates for patients with community-acquired pneumonia were to be monitored apparently accounted for the effect. The study by Yealy et al46 that compared various degrees of intensity for a pneumonia guideline implementation strategy was able to minimize attention bias through the use of interventions in all trial arms.

Regardless of study design, clinical researchers must remember that interventions that render patient care more efficient or efficacious are of benefit only if they are implemented in practice. In other words, the researcher should not only seek out evidence of benefit but also anticipate the design necessary for demonstrating an effective strategy for ongoing evidence uptake and implementation.

Future Directions 

Knowledge translation may best be viewed as the bridge that brings together continuing medical education, continuing professional development, and quality improvement in the hope of closing the research-to-practice gap.53 Some important resources already exist for accessing the evidence (in the form of original articles, systematic reviews, evidence-based practice guidelines and systems) that we should be attending to. However, emergency medicine still needs to define the greatest research-practice gaps in its specialty. This compilation of evidence-to-practice deficiencies would constitute a research agenda in knowledge translation as it relates to emergency medicine.

Be it the overuse of antibiotics in bronchitis or the underuse of validated decision rules to guide ordering of radiographs, the field of emergency medicine is replete with opportunities to design and test effective evidence uptake strategies and, in so doing, to bring emergency medicine practice fully into the era of evidence-based practice.

Back to Article Outline

References 

  1. Canadian Institutes of Health Research. Canadian Institutes of Health Research knowledge translation strategy 2004-2009. http://www.cihr-irsc.gc.ca/e/26574.html#defining2006;
  2. Sung NS, Crowley WF, Genel M, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289:1278–1287
  3. McGlynn EA, Asch SM, Adams J. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645
  4. Institute of Medicine, Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century [Institute of Medicine Web site]. http://www.iom.edu/report.asp?id=5432. Accessed September 5, 2006
  5. Cameron C, Naylor CD. No impact from active dissemination of the Ottawa Ankle Rules: further evidence of the need for local implementation of practice guidelines. CMAJ. 1999;160:1165–1168
  6. Holroyd BR, Wilson D, Rowe BH, et al. Uptake of validated clinical practice guidelines: experience with implementing the Ottawa Ankle Rules. Am J Emerg Med. 2004;22:149–155
  7. Stiell IG, Greenberg GH, McKnight RD, et al. A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med. 1992;21:384–390
  8. Bachmann LM, Kolb E, Koller MT. Accuracy of Ottawa Ankle Rules to exclude fractures of the ankle and mid-foot: systematic review. BMJ. 2003;326:417
  9. Auleley GR, Ravaud P, Giraudeau B, et al. Implementation of the Ottawa Ankle Rules in France: a multicenter randomized controlled trial. JAMA. 1997;277:1935–1939
  10. Stiell I, Wells G, Laupacis A, et al. Multicentre trial to introduce the Ottawa Ankle Rules for use of radiography in acute ankle injuries: Multicentre Ankle Rule Study Group. BMJ. 1995;311:594–597
  11. Graham ID, Stiell IG, Laupacis A, et al. Awareness and use of the Ottawa Ankle and Knee Rules in 5 countries: can publication alone be enough to change practice?. Ann Emerg Med. 2001;37:259–266
  12. Brehaut JC, Stiell IG, Visentin L, et al. Clinical decision rules “in the real world”: how a widely disseminated rule is used in everyday practice. Acad Emerg Med. 2005;12:948–956
  13. Woolley SL, Smith DR. Acute otitis media in children: there are guidelines but are they followed?. J Laryngol Otol. 2005;119:524–528
  14. Reuveni H, Asher E, Greenberg D, et al. Adherence to therapeutic guidelines for acute otitis media in children younger than 2 years. Int J Pediatr Otorhinolaryngol. 2006;70:267–273
  15. Ohman EM, Roe MT, Smith SC, et al. CRUSADE Investigators (Care of non-ST-segment elevation patients: insights from the CRUSADE national quality improvement initiative). Am Heart J. 2004;148:S34–S39
  16. Glasziou P, Haynes B. The paths from research to improved health outcomes. ACP J Club. 2005;142:A8–A10
  17. Haynes RB, Sackett DL, Gray JA, et al. Transferring evidence from research into practice, 2: getting the evidence straight. ACP J Club. 1997;126:A14–A16
  18. Buller-Close K, Schriger DL, Baraff LJ. Heterogeneous effect of an emergency department expert charting system. Ann Emerg Med. 2003;41:644–652
  19. McMaster Online Rating of Evidence. About MORE [McMaster Online Rating of Evidence Web site]. http://hiru.mcmaster.ca/more/AboutMORE.htm2006;
  20. Pluye P, Grad RM, Dunikowski LG, et al. Impact of clinical information-retrieval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies. Int J Med Inform. 2005;74:745–768
  21. Kelly KD, Travers A, Dorgan M, et al. Evaluating the quality of systematic reviews in the emergency medicine literature. Ann Emerg Med. 2001;38:518–526
  22. Haynes RB, Cotoi C, Holland J, et al. the McMaster Premium Literature Service (PLUS) Project (Second-order peer review of the medical literature for clinical practitioners). JAMA. 2006;295:1801–1808
  23. Bullard MJ, Meurer DP, Colman I, et al. Supporting clinical practice at the bedside using wireless technology. Acad Emerg Med. 2004;11:1186–1192
  24. Anderson DR, Wells PS, Stiell I, et al. Thrombosis in the emergency department: use of a clinical diagnosis model to safely avoid the need for urgent radiological investigation. Arch Intern Med. 1999;159:477–482
  25. Brown MD, Vance SJ, Kline JA. An emergency department guideline for the diagnosis of pulmonary embolism: an outcome study. Acad Emerg Med. 2005;12:20–25
  26. Lindsay P, Schull M, Bronskill S, et al. The development of indicators to measure the quality of clinical care in emergency departments following a modified-Delphi approach. Acad Emerg Med. 2002;9:1131–1139
  27. Bizovi KE, Wears R, Lowe RA. Researching quality in emergency medicine. Acad Emerg Med. 2002;9:1116–1123
  28. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293:1261–1263
  29. Green RS, Howes D. the CAEP Critical Care Committee (Hypothermic modulation of anoxic brain injury in adult survivors of cardiac arrest: a review of the literature and an algorithm for emergency physicians). Can J Emerg Med. 2005;7:42–47
  30. Trzeciak S, Dellinger RP, Abate NL, et al. Translating research into clinical practice: a 1-year experience with implementing early goal-directed therapy for septic shock in the emergency department. Chest. 2006;129:225–232
  31. Agency for Healthcare Research and Quality. Fart sheet: translating research into practice (TRIP)-II [Agency for Healthcare Research and Quality Web site]. http://www.ahrq.gov/research/trip2fac.htm2006;
  32. Grol R, Wensing M. What drives change? (barriers to and incentives for achieving evidence-based practice). Med J Aust. 2004;180(6 suppl):S57–S60
  33. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999;282:1458–1465
  34. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21:380–385
  35. O'Brien MA, Freemantle N, Oxman AD, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2001;(2):CD003030.
  36. Jamtvedt G, Young JM, Kristoffersen DT, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006;(2):CD000259.
  37. Grimshaw JM, Shirran L, Thomas R. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39(8 suppl 2):II2–II45
  38. Fesmire FM, Jagoda A. Are we putting the cart ahead of the horse: who determines the standard of care for the management of patients in the emergency department?. Ann Emerg Med. 2005;46:198–200
  39. Hayward RS. Clinical practice guidelines on trial. CMAJ. 1997;156:1725–1727
  40. Laupacis A. On bias and transparency in the development of influential recommendations. CMAJ. 2006;174:335–336
  41. Grimshaw J, Thomas RE, MacLennan G, et al. Health technology assessment 2004: vol 8: number 6. http://www.ncchta.org/execsumm/summ806.htm. Accessed
  42. Grad RM, Meng Y, Bartlett G, et al. Effect of a PDA-assisted evidence-based medicine course on knowledge of common clinical problems. Fam Med. 2005;37:734–740
  43. Sackett DL, Straus SE. Finding and applying evidence during clinical rounds: the “evidence cart.”. JAMA. 1998;280:1336–1338
  44. Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336:243–250
  45. Yealy DM, Auble TE, Stone RA, et al. The emergency department community-acquired pneumonia trial: methodology of a quality improvement intervention. Ann Emerg Med. 2004;43:770–782
  46. Yealy DM, Auble TE, Stone RA, et al. Effect of increasing the intensity of implementing pneumonia guidelines: a randomized, controlled trial. Ann Intern Med. 2005;143:881–894
  47. Chouaid C, Bal JP, Fuhrman C, et al. Standardized protocol improves asthma management in emergency department. J Asthma. 2004;41:19–25
  48. Wears RL. Advanced statistics: statistical methods for analyzing cluster and cluster-randomized data. Acad Emerg Med. 2002;9:330–341
  49. Eldridge SM, Ashby D, Feder GS. Lessons for cluster randomized trials in the twenty-first century: a systematic review of trials in primary care. Clin Trials. 2004;1:80–90
  50. Campbell MK, Elbourne DR, Altman DG. for the CONSORT Group (CONSORT statement: extension to cluster randomised trials). BMJ. 2004;328:702–708
  51. Campbell JP, Maxey VA, Watson WA. Hawthorne effect: implications for prehospital research. Ann Emerg Med. 1995;26:590–594
  52. Dean NC, Suchyta MR, Bateman KA, et al. Implementation of admission decision support for community-acquired pneumonia. Chest. 2000;117:1368–1377
  53. Davis D, Evans M, Jadad A, et al. The case for knowledge translation: shortening the journey from evidence to effect. BMJ. 2003;327:33–35

 Supervising editor: E. John Gallagher, MD

 Funding and support: The authors report this study did not receive any outside funding or support.

 Reprints not available from the authors.

PII: S0196-0644(06)02142-1

doi:10.1016/j.annemergmed.2006.08.022

Annals of Emergency Medicine
Volume 49, Issue 3 , Pages 355-363, March 2007

Article Tools

* Image Usage & Resolution