Annals of Emergency Medicine
Volume 36, Issue 3 , Pages 247-252, September 2000

Ten Challenges at the Intersection of Clinical Research, Evidence-Based Medicine, and Pain Relief☆☆☆

  • Alejandro R. Jadad, MD, DPhil

      Affiliations

    • Program in Evidence-based Care, Cancer Care Ontario; Health Information Research Unit, McMaster Evidence-based Practice Center, Canadian Cochrane Network and Centre, and Departments of Clinical Epidemiology and Biostatistics, and Anaesthesia, McMaster University, Hamilton, Ontario, Canada
    • ,
  • M.Soledad Cepeda, MD

      Affiliations

    • Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania, Philadelphia, PA.

Received 31 March 2000; accepted 4 April 2000.

Article Outline

Abstract 

[Jadad AR, Cepeda MS. Ten challenges at the intersection of clinical research, evidence-based medicine, and pain relief. Ann Emerg Med. September 2000;36:247-252.]

 

Back to Article Outline

Introduction 

Pain arguably is the best studied of all symptoms. There is a rich tradition of pain relief research spanning the past 40 years. Luminaries of the field including, among others, Henry Beecher, Louis Lasagna, Ray Houde, and Stanley Wallenstein, laid down explicit rules about the design and conduct of pain trials.1 During the second half of the 20th century, the number of randomized controlled trials (RCTs) evaluating analgesic interventions doubled every 10 years.2 In January 2000, we searched the Cochrane Controlled Trials Register, the largest collection of clinical trials in the world, and found that “pain” or “analgesia” is mentioned in more than 23,000 citations of RCTs (almost 9% of the total).

Of the many methods to evaluate analgesic interventions, RCTs are a very powerful and simple tool. A special value of RCTs in pain relief is their ability to minimize the confounding effects of unknown factors that may influence patients’ responses. In addition, analgesic trials help account for specific issues such as patient and practitioner expectations, the source of funding, and the placebo response. RCTs enroll patients or healthy volunteers who receive one intervention or another (the “control”) by random assignment. Prospective, double-blinded RCTs are a powerful technique to reduce bias in clinical trials, particularly pain relief trials in which suggestibility and placebo effects may distort the findings.

New methods, derived from clinical epidemiology and using RCTs as their “raw materials”,1 are also evolving. These methods include systematic reviews and meta-analysis and are based on the rigorous search for all evidence and the systematic and quantitative analysis of research. Yet despite the abundance of RCTs and systematic reviews regarding pain relief practices, the actual transfer of research evidence into clinical practice can be slow. Evidence-based decisionmaking is one way that can help ensure that the information coming from research will be considered and, if appropriate, incorporated with reasonable speed into decisions.3 Dealing with the abundance of RCTs and the increasing number of systematic reviews in pain relief creates numerous challenges for researchers, clinicians, funding agencies, ethics boards, journal editors, and the public.

This article, which explores the intersection of evidence-based medicine with pain relief practice and research, reviews 10 challenges regarding analgesic clinical trials and the translation of pain relief evidence into clinical practice. In addition, strategies, drawn on a more general discussion of the future of RCTs,4 for meeting these challenges will be proposed.

Back to Article Outline

Challenge 1: To avoid unnecessary duplication of effort 

Before embarking on a new trial, researchers, patients, and funding agencies should be confident that the question addressed by the trial has not been answered already. Given the abundance of trials, 2 strategies could accomplish this efficiently: systematic reviews of the literature, and an international register of ongoing trials.

Rigorous systematic reviews 

The first strategy to avoid unnecessary duplication of effort is a rigorous systematic review of the topic addressed by any proposed RCT.4 In pain relief, 2 important resources exist to facilitate this effort: the Cochrane Library and the Oxford Pain Internet Site.

The Cochrane Library is the main product of the Cochrane Collaboration, a rapidly growing international organization that facilitates well-informed decisions about health care by preparing, maintaining, and improving the health care interventions.5 The Cochrane Collaboration has been described as “an enterprise that rivals the Human Genome Project in its potential implications for modern medicine.”6 It welcomes participation from any person interested in collaborating with others. The Cochrane Collaboration recently acknowledged the importance of pain relief by registering a collaborative review group for Pain, Palliative and Supportive Care (“PaPaS”). Now issued quarterly on CD-ROM, the Cochrane Library is shifting toward continuous updating and Internet accessibility alongside MEDLINE.

The Cochrane Library, reviewed previously in the Annals Evidence-Based Emergency Medicine series,7 consists of 3 databases that contain information on analgesic interventions.

•The Cochrane Controlled Trials Register is perhaps the most advanced and comprehensive source of RCTs in health care. By the first quarter of 2000, it contained citations for more than 260,000 RCTs identified through high-yield searches of bibliographic databases and through extensive hand-searching of journals.

•The Cochrane Database of Systematic Reviews (CDSR) is a rapidly growing collection of regularly updated systematic reviews of the effects of health care. Prepared by members of the Cochrane Collaboration, the database includes protocols of reviews in progress. By the first quarter of the year 2000, it contained 213 complete reviews and 235 protocols that included “pain,” “analgesia,” “analgesics,” and related terms.

•The Database of Abstracts of Reviews of Effectiveness (DARE) contains structured abstracts of thousands of systematic reviews from around the world, collected and appraised at the National Health Service Centre for Reviews and Dissemination at the University of York, England. DARE also includes brief records of reviews useful for background information, abstracts of reports of health technology agencies worldwide, and abstracts of reviews in the journals ACP Journal Club and Evidence-Based Medicine. In March 2000, this database included 196 potentially relevant reviews.

The second resource for identifying existing RCTs or systematic reviews in pain relief is the Oxford Pain Internet Site (www.jr2.ox.ac.uk/Bandolier/painres/painpag/index.html ). This site provides electronic access to systematic reviews of pain relief RCTs. Many of these systematic reviews are also available in the textbook, An Evidence-Based Resource for Pain Relief. 1

International register of ongoing trials 

If ethics committees registered trials on approval, it would be easy to create databases or registers with information on such trials and to make them accessible to people anywhere in the world via the Internet. Such databases would decrease duplication of research efforts and diminish publication bias that overrepresents positive results. Despite many potential benefits to users, most efforts to create such databases and registers have failed.8, 9 Public debates about the need for such databases in pain relief should be promoted, with full disclosure from parties opposed to compulsory registration of trials. Representatives from mass media and consumer groups should be invited to these discussions.4

Back to Article Outline

Challenge 2: To use all existing trials to guide clinical decisions 

Frequently, more than one trial has addressed the same clinical question. Because these trials enrolled different groups of people, in different settings, and used the interventions differently, it is unlikely that they would all agree. Sometimes, trials on the same topic yield opposite results. Thus, it may be risky for clinicians or patients to make decisions based on a single trial. To take advantage of the best available knowledge, decisionmakers should consider as many relevant trials as possible as well as other types of information. A rigorous systematic review is the most efficient way to access the information from all existing trials on a given topic. Clinicians and patients could benefit from consulting resources such as the Cochrane Library or the Oxford Pain Internet Site when making treatment decisions.

Back to Article Outline

Challenge 3: To improve the quality of RCTs in pain relief 

The following are examples of efforts needed to improve the usefulness of RCTs.

Reduction in the likelihood of bias in RCTs 

Human beings are prone to bias. Empirical research consistently shows that most RCTs exaggerate the benefits of treatments studied.4, 9 Several factors contribute to this exaggeration of benefits.4 Many RCTs are designed, conducted, and reported by researchers whose careers are closely linked to the interventions they evaluate. Most patients want interventions to be effective. And funding is often provided by organizations that thrive on breakthroughs and positive results. The latter is particularly true for studies funded by pharmaceutical companies.10 Herculean efforts will be required to minimize the influence of potential secondary gain on trial findings. Such efforts could include prevention of direct funding of RCTs by developers of interventions, and minimization of the participation of individuals with clear conflicts of interest in the design, execution, and reporting of trials.4

Address clinically relevant questions 

Many trials fail to meet the needs of clinicians, policymakers, and patients because they are designed to meet the needs of academics and funding organizations. For example, relatively few RCTs include head-to-head comparisons of current analgesic interventions, assess the impact of analgesics on quality of life or resource utilization, or express their results in ways that clinicians and patients can easily apply to their decisions. We cannot afford to perpetuate this situation. Including consumers and health care providers as active members of research teams could easily increase the clinical relevance of RCTs. This approach is proving successful in the development of tools to promote shared decisionmaking and in the design and dissemination of systematic reviews.11, 12, 13 We also need effective strategies to increase the role of health planners, managers, and policymakers in the design, execution, and dissemination of RCTs.

More collaborative efforts needed 

Often, the sample size of RCTs evaluating analgesic interventions is so small that they provide incomplete answers to complex questions.14 Increasing the sample size of RCTs in pain relief will require efficient collaboration among different groups of researchers, often working in different countries. Successful collaboration will not be easy, as it will involve an important departure from current mechanisms to fund research and to reward researchers. Important lessons could be learned from “mega-trials” being conducted in cardiovascular and oncology research.

Back to Article Outline

Challenge 4: To improve the quality of reporting of RCTs 

RCTs in all areas of health care are reported incompletely, and pain trials are no exception.15 Most reporting problems could be eliminated if all journals that published pain relief research, funders, and researchers endorsed and embraced the CONSORT statement.16, 17, 18 Although more than 70 journals including the Annals of Emergency Medicine endorse the CONSORT statement, many journals devoted solely to pain research still do not. The CONSORT statement, the product of an extensive international collaborative process, was developed to help authors, peer reviewers, and editors improve the standard published RCT. The statement includes a checklist of 21 items that address different aspects of an RCT and provides a flow diagram that can be used to map the progress of participants through a trial. In addition, readers can use the statement to judge the completeness of articles describing RCTs. A similar checklist for improving the quality if reporting of meta-analyses was recently published in the QUOROM statement.19

Journals can also assist the interpretation and application of research data by clinicians by making their data research more “evidence-based medicine friendly.” Journals could require pain relief researchers to report their results as categorical data (eg, how many subjects experienced “adequate,” “complete,” or “satisfactory” pain relief). Continuous data (eg, mean pain scores) from visual analog scales are appropriate for statistical and regulatory purposes, but are almost devoid of meaning for clinicians and patients.

Among the many ways to represent results from clinical trials, number needed to treat (NNT) is gaining widespread use and acceptance.20 The computation of NNT, however, requires dichotomous data21 and cannot be computed directly in a suitably reliable fashion from the continuous data, particularly when the results do not follow a normal distribution. Dichotomous data contain only 2 classes such as male or female or lived or died. In addition to categorical data, journals could also require pain relief researchers to report their results as dichotomous data (eg, how many subjects experienced 75% pain relief—either they experienced 75% relief or they did not; how many experienced unacceptable adverse effects—either the adverse effects were acceptable or not) and publish the NNTs for the beneficial or harmful aspects of different analgesic therapies.

The success of any effort to improve reporting of RCTs in pain relief also depends on the commitment of researchers to produce good reports of good RCTs. Irresponsible, ambitious researchers may understate flaws in their studies, producing good reports of bad RCTs to comply with editors’ requests and to have their manuscripts published.4

Back to Article Outline

Challenge 5: To find better ways to present the results of trials to frontline users 

If RCTs are to influence decisions on the treatment of pain, better dissemination of their findings is required. Written by researchers for other researchers, most reports of RCTs are unpalatable to clinicians, patients, journalists, policymakers, and other audiences. Although narrow targeting may be appropriate for preliminary results, it should be avoided when a trial answers a clinically important question. The separation of research-oriented and clinician-oriented journals would be a good starting point.22 Perhaps we also need consumer-oriented and policymaker-oriented journals.4 Regardless of the strategy, if trials are to have the greatest impact, we need to depart from the traditional paper-based, jargon-laden, and lengthy article that is unintelligible to most readers, and move to more engaging and intellectually appealing ways to present information to users.23 The modern decisionmaker prefers clear, short, unequivocal, personal, vivid, engaging, meaningful, and relevant messages.24 To succeed, we will need input from individuals with expertise in marketing, education, graphic design, and advertising.4 New modes of electronic publication, particularly those based on multimedia delivered through the Internet, are likely to offer more attractive ways to present scientific information in the near future.25

Back to Article Outline

Challenge 6: To ensure that all RCTs, once completed, are promptly available to the users 

Trials have little value, even when perfectly designed, conducted, and reported, if they are not available to users. Delays in releasing research findings can have harmful effects, especially if trials with positive results are published years before those with negative results.26 Ensuring that all trials, regardless of whether findings are positive or negative, are published soon after completion will reduce publication bias and time lag bias.9

We should educate patients to demand publication of study results as a prerequisite for their participation in research studies. Individuals who consent to participate in research do so with the understanding that the work will make a contribution to knowledge.27 Ethics committees can encourage researchers and funding agencies to make the results of their studies available, either through biomedical journals or other media such as the Internet. Because of their potential impact on health care decisions, it could be argued that trials that have flawed designs or provide incomplete or delayed reports are unethical.9

Ideally, decisionmakers would access RCTs as needed. Achieving this will require the best available information technology and decisionmakers who can and do use it. Never before have there been so many powerful tools that allow providers and consumers to access information. As described in Challenge 1, resources now provide fast, easy access to RCTs and other types of evidence.28 The rapid development of the Internet promises even better opportunities. Keeping up with technologic developments will not be an easy task for most decisionmakers. Further, we lack knowledge about patterns of use of different information technologies by different users. If users’ skills are to match technologic possibilities, we will need research to address existing knowledge gaps and develop strategies to hasten the adoption of new technologies by people with different backgrounds, expectations, motivations, and skills.

Back to Article Outline

Challenge 7: To ensure that decisionmakers understand RCTs 

Even if RCTs were perfectly designed and their results readily available, they could not influence health care decisions and outcomes if users could not understand them. Most people still do not understand the concept of randomization and its strengths, the sources of bias in RCTs, and the role of RCTs in health care decisionmaking. Most efforts to promote the understanding of RCTs (and of research in general) focus on researchers, particularly those in training. Little has been done to promote an understanding of research among many of its end users. Although health professionals, policymakers, health planners, and managers may have access to courses on research methodology, such courses may not include specific, structured content about RCTs. The situation for patients, their family members, and other laypersons is worse.29 These latter groups, whom RCTs exist to serve, are not equipped to handle research information, let alone RCTs. Journalists, who have a profound influence on the dissemination and impact of research information, are likely to be in a similar situation.

Against this background, development and implementation of effective strategies to increase users’ understanding of research, and RCTs in particular, are top priorities. The success of these strategies will depend not only on how research results are presented to users, but also on our ability to recognize, understand, and overcome specific barriers to the adequate use of RCTs by decisionmakers. Success will also depend on how well these strategies target all groups of potential and actual users of research, not only researchers and academic clinicians.

Back to Article Outline

Challenge 8: To integrate the findings of clinical trials with information from other types of research 

Researchers, clinicians, funding agencies, and policymakers tend to place RCTs at the top of the evidence hierarchy, assuming that such trials are always better than other study designs. Although this hierarchy may be valid in theory, it may in practice be inappropriate because it is incorrect to give more weight to a flawed RCT than to a rigorous observational study (eg, a cohort study). Furthermore, RCTs are just one type of research design. Clinical decisions should take into account other types of quantitative studies (eg, cohort and case-control studies) and qualitative research efforts. However, we still lack a clear understanding of how to weight other pain relief research study designs with differing degrees of methodologic rigor and how to integrate their results with the information provided by RCTs.

Back to Article Outline

Challenge 9: To balance the findings of clinical research with other ways of knowing 

Over thousands of years, human beings have developed powerful but nonscientific ways of knowing. These “hard-wired” primal knowledge tools still have a profound effect on all the decisions we make in our everyday lives, as well as on the health care we advocate, practice, or receive. They include anecdotes, rules of thumb, and tacit knowledge.30, 31 Efforts to increase our understanding of RCTs and other types of research in pain relief should be coupled with efforts to understand those other ways of knowing.

Back to Article Outline

Challenge 10: To balance the findings of clinical trials with our values, preferences, and circumstances 

Research information is modulated, not only by other types of information, but also by the values and preferences of all the decisionmakers involved and the specific circumstances in which most decisions are made.23 Our efforts to understand the role of values, preferences, and circumstances should be extended beyond the individual level. We also need a better understanding of the dynamics of collaborative decisionmaking (ie, between nurses and physicians, pharmacists and hospital administrators, nurses and patients, and politicians). This understanding will require different research approaches and interdisciplinary input.

In summary, the RCT, one of the simplest yet most powerful tools of research, has been used extensively to evaluate analgesic interventions. We now face many challenges in ensuring maximum practical benefit from RCTs. These challenges are as diverse as the efficient design of new studies and the integration of the findings of RCTs into clinical decisions.32 Meeting these challenges will require unprecedented levels of commitment, communication, and collaboration among frontline clinicians, researchers, research funding agencies, journal editors, policymakers, journalists, and the public. We all will be the benefactors of such commitment.

Back to Article Outline

References 

  1. McQuay HJ, Moore RA. An Evidence-Based Resource for Pain Relief. Oxford, NY: Oxford University Press; 1998;
  2. Jadad AR, Carroll D, Moore A, et al.  Developing a database of published reports of randomised clinical trials in pain research. Pain. 1996;66:239–246
  3. Waeckerle JF, Cordell WH, Wyer P, et al.  Evidence-based emergency medicine: integrating research into practice. Ann Emerg Med. 1997;30:626–628
  4. Jadad A. Randomised Controlled Trials: A User’s Guide. London: BMJ Books. http://www.bmjpg.com/rct1998; Available at
  5. Jadad AR, Haynes RB. The Cochrane Collaboration—advances and challenges in improving evidence-based decision making. Med Decis Making. 1998;18:2–9
  6. Naylor CD. Grey zones of clinical practice: some limits to evidence-based medicine. Lancet. 1995;345:840–842
  7. Rowe BH, Alderson P. The Cochrane Library: a resource for clinical problem solving in emergency medicine. Ann Emerg Med. 1999;34:86–90
  8. Dickersin K. How important is publication bias? A synthesis of available data. AIDS Educ Prev. 1997;9:15–21
  9. Jadad AR, Rennie D. The randomized controlled trial gets a middle-aged checkup. JAMA. 1998;279:319–320 [editorial]
  10. Bero LA, Rennie D. Influences on the quality of published drug studies. Int J Technol Assess Health Care. 1996;12:209–237
  11. Jadad A, Whelan T, Rayno L, et al.  “A team approach to pain relief”: a guide developed with patients and family members. Support Care Cancer. 1996;3:245; [abstract]
  12. Bero LA, Jadad AR. How consumers and policymakers can use systematic reviews for decision making. Ann Intern Med. 1997;127:37–42
  13. Vanchieri C. Cancer researcher’s odyssey benefits many patients. J Natl Cancer Inst. 1998;90:184–186 [news]
  14. Moore RA, Gavaghan D, Tramer MR, et al.  Size is everything—large amounts of information are needed to overcome random effects in estimating direction and magnitude of treatment effects. Pain. 1998;78:209–216
  15. Jadad AR, Moore RA, Carroll D, et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996;17:1–12
  16. Begg C, Cho M, Eastwood S, et al.  Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–639
  17. CONSORT Web site. Available at: http://www.consort-statement.org .
  18. Moher D. CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials. Consolidated Standards of Reporting Trials. JAMA. 1998;279:1489–1491
  19. Moher D, Cook DJ, Eastwood S, et al.  Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement. Lancet. 1999;354:1896–1900
  20. Cordell WH. Number needed to treat (NNT). Ann Emerg Med. 1999;33:433–436
  21. Moore A, McQuay H, Gavaghan D. Deriving dichotomous outcome measures from continuous data in randomised controlled trials of analgesics. Pain. 1996;66:229–237
  22. Haynes RB. Loose connections between peer-reviewed clinical journals and clinical practice. Ann Intern Med. 1990;113:724–728
  23. Locke D. Voices of science. Am Scholar. 1998;67:103–114
  24. Enkin MW, Jadad AR. Using anecdotal information in evidence-based health care: heresy or necessity?. Ann Oncol. 1998;9:963–966
  25. Jadad AR, Haynes RB, Hunt D, et al.  The Internet and evidence-based decision-making: a needed synergy for efficient knowledge management in health care. CMAJ. 2000;162:362–365 Available at http://www.cma.ca/cmaj/vol-162/issue-3/0362.htm
  26. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA. 1998;279:281–286
  27. Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990;263:1405–1408
  28. Haynes RB, Jadad AR, Hunt DL. What’s up in medical informatics?. CMAJ. 1997;157:1718–1719
  29. Health literacy: report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. JAMA. 1999;281:552–557
  30. Polanyi M. The Tacit Dimension. New York, NY: Doubleday; 1966;
  31. Jadad A, Enkin M. The new alchemy: transmuting information to knowledge in an electronic age. CMAJ. In press.
  32. Haynes RB, Sackett DL, Gray JM, et al.  Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions. [editorial]. ACP J Club. 1996;125:A14–A16

☆ Editor’s Note: This article is based on an original article that appeared in the Summer 1999 edition of the newsletter Pain: Clinical Updates, published by the International Association for the Study of Pain (IASP). The original newsletter article has been modified and updated for Annals of Emergency Medicine. We gratefully acknowledge the support of Daniel B. Carr, MD, (newsletter editor), and Louisa E. Jones (IASP) for allowing us to use and revise the original article.

☆☆ Reprints not available from the authors. Address for correspondence: Alejandro R. Jadad, MD, DPhil, Department of Clinical Epidemiology and Biostatistics, McMaster University, HSC-3H7, 1200 Main Street West, Hamilton, ON L8N 3Z5 Canada; 905-525-9140; E-mail: jadada@fhs.mcmaster.ca.

PII: S0196-0644(00)36113-3

doi:10.1067/mem.2000.108095

Annals of Emergency Medicine
Volume 36, Issue 3 , Pages 247-252, September 2000

Article Tools