Identification of Randomized Controlled Trials From the Emergency Medicine Literature: Comparison of Hand Searching Versus MEDLINE Searching☆☆☆★

Article Outline

• Abstract
• INTRODUCTION
• MATERIALS AND METHODS
• RESULTS
• DISCUSSION
• Acknowledgements
• References
• Copyright

Abstract 

Study objective: As part of an ongoing project to identify all the randomized controlled trials (RCTs) in the emergency medicine literature, in association with the Cochrane Collaboration, 2 discrete studies were undertaken; the first, to compare motives for active participation in hand searching of the literature by emergency medicine professionals, and the second, to compare hand searching with MEDLINE searching of a number of emergency medicine journals. Methods: All listed members of the British Association for Emergency Medicine (BAEM) and the Society for Academic Emergency Medicine (SAEM) received a standard letter outlining the objectives of the project, with 1 of 3 headings assigned on an alternate basis. Recruited volunteers hand searched journals prioritized from the emergency medicine literature. Each issue of each journal was hand searched for RCTs. In addition, a comprehensive MEDLINE search was conducted for each journal. The yields of RCTs from the 2 searching methods were compared for all journals and for each journal individually. Results: No clear motivation for participation in this work could be ascertained because of the low response rates from BAEM and SAEM (10.1% and 1.8%, respectively). Only 18 (29.0%) of the 62 journals identified were indexed by MEDLINE. In the 14 journals indexed by MEDLINE for which hand searching was completed, a total of 710 RCTs were identified by a combination of the 2 approaches; of these, 592 (83.4%) were identified by hand searching alone and 483 (68.0%) by MEDLINE searching alone. Both methods identified 365 (51.4%) RCTs; hand searching revealed an additional 227 (32.0%) that were not identified by MEDLINE searching, and MEDLINE searching found 118 (16.6%) that were not identified by hand searching. The difference between the proportions identified by hand searching and by MEDLINE searching (15.4%; 95% confidence interval [CI], 12.7% to 17.9%) was statistically significant (McNemar’s χ² test, 1 df , 33.8; P <.0001). This difference was not significant for 8 of the journals. Conclusion: The response rates from mailing to members of the relevant professional organizations letters requesting participation in this work were very low and suggested that such an approach was not cost-effective. However, no formal costing exercise was undertaken. Searching results showed that, in the 14 emergency medicine journals indexed by MEDLINE for which hand searching was completed, hand searching led to identification of additional RCTs (primary articles) not found through MEDLINE searching. However, hand searching, although statistically significantly better than MEDLINE searching, failed to identify some of the RCTs found by MEDLINE searching, suggesting that hand searching is not a “gold standard” method and that the dual approach, promoted by the Cochrane Collaboration, may be the optimal approach for journals indexed by MEDLINE. [Langham J, Thompson E, Rowan K: Identification of randomized controlled trials from the emergency medicine literature: Comparison of hand searching versus MEDLINE searching. Ann Emerg Med July 1999;34:25-34.]

See related articles, p. 75 and p. 86.

Back to Article Outline

INTRODUCTION 

With the advent of new journals, the amount of scientific literature is ever increasing. As a result, health care professionals are more likely to rely on review articles to keep abreast of developments. Review articles are therefore very influential, but they often are subjective appraisals and syntheses of selected published works and consequently carry a potential for the introduction of bias by the reviewers.¹ In addition, reviews carried out nonsystematically may lead to seriously misleading recommendations for clinical practice.² In contrast, rigorous systematic reviews of the scientific literature employ more standardized, reproducible methods to identify eligible studies and to abstract and analyze relevant data.³

Rigorously conducted experiments—that is, randomized controlled trials (RCTs) of sufficient statistical power—are deemed to represent the “gold standard” in research design to detect important beneficial or harmful effects of existing or new interventions, particularly if a pragmatic rather than an explanatory design is adopted.⁴ However, because individual RCTs are often too small to provide estimates of treatment differences of acceptable precision,⁵ any single RCT can provide only limited guidance for clinical practice. As a result, systematic reviews of all relevant RCTs have been suggested as useful to guide both clinical practice and future research.

There are 2 initial and essential steps in preparing systematic reviews of RCTs: identifying the relevant sources and searching the relevant sources to identify the eligible studies. These tasks are not straightforward,⁶, ⁷ because not all journals are indexed by MEDLINE, and those that are indexed are more likely to be English-language journals. Even for journals indexed by MEDLINE, it may be difficult to retrieve all relevant studies. Research in other areas has shown that MEDLINE electronic searching for RCTs results in retrieval of approximately only half of the relevant studies, due to inaccurate indexing by MEDLINE.⁸ RCTs are sometimes published as conference proceedings, which are not indexed by MEDLINE. Again, research in other areas has shown that, on average, only half of conference abstracts reporting results of RCTs reach full publication,⁹ and such studies are more likely to be published if they show a positive result.¹⁰ Therefore reviews of primary research using only trials identified through MEDLINE are liable to be subject to selection bias, English-language bias, and publication bias.

In April 1993, a project commenced with the objective of identifying all RCTs in the emergency medicine literature. This work formed part of the then-emerging international Cochrane Collaboration, which was developed in response to the need for more reliable reviews of research evidence. The Cochrane Collaboration has an overall objective to prepare, maintain, and disseminate systematic, up-to-date reviews of the effects of health care.¹¹ The Cochrane Collaboration consists of topic-based Collaborative Review Groups, with each group responsible for the preparation of systematic reviews relevant to their particular topic (eg, health care interventions in pregnancy and childbirth). To ensure wide dissemination of the systematic reviews, they are published by the electronic Cochrane Library in the Cochrane Database of Systematic Reviews.¹²

To overcome the potential biases of relying solely on identification of RCTs through MEDLINE and other electronic databases, the Cochrane Collaboration promotes the use of hand searching of the literature. Given the laborious nature of hand searching, 2 discrete studies were undertaken as part of the ongoing project. The objective of the first study was to compare motives for active participation in hand searching of the literature by emergency medicine professionals. The objective of the second study was to compare hand searching with MEDLINE searching of a number of emergency medicine journals.

Back to Article Outline

MATERIALS AND METHODS 

Volunteers to assist with hand searching of the emergency medicine literature were identified by 4 different approaches. First, all listed members of the British Association for Emergency Medicine (BAEM) and the Society for Academic Emergency Medicine (SAEM) received a standard letter outlining the objectives of the work and the background to the Cochrane Collaboration and requesting help with the hand searching of the emergency medicine literature. Every member from the alphabetical listing for both organizations was assigned, on an alternate basis as part of a quasirandomized study, to receive the same letter with 1 of 3 headings: “The pursuit of medical practice based on scientific evidence”; “The chance of a free journal subscription”; or “Join leading colleagues in an exciting new venture” (Figure 1).

Figure 1. Text of recruitment letter

The second approach to identify volunteers was to ask known professional contacts to assist with hand searching and to nominate colleagues who might be interested in collaborating. The third approach was through publicity: The authors were invited both to exhibit and to present a paper about the work at the annual conference of the BAEM in the years 1994 and 1995. In 1994, the BAEM hosted the Fifth International Conference on Emergency Medicine in association with the American College of Emergency Physicians, the Australasian College of Emergency Physicians, and the Canadian Association of Emergency Physicians, providing an international audience. During the presentation of the work in the plenary session, delegates were encouraged to visit the Cochrane exhibit to register their interest in participating in the work. Indications of interest were followed by direct correspondence after the conference. The fourth approach was to journal editors, requesting assistance with the hand searching of their own journals.

When a volunteer indicated an interest in a specific topic in emergency medicine or an interest in collaborating in a systematic review, such information was communicated to the volunteer’s local Cochrane Centre and to the Australasian Cochrane Centre, which maintains a global directory of individuals’ interests on behalf of the Cochrane Collaboration.

For the second study, to compare hand searching with MEDLINE searching in a subset of the emergency medicine literature, the relevant journals had to be identified and prioritized. Four approaches were used. First, relevant journals were identified in ULRICH’s Plus International Serials Database¹³ by a text search using the keywords “trauma,” “accident medicine,” and “emergency medicine.” Second, the shelves of major medical libraries in Boston (USA) and London (UK) were searched for relevant journals. Third, all relevant journals indexed by MEDLINE¹⁴ were identified with the use of keywords similar to those used for ULRICH’s. Finally, professional contacts in the field of emergency medicine were asked to identify any additional journals omitted from the list. Special emphasis was placed on the identification of non–English-language journals through professional contacts.

A priority list of journals for hand searching was identified in 2 ways. First, all of the identified journals indexed by MEDLINE were ranked by potential yield of RCTs. This was achieved either by undertaking a quick search on MEDLINE for the word “random$” ($=wildcard) in the text of the title or abstract of articles published from 1989 to 1993 or by using Phase One of a comprehensive search strategy for RCTs (Figure 2), which was developed by, and is available from, the Cochrane Collaboration.⁸ Second, as part of the ongoing work, hand searching priority was given to journals for which volunteers showed a particular interest and to journals readily available to a volunteer or to the project team.

Figure 2. Comprehensive MEDLINE search strategy to identify RCTs.

Note: Format shown is for silver platter version 3.10. Upper case denotes controlled vocabulary; lower case denotes freetext terms. Those wishing to run this search strategy are recommended to seek the advice of a trained medical librarian.

Prioritized journals were hand searched retrospectively, from January 1948 or from the first issue forward, and prospectively, as each new issue of the journal was published. The year 1948 was selected by the Cochrane Collaboration because it is generally agreed that the first clinical trial with a properly randomized control group (a trial of streptomycin for the treatment of pulmonary tuberculosis) was published in that year.¹⁵ Each volunteer was asked to hand search the journal to which the volunteer was assigned, either retrospectively for a given period (usually 5 years) or prospectively.

Each hand searcher received an information package that included details of the allocated issues of the journal to be hand searched; guidelines for hand searching developed by the project team; a thesaurus of terms used to describe RCTs; a form to be completed for each RCT identified; examples of RCTs identified in other emergency medicine journals; information about the Cochrane Collaboration; and an agreement form to undertake the work, which was to be signed and returned by the hand searcher.

The RCTs to be included were primary articles reporting the results of an RCT, conference abstracts reporting the results of an RCT, and abstracts reporting the results of an RCT published elsewhere. For all RCTs identified, a form was completed and sent with the photocopied RCT to the project team.

To assist those prospectively hand searching journals, editors of journals on the priority list were approached by letter explaining the objectives of the work and requesting a complimentary subscription to their journal so that it could be allocated to a volunteer hand searcher.

To ensure that there was no duplication in hand searching activities within the Cochrane Collaboration, information about ongoing and completed hand searching activities was communicated to the International Register of RCTs located at the Baltimore Cochrane Center.

The reliability of the hand searching done by an individual volunteer was checked by assigning an overlapping year of the same journal to a second volunteer and to a member of the project team (JL, ET). However, formal testing was not undertaken.

A comprehensive MEDLINE search strategy for RCTs (Figure 2) was conducted by a member of the project team (JL, ET) for each of the journals indexed by MEDLINE for which hand searching was completed. The searches were carried out from 1966 (when MEDLINE commenced indexing), or from the year the journal was first indexed, until the end of 1995. To account for the lag that occurs between publication of an article and data entry onto MEDLINE, searches were conducted from at least 6 months after the end of 1995.

All studies identified as RCTs, either by hand searching or by MEDLINE searching, were coded by the project team according to the Cochrane guidelines.¹⁶ Studies were coded as definite RCT —a clinical study on humans in which individual subjects were definitely assigned prospectively to 1 of 2 or more alternative forms of health care using a random method of allocation (eg, randomization table); as a quasi-RCT —a similar study but one in which the method of allocation was quasirandom (eg, alternation, date of birth, case record number); or as a possible RCT —a study in which the subjects were possibly assigned prospectively by a random or quasirandom method of allocation but for which inadequate reporting made the method impossible to discern. For RCTs identified by MEDLINE, when such coding was not possible from the title and abstract alone, the original article was checked.

Back to Article Outline

RESULTS 

The results of the comparison of motives for active participation in hand searching of the literature by emergency medicine professionals are presented as response rates from members of the BAEM and SAEM by type of letter heading in Table 1.

Table 1. Comparison of motives for active participation in hand searching of the emergency medicine literature by emergency medicine professionals.

BAEM, British Association for Emergency Medicine (UK); SAEM, Society for Academic Emergency Medicine (USA); Heading 1, “The pursuit of medical practice based on scientific evidence”; Heading 2, “The chance of a free journal subscription”; Heading 3, “Join leading colleagues in an exciting new venture.”

Response rates were very low for both organizations, although significantly more members responded from the BAEM (10.1% versus 1.8%, χ², 1 df , 107.2; P <.0001). For both organizations, Heading 1 (“The pursuit of medical practice based on scientific evidence”) attracted the largest response, followed by Heading 2, then Heading 3. However, differences were not significant: for BAEM, 11.6% versus 10.4% versus 8.4% (χ², 2 df , 1.42; P =.23); for SAEM, 2.3% versus 1.9% versus 1.5% (χ², 2 df , 1.46; P =.23).

In total, 179 volunteers indicated an interest in assisting with the hand searching of the emergency medicine literature. However, interest was translated into action by only 46 persons (25.7%). The final yield of active hand searchers was lowest for journal editors and then for the professional organizations. Editors of 16 journals were approached for complimentary subscriptions of their journals; 11 (68.8%) obliged. Although no editor refused, 5 did not respond despite repeated requests by mail.

As part of the ongoing project, 62 journals were identified by the 4 approaches described. The journals are listed alphabetically by country of publication in Table 2.

Table 2. Alphabetical list by country of publication of the 62 emergency medicine journals identified to date, with translations () or former names {}.

There are obvious omissions from the list, for example the journal Chest (formerly Diseases of the Chest ). These omissions arise in part because of the existence of other allied Cochrane groups, such as those relevant to intensive care, and surgery and anesthesia. For example, hand searching of the journal Chest has been undertaken by the group working on intensive care medicine.

Of the 62 emergency medicine journals identified to date, only 18 (29.0%) are indexed by MEDLINE. The list in Table 3 shows the initial 12 journals ordered by the yield from the MEDLINE search of entries from 1989 through 1993 for “random$” in the text of the title or abstract.

Table 3. Priority list of journals for hand searching (N=26).

—, Journal not indexed on MEDLINE.

A further 6 journals are ordered by the yield from the MEDLINE search using Phase One of the comprehensive search strategy for RCTs (Figure 2) for the years 1991 through 1995. Finally, an additional 8 journals were added to the priority list because of interest or availability to volunteers. Of the 26 prioritized journals, retrospective hand searching was completed for 18 (69.2%); of these, 14 were indexed by MEDLINE.

A comparison of hand searching and MEDLINE searching is presented in Table 4.

Table 4. Number of primary RCTs identified from MEDLINE searching and hand searching in 14 emergency medicine journals indexed by MEDLINE.^*

The table excludes RCTs from both conference abstracts and abstracts (defined previously), because they are not indexed by MEDLINE. Results are presented solely for primary articles in the 14 journals indexed by MEDLINE for which hand searching was completed.

A total of 710 RCTs were identified by a combination of the 2 approaches, with hand searching identifying 592 (83.4%) and MEDLINE 483 (68.0%). Both methods identified 365 (51.4%) RCTs; hand searching revealed an additional 227 (32.0%) that were not identified by MEDLINE searching, and MEDLINE searching found 118 (16.6%) that were not identified by hand searching. The difference between the proportions identified by hand searching and by MEDLINE searching (15.4%; 95% CI, 12.7% to 17.9%) was statistically significant (χ², 1 df , 33.8; P <.0001). The same results are presented for each journal separately in Table 5.

Table 5. Number of primary RCTs identified by hand searching and MEDLINE searching for the 14 journals indexed by MEDLINE, by journal.

The proportional difference between RCTs identified by hand searching and those identified by MEDLINE searching was not significant for 8 of the journals. The total hand searching yield, coded by type of RCT, and comparative results for MEDLINE searching are shown in Table 6.

A total of 1,708 RCTs were identified (1,234 definite RCTs, 131 quasi-RCTs, and 343 possible RCTs) in the 18 hand searched journals.

Back to Article Outline

DISCUSSION 

The response rates from mailing to members of the relevant professional organizations letters requesting participation in this work were very low and suggested that such an approach was not cost-effective. However, no formal costing exercise was undertaken, nor do any formal cost comparisons for other recruitment methods exist. The higher response rate from the British organization may have occurred because the Cochrane Collaboration was established in the United Kingdom and was only emerging at the time the letters were sent. In addition, the letter referred to UK- rather than US-based experts. No clear motivation for collaboration in this work could be determined because of the low response rates. The heading, “The pursuit of medical practice based on scientific evidence” yielded the highest response from both organizations, but the results did not reach statistical significance.

The 62 journals on the current list of emergency medicine journals is by no means comprehensive. In part, this is a result of the incorporation of some relevant journals into the work of other groups within the Cochrane Collaboration. However, it may also reflect a failure to identify other relevant journals, especially non–English-language ones. It is acknowledged that not all journals on the current list publish primary research. It should be noted, however, that only 18 (29.0%) of the 62 emergency medicine journals identified in this study were indexed by MEDLINE.

Searching results showed that, for the 14 emergency medicine journals indexed by MEDLINE for which hand searching was completed, hand searching led to the identification of 227 (32.0%) RCTs (primary articles) not found through MEDLINE searching. However, hand searching, although statistically significantly better than MEDLINE searching, failed to identify 118 (16.6%) of the RCTs found by MEDLINE searching, suggesting that hand searching is not a “gold standard” method and that the dual approach, promoted by the Cochrane Collaboration, may be the optimal approach for journals indexed by MEDLINE.

One possible explanation for this last result may be that the volunteer hand searchers did not have the specific knowledge required to identify RCTs and were trained at a distance through receipt of a training package. The number of missed RCTs may have been reduced if we had instituted a minimum scientific knowledge requirement or trained hand searchers in person before commencing the task. Results of hand searching for RCTs by persons recruited and trained by other techniques have not been published.

The results of the comparison of hand searching with MEDLINE searching should be interpreted with reference to the following issues. The comparison was based on the use of a comprehensive search strategy (Figure 2), optimally developed by the Cochrane Collaboration for MEDLINE. The results obtained cannot be generalized to MEDLINE searching by ad hoc search strategies nor to other electronic databases (eg, EMBASE). In addition, since this study was conducted, the Cochrane Collaboration has worked with the National Library of Medicine in the USA to ensure that RCTs not coded as such on MEDLINE are retagged. Quasi-RCTs and possible RCTs are being retagged with the medical subject heading (MeSH) and publication type (PT) “CONTROLLED CLINICAL TRIAL (CCT).” This program has dramatically increased the number of RCTs correctly indexed on MEDLINE.¹⁷, ¹⁸

Complete identification of relevant studies is particularly important for a systematic review. In this study, an additional 21 RCTs (primary articles) were identified in the 4 journals not indexed by MEDLINE. Similarly, 285 conference abstracts reporting the results of RCTs and 692 abstracts reporting the results of RCTs published elsewhere were identified in the 18 journals searched. It is possible that a proportion of these latter 2 categories could have been identified by MEDLINE searching as a result of being published as primary articles in journals indexed by MEDLINE. However, the rate of subsequent publication of conference abstracts as primary articles determined from research in other areas, approximately 50%, indicates that MEDLINE searching would still miss 143 of these trials.⁹ Excluding conference abstracts and abstracts of RCTs published elsewhere, there was still an excess of 248 primary RCTs identified by hand searching that were not identified by comprehensive MEDLINE searching in the 18 journals.

Systematic reviews require that every effort be made to identify all eligible RCTs, so as to avoid any bias that may arise if only “selected” reports are used. It remains to be demonstrated that the additional RCTs found by hand searching would change the results of systematic reviews comprising RCTs identified by comprehensive MEDLINE searching alone. Does selection bias, English-language bias, or publication bias have an impact on the results of systematic reviews? It is usual at the stage of systematically reviewing the evidence to assess its quality. Are better-quality studies reported more clearly and in higher-profile journals, and are they therefore more likely to be identified from comprehensive MEDLINE searching?

All of the RCTs identified in this project are being added to the Cochrane Controlled Trials Register, available in the Cochrane Library, and to the International Register of RCTs coordinated by the Baltimore Cochrane Center. In addition, we are currently undertaking research study to further describe the RCTs identified in this study. This will help answer important questions concerning the prevalence and quality of reporting of RCTs relevant to emergency medicine. It will also make clearer those aspects of emergency medicine that have been evaluated in RCTs and are therefore ripe for systematic review. Such reviews will provide important information on the effects of interventions in emergency medicine and identify those areas that need further primary research. It is hoped that improved identification of RCTs in the emergency medicine literature will facilitate the creation of problem-based Collaborative Review Groups for carrying out systematic reviews.⁸

Finally, improved indexing of RCTs on MEDLINE relies on journal editors’ and authors’ ensuring that future reports of RCTs use the term “randomized controlled trial” in the title or abstract and incorporate an explicit description of how the comparison groups were generated in the methods section, to allow readers to assess the method of randomization or allocation to treatment group, as has been agreed by leading journal editors.¹⁹

Back to Article Outline

Acknowledgements 

We would like to acknowledge the complimentary subscriptions received from 11 journals and the efforts of the 46 volunteers who retrospectively and prospectively hand searched 18 journals. In addition, we thank Iain Chalmers, Kay Dickersin, Chris Silagy, Caroline Goldfrad, Phil Alderson and, more recently, the anonymous journal referees for comments on earlier drafts of this manuscript.

Journals providing complimentary subscriptions: Injury , Burns, Accident and Emergency Nursing , Resuscitation , American Journal of Emergency Medicine , Annals of Emergency Medicine , Journal of Emergency Medicine , Journal of Head Trauma Rehabilitation, Journal of Neurotrauma, Journal of Trauma , and Pediatric Emergency Care .

Contributors to hand searching in emergency medicine: Jerry Balentine, Don Benson, Philip Bossart, MG Cameron, John Chambers, Charmaine Childs, Christine Dearden, Geraint Evans, Dale Gerdes, Mark Goetting, HR Guly, SN Harrop, John Heyworth, Philip Hombrey, Allen Klippel, Paul Lawler, Joel Lexchin, Douglas Lindsey, Rod Little, Bruce Lobitz, Lilian MacDonald, K Mackway-Jones, Peter Mariani, MC McCann, David Morgan, Noelle Murphy, Yoichi Nagayama, Carin Olson, Jason Payne James, Barbara Phillips, Janet Porter, N Ramesh, Laurence Rocke, William Rutherford, Ian Stewart, Lawrence Stock, Gary Vilke, Michele Wagner, Marion Waters, Robert Wears, Arlo Weltge, PB Wilson, Jill Windle, Jonathan Wyatt, David Yates, and Nigel Zoltie.

Back to Article Outline

References 

1. Sackett DL, Haynes RB, Guyatt GH, et al.  Clinical Epidemiology : A Basic Science for Clinical Medicine. Toronto: : Little, Brown; 1991;
   • View In Article
2. Antman EM, Lau J, Kupelnick B, et al.  A comparison of results of meta-analyses of randomized controlled trials and recommendations of clinical experts: Treatment for myocardial infarction. JAMA. 1992;268:240–248
   • View In Article
   • MEDLINE
3. Peto R. Why do we need systematic overviews of randomized controlled trials?. Stat Med. 1987;6:233–240
   • View In Article
   • MEDLINE
   • CrossRef
4. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20:637–648
   • View In Article
   • MEDLINE
   • CrossRef
5. Anonymous . Systematic overview of controlled trials (meta-analyses) helps clarify treatment effects. Drugs Ther Bull. 1992;30:25–27
   • View In Article
   • MEDLINE
6. Chalmers I, Hetherington J, Newdick M, et al.  The Oxford Database of Perinatal Trials: Developing a register of published reports of controlled trials. Control Clin Trials. 1986;7:306–324
   • View In Article
   • MEDLINE
   • CrossRef
7. Chalmers I, Dickersin K, Chalmers TC. Getting to grips with Archie Cochrane’s agenda. [editorial] BMJ. 1992;305:786–788
   • View In Article
   • MEDLINE
   • CrossRef
8. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309:1286–1291
   • View In Article
   • MEDLINE
   • CrossRef
9. Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts: A meta-analysis. JAMA. 1994;272:158–162
   • View In Article
   • MEDLINE
10. Dickersin K, Min YI. NIH clinical trials and publication bias. Online J Curr Clin Trials. 1993; [serial online] doc 50
    • View In Article
11. Anonymous . Cochrane’s legacy. [editorial] Lancet. 1992;340:1131–1132
    • View In Article
    • CrossRef
12. The Cochrane Library. Oxford: : Update Software; 1999; Issue 2
    • View In Article
13. ULRICH’s Plus [database on CD-ROM]. New Providence, NJ: : RR Bower/Reed Reference Electronic Publishing; Winter 1992-1993;
    • View In Article
14. MEDLINE: OVID for Windows 3.0, release version 5.3. New York: : OVID; 1995;
    • View In Article
15. Pocock SJ. Clinical Trials: A Practical Approach. Chichester, NY: : John Wiley & Sons; 1997;
    • View In Article
16. Dickersin K, Larson K. Establishing and maintaining an international register of RCTs. In:  Sackett D,  Oxman A editor. Cochrane Collaboration Handbook. Oxford: : The Cochrane Collaboration; 1995;p. V1–V51
    • View In Article
17. McDonald SJ, Lefebvre C, Clarke MJ. Identifying reports of controlled trials in the BMJ and the Lancet. BMJ. 1996;313:1116–1117
    • View In Article
    • MEDLINE
    • CrossRef
18. Cochrane Collaboration . CENTRAL: The building blocks of Cochrane reviews. Cochrane News Letter. July 1997;1257–1324 No. 11:1. ISSN
    • View In Article
19. Begg C, Cho M, Eastwood S, et al.  Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA. 1996;276:637–639
    • View In Article
    • MEDLINE