Journal Club: The NHAMCS Database, Chart Review Methods, and More on Regression Modeling
Article Outline
Editor's Capsule Summary for Schuur et al1
What is already known on this topic
The National Quality Forum Expert Panel for Emergency Care recently endorsed a quality indicator to measure the performance of pregnancy testing in women aged 14 to 50 years who present to the emergency department (ED) with abdominal pain.
What question this study addressed
The performance of this quality measure was assessed in a national dataset (National Hospital Ambulatory Medical Care Survey [NHAMCS]) and by chart review at 4 academic hospital EDs to see whether the 2 methods produced consistent results.
What this study adds to our knowledge
The nationally representative database (NHAMCS) significantly underestimated the measurement of pregnancy testing compared with the chart review.
How this might change clinical practice
Although these findings will not change clinical practice, they demonstrate the dangers and difficulties of using databases to measure quality indicators.
SEE RELATED ARTICLE, P. 449.
Discussion Points
The National Hospital Ambulatory Medical Care Survey (NHAMCS) is a federally funded survey that includes an annual abstraction of a sample of emergency department (ED) cases. A quick search of Annals of Emergency Medicine database revealed more than 50 articles based on this survey. Because it is so frequently used, this journal club directs some attention to the way this database is assembled and some of its strengths and weaknesses as a research tool.
A. Describe the sampling methods, including what types of institutions are selected, and the 4-stage probability sampling.
B. Describe the methods of data abstraction, who extracts the data, what training is there, what assessment is performed to ensure that data extraction is reliable, and what variables are routinely collected.A. What are the 8 basic elements of chart review? [Hint: there have been at least 3 articles on this topic in Annals of Emergency Medicine and you would be wise to consult one or more of them.]B. Do these standardized chart review methods (the 8 elements) ensure validity? Reliability? What forms of bias and measurement error are most likely to be avoided by meeting chart review requirements? Can you provide examples from this or another study?C. What is reported in the article by Schuur et al1 in the results that suggests that there may be some problems with the reported reliability of the NHAMCS data extraction?A. There are several plausible explanations for these findings. List at least 4 competing hypotheses.B. List the information in the Schuur et al1 article that supports or refutes each of your proposed hypotheses.C. What might you do to determine which of these hypotheses is true?
A. The authors control for sex, age, ethnicity, source of payment (insurance status), visit characteristics (disposition, trainee provider or midlevel provider, and urgency of visit triage code), hospital characteristics (urban, hospital ownership, region of the country), and year of visit. Why were those variables considered?B. The authors analyze all women aged 14 to 50 years in the NHAMCS dataset.1 Can you think of reasons to exclude some women from the analysis? What information not available in NHAMCS might better inform your decision of who to include in the analysis?C. Consider the question “which women need a pregnancy test in the ED and are they getting one” and assume you are not limited to the NHAMCS data set. Develop a theoretical model of the question. What variables would you need in a data set to confidently answer this question?A. List some of those reasons.B. On a list of all the quality issues in the delivery of emergency medical care ranked from most to least important, where would you place the appropriateness of pregnancy testing?C. Regardless of where you placed it on the list, why do you think organizations such as the National Quality Forum are advocating that this measure be incorporated?D. What does the article by Schuur et al1 suggest about the reasonableness of this approach?
Section editors: Tyler W. Barrett, MD; David L. Schriger, MD, MPH
Editor's Note: You are reading the 15th installment of Annals of Emergency Medicine Journal Club. This bimonthly feature seeks to improve the critical appraisal skills of emergency physicians and other interested readers through a guided critique of actual Annals of Emergency Medicine articles. Each Journal Club will pose questions that encourage readers—be they clinicians, academics, residents, or medical students—to critically appraise the literature. During a 2- to 3-year cycle, we plan to ask questions that cover the main topics in research methodology and critical appraisal of the literature. To do this, we will select articles that use a variety of study designs and analytic techniques. These may or may not be the most clinically important articles in a specific issue, but they are articles that serve the mission of covering the clinical epidemiology curriculum. Journal Club entries are published in 2 phases. In the first phase, a list of questions about the article is published in the issue in which the article appears. Questions are rated “novice,” () “intermediate,” () and “advanced” () so that individuals planning a journal club can assign the right question to the right student. The answers to this journal club will be published in the October 2010 issue. US residency directors will have immediate access to the answers through the Council of Emergency Medicine Residency Directors Share Point Web site. International residency directors can gain access to the questions by going to http://www.emergencymedicine.ucla.edu/annalsjc/ and following the directions. Thus, if a program conducts its journal club within 5 months of the publication of the questions, no one will have access to the published answers except the residency director. The purpose of delaying the publication of the answers is to promote discussion and critical review of the literature by residents and medical students and discourage regurgitation of the published answers. It is our hope that the Journal Club will broaden Annals of Emergency Medicine's appeal to residents and medical students. We are interested in receiving feedback about this feature. Please e-mail journalclub@acep.org with your comments.
Reference
PII: S0196-0644(10)00266-0
doi:10.1016/j.annemergmed.2010.03.021
© 2010 Published by Elsevier Inc.
Refers to article:
-
Pregnancy Testing in Women of Reproductive Age in US Emergency Departments, 2002 to 2006: Assessment of a National Quality Measure
, 23 November 2009
