Is the Golden Hour Tarnished? Registries and Multivariable Regression:

Article Outline

• Discussion Points
• Answer 1
• Answer 2
• Answer 3
• Answer 4
• References
• Copyright

Back to Article Outline

Discussion Points 

2.Medical helicopters have assisted with the rapid transport of critically ill trauma patients since the 1970s. However, in February 2009, the National Transportation Safety Board (NTSB) held a 4-day hearing to investigate the concerning increase in helicopter-ambulance crashes.A. Conduct a brief review of the medical literature and lay press, tracing the introduction of medical helicopter transport to the current debates over patient and flight personnel safety.B. According to your review for question 2.A and the conclusions from the Newgard et al article, what is your opinion on the proper role for medical helicopter transport in trauma resuscitations? Should medical helicopter transport be used when anticipated ground transport times are greater than 60 minutes, 90 minutes, 120 minutes, or none of the above? What additional patient or emergency medical services (EMS) system factors must be considered when determining the best transport mode for a critically ill trauma patient?C. The Federal Emergency Treatment and Active Labor Act (EMTALA) requires that the transferring physician “ensure that the transfer of an unstabilized individual is effected through qualified personnel and transportation equipment, including the use of medically appropriate life support measures.” Discuss how EMTALA might affect the decisions on transporting a trauma patient to the regional trauma center. What other measures does EMTALA require of hospitals and physicians when transferring or accepting patients?

3.This study was a secondary analysis of a prospective cohort registry of adult trauma patients transported by nearly 150 EMS agencies in North America. The authors analyzed the data by using multivariable regression models to test the association between EMS transport intervals and patient mortality. A. One advantage of a registry study is that one can enroll large numbers of patients, thereby increasing precision. Unfortunately, registries are subject to certain biases that can result in erroneous conclusions. Describe the advantages and limitations of this study design. What are some potential biases of registries? What types of quality assurance measures did the investigators incorporate into the study's operating procedures to minimize these limitations?B. Explain in layman's terms what multivariable regression is. Why is multivariable regression especially important to biomedical research? Contrast a fully explicated model (one that includes all interaction terms) versus one that has no interaction terms. How do logistic regression, linear regression, and Cox proportional hazards survival regression analyses differ in their outcome variables and output? What are some of the assumptions and limitations associated with each type of regression analysis?C. In the “Limitations” section, the authors mention that they used a fixed-effect model to account for the “likely variation in field care, hospital care, and injury characteristics between sites, EMS agencies, and hospitals.” Define what a fixed-effect model is and how it differs from a random-effects model.

4.In your opinion, what are the most important conclusions from this article? How might these conclusions affect EMS transport of trauma patients in urban and rural settings? How might the limitations mentioned by the authors affect your medical control recommendations for patient transport by medical helicopter versus ground transport with or without lights and sirens?

Back to Article Outline

Answer 1 

Q1. Missing data can bias a study's conclusions. Large multicenter observational studies are likely to encounter more missing data than a small, prospective, single-center trial.

Q1.a Explain how missing data might bias a study's conclusions. Include in your answer a discussion about how missing data on patients' initial vital signs might have altered this study's conclusion.

Missing data can occur at random (think of a monkey randomly erasing values from an otherwise complete spreadsheet), in a nonrandom manner (persons who are very overweight refuse to report their weight on questionnaires, so their “weight” value is missing in the data set), or in a pattern that blends these extremes. The effects of missing data on validity and precision are quite different for the 2 extremes. Randomly missing data, by decreasing the effective sample size, may decrease a study's precision but seldom affect the study's validity. In contrast, data that are missing in a nonrandom way can create a special type of selection bias (because only cases for which there are data are “selected” for analysis). If “selected” cases are dissimilar to the underlying population, bias can result (see the answer to question 1b for a more complete consideration of “patterns of missingness”).

For all of these reasons, investigators should do everything possible to avoid missing data. In a reasonably sized study for which the outcome measure is not rare, a few missing data points seldom make interpretation of the study difficult, but when the amount of missing data increases, it is difficult to know whether study results are valid. Consider a study of 200 persons for whom there are 6 positive outcomes and 12 missing outcomes. If the missing outcomes truly occurred at random, then the only consequence is that the study has 188 persons instead of 200, with a consequent small widening of the confidence interval (CI) and a slight imprecision caused by the quantized nature of outcomes (we would expect 0.3 extra outcomes in the 12 missing subjects, but there can be only 0, 1, or 2, so we are likely to be at 6/200 [3%] or 7/200 [3.5%] but not 6.3/200 [3.3%] [the equivalent of 6/188]), and there will be a slight difference in the estimate of the probability of a positive outcome. If, however, we are not certain the missing cases occurred at random, we cannot be as certain about the prevalence of outcomes. There could be no events in the 12 missing cases so that 6 of 200 (3%) would have been the result, or the 12 missing cases could all have had an event so that the event proportion is 18 of 200 (9%).

Depending on the purpose of the study, the difference between 3% and 9% could be critical or unimportant. That is why there can be no hard and fast rules about the amount of missing data that can be tolerated. In some studies, missing data will have little influence on the conclusion; in others, it can render the study uninterpretable. From this discussion, it should be apparent that when planning the size of a study, investigators should consider both random effects (power) and the amount of anticipated missing data.

A well-designed study, regardless of its design, acknowledges the potential for missing data and incorporates measures to limit incomplete information. Studies with numerous missing data points should clearly report the number of missing and nonmissing cases, the methods for handling missing information, and the reasons for “missingness.”¹ The article should also include a table comparing the baseline characteristics of the subjects with complete and incomplete information.¹

When reading a study with missing data, the reader must decide whether the missing data likely affect the study's results and conclusions. Readers should be skeptical of a study when the investigation or analysis does not account for potential confounding caused by missing information. Missing data can cause the under- or overestimation of the parameter estimates, thus altering the variables' predictive performance and study conclusions.¹ Might there be some systematic difference between the subjects with complete information and those with missing data? One can discover this difference only if the authors disclose it or present a table that compares the 2 groups. Missing information on child immunization status is unlikely to bias a study measuring analgesia in long bone fractures but would be critical to an investigation in the treatment of febrile infants.

The reason data are missing is often as important as the missing information. Consider the following hypothetical study in which the investigators reviewed the emergency department (ED) ECGs of 1,000 patients with chest pain. The study's objective was to determine whether ST-segment elevation was associated with acute myocardial infarction. When the investigators reviewed the medical records, they found that 300 ECGs were missing. The investigators excluded those patients and analyzed the remaining 700, finding no apparent association. While preparing their article, the authors examined the baseline characteristics of the excluded patients; it became clear that a higher percentage of the patients with missing ECGs had acute myocardial infarction and emergency cardiac catheterization. Further investigation discovered that the ECG was often taken from the ED record by the catheterization team. A repeated analysis including these patients showed a dramatic association between ST-segment elevation and acute myocardial infarction. This outlandish example depicts the problem with excluding cases with missing data from the analysis.

In the Newgard et al article, 620 of the 4,276 patients (14.5%) had some missing, erroneous, or incomplete data, including 152 (3.6%) with missing survival information.² The authors found no association between emergency medical services (EMS) intervals and inhospital mortality. How likely is it that missing initial vitals signs have affected this conclusion? Given the large size of this cohort and the well-done statistical analyses that include multiple sensitivity analyses, we can be fairly confident that missing data do not bias results in an important way. However, if this were a smaller study (n=100) and the initial vital signs were missing for 10 of the patients, might it be sensible to think that EMS personnel might drive faster (ie, shorter transport interval) when transporting patients with very unstable vital signs? Or might the EMS crew spend more time at the scene with unstable patients? Might EMS drive more cautiously when transporting a blunt trauma arrest patient with no probability of survival? Might EMS treatment and transport policies and procedures differ between agencies, resulting in additional practice variance? All of these potential scenarios might bias the association between mortality and intervals. The best remedy for the missing data bias is to minimize the potential for incomplete data when designing a study's data collection mechanism (see question 1d).

Q1.b Describe what the term “pattern of missingness” means. Why is it important to understand whether data are missing at random or not?

The term “pattern of missingness” refers to whether data are missing randomly or whether there is a specific pattern or mechanism associated with missing data. For example, and as noted above, for surveys that ask about an individual's weight, the probability of missing data may be higher for those who are heavier. In this example, the common strategy of assigning the mean or median weight to those individuals with missing data would underestimate the mean (or median) weight of the population. In contrast, a question about how much these individuals weighed at birth is less likely to have an association between missing values and birth weight, and a strategy of replacing missing values with the mean weight might produce an unbiased estimate.

There are multiple types of missing data mechanisms: missing completely at random (MCAR), missing not at random, and missing at random (MAR).³, ⁴ MCAR mechanisms include administrative errors or accidents such as when the laboratory loses or spills a blood sample, resulting in a missing diagnostic study result. MCAR assumes that the individuals with missing data are representative of the population with complete information and that the missing values are not associated with any other patient characteristic (hence the “C” for completely).³

The missing not at random mechanism assumes that missing data are related either directly to the information being measured (eg, weight example) or to some other unmeasured patient characteristic.³, ⁴ This type of missing data results in the loss of valuable information that is not well corrected by the basic methods such as case deletion or assigning the sample mean.

The most common classification of missing data is MAR, in which the likelihood that data are missing depends on other observed patient characteristics. Allison⁵ summarized well that “missing data on a variable, say Y, are held to be random if, after controlling for other variables, the value of Y cannot predict the location of the missing scores.” For example, a study that enrolled all adult patients with chest pain but performed cardiac enzyme tests only on individuals older than 35 years would have more missing data for younger patients. Consequently, those with missing cardiac enzyme values are not representative of the sample, they are younger, even though the missing values are distributed randomly among younger members of the sample. We will discuss in Answer 1c how to best address MAR missing data.

Q.1c There are multiple methods for handling missing data. Discuss the advantages and disadvantages of some of the more common methods including excluding cases with missing data, treating binary categorical as yes/no/missing rather than yes/no, carrying forward the last known value, assigning the sample mean for missing data, and performing multiple imputation.

The first issue is to quantify the amount of missing data and characterize the subjects with missing information.⁶ Is there an obvious pattern to the missing data? Do certain variables seem to be missing in the same patients? Does one study site have more missing information than the other sites? Such a finding would raise concern about that center's data abstraction quality and overall data integrity. Once the data have been reviewed for patterns of missingness, the investigators need to decide how to handle their missing data. There is no single best approach. The circumstances will dictate which approach is preferred, and it is often wise to compare results achieved with a series of best alternatives to see whether results are stable regardless of the method used. Such stability can increase one's confidence in the validity of the methods, whereas instability suggests that caution is required.

The easiest approach is to exclude cases with missing information. There are 2 primary methods: listwise deletion and pairwise deletion. Listwise deletion deletes from the database any observations that have missing data on any of the variables of interest.⁵ Many standard regression protocols do just this; they include only cases that have data for every variable in the regression. Pairwise deletion can be used with linear models and calculates each of the summary statistics, using all the cases that are available. Both of these methods are valid only when the data are MCAR and should be considered only when the percentage of missing variables is small and the overall sample size is very large.⁶ Deletion of records with MAR or missing not at random mechanisms results in badly biased parameter estimates when the pattern of missing information for a certain variable, X, is unexplained by the group of nonmissing variables, Y.⁶

The addition of an “unknown” category to binary categorical variables (eg, pregnant? “yes/no/unknown” instead of “yes/no”) will decrease the amount of missing data and is a useful preventive measure against missing data. Investigators who use a strategy like this can analyze a binary variable as if it has 3 values, “yes/no/missing,” and can compare results among the 3 groups. This technique can be useful for asking questions such as “are persons who left the question blank more similar to those who said yes or those who said no.” This method can produce biased estimates, however, because it treats an incomplete data set as if it were complete.⁵, ⁶

Another method for dealing with missing data on serial measurements is to carry the last known value forward. For example, the discharge systolic blood pressure is missing from the record but the triage blood pressure is available. One might be tempted to substitute the triage value for the missing discharge value, which might be acceptable if the patient presented for a wound check and received no ED treatment. But what if the patient presented with a long bone fracture or hypertensive emergency? Would the triage blood pressure be an accurate estimate of the blood pressure after treatment for the patient's condition before ED discharge? Unlikely. This method is fraught with the possibility for confounding because the reason for the missing data may be a function of both the missing blood pressure and the initial blood pressure. Even when the value brought forward is a reasonable proxy for the missing value, this strategy will underestimate the variance (because the N is artificially inflated beyond what is actually known) and will produce overly narrow CIs. ⁶

An alternative approach is to substitute the missing data with the mean or median value from the nonmissing values. This method can suffice if the amount of missing data is small and the mechanism is MCAR.⁶ As shown in earlier sections, when the number of cases with missing data is substantial or there is a pattern to why variable information is missing, this mechanism can create bias. As with last known value carried forward, substituting the mean or median value will underestimate the variance and produce CIs that are too narrow.

Multiple imputation is the most robust method for handling missing data. Burton and Altman¹ write that multiple imputation “results in valid statistical inferences that properly reflect uncertainty due to missing values.”⁷ Imputation techniques are based on the assumption that any subject from a study population can be replaced with a randomly selected subject from the same population.³ Multiple imputation incorporates estimates from multiple variables to produce a value for the missing information that appropriately reflects the uncertainty of the estimated distribution of the variable. Multiple imputation is essentially the only mechanism for dealing with missing information caused by MAR. The previously discussed methods will all give biased estimates if the data are missing because of a patient characteristic that is observed (MAR).³ Imputation uses the nonmissing data on one or more variables to calculate an educated estimate of the missing data. For example, in the above weight example, imputation might incorporate patient sex, age, ethnicity, height, activity level, blood pressure, cholesterol level, and occupation into a calculation to estimate the distribution for weight in that study sample. Then a random draw from that estimated distribution is selected as the replacement value for the missing data. This single imputation method produces a complete data set that is then analyzed as if there were never any missing information. This results in an underestimation of the parameter's standard errors, overly small P values, and exaggerated study precision.³ Multiple imputation corrects these problems by creating more conservative standard errors that account for the imprecision of estimating the distributions for the variables with missing data.³ The details of the methodology are beyond the scope of this Journal Club article. Readers interested in greater detail about multiple imputation techniques are referred to other articles.³, ⁸, ⁹, ¹⁰

Q1.d The most successful investigators design studies and data collection forms to minimize the potential for missing data. If you were planning a large registry study, what design features might you incorporate to decrease the amount of missing data?

It should be obvious that the best way to avoid missing data is to create data collection and entry mechanisms that (1) provide the universe of possible choices so that respondents are able to find an appropriate choice, and (2) are designed to identify and correct missing data while the respondent is still participating in the process. Fatigue is another determinant of the amount of missing data, and although an investigator may be tempted to examine hundreds of potential variables, data quality will be better when a more selective approach is used.

Data collection instruments should be organized in a format that matches the logical sequence of data collection. When there are skips in the data collection process because of questions that do not apply to individuals, care should be taken to develop mechanisms to ensure that the skips are done properly.

We cannot overemphasize the importance of piloting instruments on actual data, using the persons who will be using the instruments. This is the only way to discover and remedy glitches in the process.

Although there may be a few situations in which paper is preferred for initial data collection, the availability of highly secure electronic data submission options, handheld devices for data entry, and smart databases that can identify and correct omissions and errors in real time makes centralized computerized data entry the default standard. Such methods eliminate the possibility of lost forms and transcription errors.

For this study, Newgard et al² developed standardized data collection forms, used a central processing center for data processing and analysis, and performed multiple quality assurance processes throughout the study period.

Back to Article Outline

Answer 2 

Q2. Medical helicopters have assisted with the rapid transport of critically ill trauma patients since the 1970s. However, in February 2009, the National Transportation Safety Board (NTSB) held a 4-day hearing to investigate the concerning increase in helicopter-ambulance crashes.

Q2.a Conduct a brief review of the medical literature and lay press, tracing the introduction of medical helicopter transport to the current debates over patient and flight personnel safety.

Air transport of patients dates back to the 1800s, when hot air balloons were used to transport soldiers from Paris, France.¹¹ Rotary wing aircraft such as helicopters were first used to evacuate soldiers injured on the field of battle in the 1950s during the Korean War. Because of the difficult landscape in Korea, the Third Air Rescue Squadron received requests to evacuate injured soldiers from locations inaccessible to ground ambulances. The rapid air transport resulted in markedly decreased mortality for major injuries compared with that in World War II.¹² The role of helicopters greatly expanded during the Vietnam conflict. Helicopters ensured that no injured solider was more than a half-hour from a medical facility capable of performing lifesaving treatments.¹³ The positive combat aeromedical experience led to helicopter ambulance inclusion in comparable civilian emergency health programs.¹¹ Proponents believed that helicopters provided rapid patient transport to an appropriate trauma care facility and delivered highly trained personnel to the scene of an accident for early resuscitation and stabilization.¹⁴

The first helicopter service devoted to out-of-hospital resuscitation of the critically injured patient was established in Colorado in 1972. The Colorado system successfully transported more than 2,000 patients during a 27-month period, without incident.¹⁵ During the next several decades, many other hospitals began to establish aeromedical transportation services. Helicopter EMS evolved into a “second responder” that assumed care once local EMS triage decided that injured patients would benefit from rapid evacuation to a tertiary care facility. The roles of air ambulance services continue to evolve and now include interhospital transport of neonates, patients with high-risk pregnancies, and those with acute coronary syndromes.¹⁶

During the last decade, helicopter transport greatly increased and the private sector became more involved in out-of-hospital aeromedical care.¹⁷ Questions arose concerning the appropriate use, and possibly overuse, of this method of patient transport, given the inherent risks to the patient and the providers. In 1999, the American College of Emergency Physicians recognized that helicopter air evacuation is a crucial component in a tiered response system for the delivery of rapid care and transport to an appropriate health care facility. The College stated that dispatch of the air ambulance should be under the control of the appropriate EMS regional entity and should be recognized as a regional resource that is available to all, regardless of the ability to pay.¹⁸ More private helicopter transport agencies have entered the market since the federal reimbursement for helicopter transportation became standardized.¹⁹

The expanded use of helicopter medical transports has resulted in a concomitant increase in morbidity and mortality from accidents. Between 1980 and 1998, there were 264 total helicopter transport–related fatalities, including crew members and transported patients (average of 14.6 fatalities/year).¹⁷ In 2008, helicopter ambulances experienced its worst year on record, with 15 accidents and 29 fatalities.¹⁷ This dramatic increase in helicopter-transport fatalities prompted the National Transportation Safety Board to hold 3 days of hearings in February 2009. The board subsequently published recommendations to improve the safety of helicopter transportation. These recommendations included improved pilot training, additional safety instrumentation and technology, national usage guidelines, and annual reporting of the number of hours flown and patient transport volume. The board also recommended that reimbursement be tied to the helicopter agency's compliance with these recommended safety measures.¹⁷

Q2.b According to your review for question 2.a and the conclusions from the Newgard et al article, what is your opinion on the proper role for medical helicopter transport in trauma resuscitations? Should medical helicopter transport be used when anticipated ground transport times are greater than 60 minutes, 90 minutes, 120 minutes, or none of the above? What additional patient or emergency medical services (EMS) system factors must be considered when determining the best transport mode for a critically ill trauma patient?

The article by Newgard et al² did not find evidence to support the contention that shorter out-of-hospital times improve survival among injured adults with field-based physiologic abnormality. The article further reports that these findings persisted across subgroups, including EMS first-responder expertise level, mode of transport, country, age, injury type, and severity of physiologic derangement. This conclusion is consistent with that of other literature addressing helicopter transport in the out-of-hospital setting. Talving et al²⁰ evaluated the trauma registry of Los Angeles County + University of Southern California Medical Center to identify all injured patients evacuated by emergency medical services during a 20-year period. They divided the study population into those who were airlifted and those who were transported by ground EMS with transportation times exceeding 30 minutes. This study concluded that in this trauma system, helicopter transportation of injured patients did not improve overall adjusted survival after injury. There was, however, a potential benefit for the severely injured subgroup of patients because of shorter out-of-hospital times. Ringburg et al²¹ addressed this concept of “on-scene time” and its association with mortality. Trauma patients transferred to a Level I trauma center during a 2-year period were analyzed, comparing on-scene time of those treated with nurse-staffed EMS and those treated with a combination of physician-staffed helicopter EMS and ground EMS crews. The study concluded that combined EMS/helicopter EMS assistance at an injury scene was associated with longer on-scene times. The analysis found no improvement in mortality with longer on-scene intervals.

Biewener et al²² reported that primary helicopter transport to a Level I trauma center significantly reduced mortality, although this was likely attributed to the out-of-hospital therapy provided and the direct transfer to a trauma center. Studies have also reported that helicopter EMS decreases mortality from scene transport despite increased transport times and is associated with a 6-fold increase in return of spontaneous circulation in traumatic arrest patients compared with ground transport.²³, ²⁴ Return of spontaneous circulation in the field, however, usually does not result in better patient outcomes in blunt trauma arrest, regardless of mode of transportation.

The evidence remains unclear whether there is a universal standard about when helicopter transport absolutely benefits trauma patients according to intervals to definitive care. The data from the Newgard et al² study support the contention of previous research that helicopter transport may not be beneficial in large, metropolitan areas with close proximity to capable hospital facilities.²⁵ There are inherent risks to the air medical crew and pilots with each flight. Patients in more rural settings with delayed transfer to definitive trauma centers may benefit from air transport, although their ultimate outcome may depend on the skill set of the providers involved.²³ For example, an extended-care flight nurse may be trained and credentialed to perform lifesaving procedures during transport, such as rapid sequence intubation, chest tube thoracostomy, and blood product administration. Rural volunteer EMS are unlikely to be trained and facile in these advanced resuscitation procedures.

Q2.c The Federal Emergency Treatment and Active Labor Act (EMTALA) requires that the transferring physician “ensure that the transfer of an unstabilized individual is effected through qualified personnel and transportation equipment, including the use of medically appropriate life support measures.” Discuss how EMTALA might affect the decisions on transporting a trauma patient to the regional trauma center. What other measures does EMTALA require of hospitals and physicians when transferring or accepting patients?

The United States Congress passed the EMTALA legislation in 1986, with the purpose of preventing patient injury arising from economically based treatment delays. EMTALA guarantees access to emergency services without regard to economic status.²⁶, ²⁷ This law was intended to prevent the reported practice known as “patient dumping.” Violation of EMTALA can result in substantial financial penalties to a hospital or individual physician involved, even suspension from the Medicare and state Medicaid programs. Physicians and hospitals that intentionally violate EMTALA are subject to fines up to $50,000 that are not covered by most malpractice insurers. The Centers for Medicare & Medicaid Services, in conjunction with the Department of Health and Human Services, enforces EMTALA compliance.²⁶ EMTALA requires that every patient presenting to a hospital ED have a medical screening examination to determine whether an emergency condition exists. If there is no acute emergency condition, EMTALA does not apply. If a patient indeed has an emergency medical condition, he or she must be stabilized by on-call specialty physicians as needed. However, if the hospital is unable to provide the necessary treatment to improve a critically ill patient, arrangements must be made for transfer to a higher level of care. A hospital with the needed physicians and interventions cannot refuse a patient regardless of ability to pay.²⁶

EMTALA frequently is enacted in the treatment and transfer of patients with severe multisystem trauma. Literature supports the benefit of transfer of these patients to a regional trauma center. McConnell et al²⁸ concluded that patients with severe head injuries who were transferred to a Level I trauma center had an absolute mortality risk reduction of 10.1% compared with those transferred to a Level II center. These authors also performed a retrospective cohort analysis of all consecutive injured children and adults meeting state trauma criteria and presenting to one of 42 nontertiary hospital EDs from 1998 to 2003. Adjusting for the propensity to be transferred, ED transfer to a tertiary care hospital was associated with a reduction in mortality (odds ratio [OR] 0.67; 95% CI 0.48 to 0.94) that was strongest among patients transferred to a Level I trauma center.²⁵ Cudnik et al²⁹ performed retrospective studies of all patients with trauma who were older than 15 years, met Ohio trauma criteria, and were transported directly from the scene to a Level I or II hospital between January 2003 and December 2006. Of the more than 18,000 patients included, patients taken to a Level I trauma center had more severe injuries, more penetrating injuries, and more complications, yet similar unadjusted mortality compared with patients taken to Level II centers. Adjusted analysis found that patients taken to a Level I hospital had improved survival compared with those taken to Level II centers (OR 0.75; 95% CI 0.56 to 0.98). The literature supports that severely injured patients have improved survival if treated initially or transferred to a Level I trauma center. Many trauma centers fear that the poor payer mix may cause severe financial difficulties in treating certain patients unable to pay the large bills associated with trauma admissions. A retrospective case-control national database study evaluated the possible risk factors for hospital transfer in a population unlikely to require transfer to a Level I trauma center for medical reasons. The ORs adjusted for all risk factors indicated that transfer rates were higher for male patients, children, blacks, evening or night transfers, and patients with Medicaid.³⁰ Spain et al³¹ studied all transfer calls to a dedicated Level I trauma center during a 2-year period. Calls were reviewed for age, surgical service requested, and outcome of request. The trauma registry was queried to compare Injury Severity Scale, hospital stay, operations, mortality, and payer status for transfer and primary catchment patients. Contrary to the author's assumptions, EMTALA patients had an identical payer mix and similar operative need compared with the primary catchment patients. The article concludes that these data suggest that the primary motivations for transfer are specialty availability and complexity of care, rather than financial concerns.

Back to Article Outline

Answer 3 

Q3. This study was a secondary analysis of a prospective cohort registry of adult trauma patients transported by nearly 150 EMS agencies in North America. The authors analyzed the data by using multivariable regression models to test the association between EMS transport intervals and patient mortality.

Q3.a One advantage of a registry study is that one can enroll large numbers of patients, thereby increasing precision. Unfortunately, registries are subject to certain biases that can result in erroneous conclusions. Describe the advantages and limitations of this study design. What are some potential biases of registries? What types of quality assurance measures did the investigators incorporate into the study's operating procedures to minimize these limitations?

Advocates for registry observational studies claim that these studies provide a more accurate portrayal of true clinical practice and treatment effectiveness than do clinical trials.³² Clinical trials are criticized for their lack of external validity because of the unusual treatment environment that does not mirror typical clinical practice. The Hawthorne effect and strict adherence to study protocols remove clinical practice variation that commonly occurs outside of the research realm. Registry studies are intended to improve external validity by capturing vast amounts of in situ data on large numbers of patients from various populations, hospital types, and geographic regions.³³ The massive size of many registries provides sufficient power to answer multiple study questions simultaneously. Registry studies also provide an opportunity to study rare adverse events, treatment patterns, health care utilization, clinical course of the disease, and long-term outcomes.

Despite these advantages, registry studies are not without limitations. Cooper³² discussed the limitations of registry databases in a January 2006 Annals editorial. Convenience samples and selective enrollment by site investigators, who may or may not be blinded to the study objective, create a study design at risk for selection bias. Large multicenter registries must have explicit inclusion and exclusion criteria to ensure that the patients of interest are being studied. For example, a large multicenter heart failure registry must clearly define a priori what constitutes a diagnosis of acute heart failure. If ambiguity exists, patients with heart failure symptoms cased by other diseases (eg, chronic obstructive pulmonary disease, renal failure) will likely be included and potentially confound the analyses. Registry study supervisors often compensate site investigators for enrolling patients with an additional payment if the patient completes a follow-up telephone conversation. This type of compensation structure might encourage site investigators to bend the rules and enroll patients who do not quite match inclusion criteria or to avoid enrolling less compliant patients who might not complete the follow-up. These incentives can introduces selection bias and potentially spectrum bias if follow-up might be affected by the patient's illness severity. Convenience sampling and other forms of selection bias prevent accurate estimation of population characteristics and outcome.³²

Medical record review is another frequent component of registry studies and is prone to reporting bias, interpretation bias, erroneous data entry, missing records, and poor interrater agreement.³⁴ Clinician and nursing documentation varies among hospitals, resulting in differences in the reporting and interpretation of the study variables and outcomes of interest. Different hospital laboratories might report laboratory values with different units of measurement that could disrupt statistical analyses if only the numeric value was entered for each test. International physicians with limited access to advanced radiographic imaging likely perform more thorough physical examinations than their US counterparts, which could result in poor agreement on association of physical examination findings and certain outcomes. Erroneous or incomplete data entry threatens the internal validity of the registry. We will discuss proper chart review methodology in a future Journal Club article. Readers interested in learning more about the topic are directed to the superb article by Gilbert et al.³⁴

We commend Newgard et al² for a well-designed registry study. The study enrolled a well-defined cohort of consecutive injured adults (aged ≥15 years) requiring 911 activation at each of the study sites between December 1, 2005, and March 31, 2007. The inclusion criteria were clearly defined and included “standard field trauma triage guidelines that have previously demonstrated high specificity for serious injury and need for specialized trauma resources” to minimize enrollment ambiguity and selection bias.³⁵ The investigators intentionally omitted hospital admission and Injury Severity Scores from the inclusion criteria, given their potential to result in selection bias. Although there is the potential for missed cases, this strategy is more robust than a convenience sample design. The study variables had clinically sensible precise definitions agreed on by a working group of investigators at all 11 sites before study initiation.³⁵ The data elements were restricted to the most essential variables and outcomes to optimize data quality. Standardized data forms and operation manuals were used and the analysis was performed at a central processing site. The Epistry incorporated 2 types of data entry: Web based and batch uploading. Both systems had data entry error and range checks to maintain data integrity. Intervals were recorded as continuous rather than categorical variables. The importance of this distinction has been discussed in a previous Journal Club article.³⁶ The study included multiple quality assurance processes, including data collection training, site visits to monitor data quality and data capture processes, and data information range and consistency checks.

Q3.b Explain in layman's terms what multivariable regression is. Why is multivariable regression especially important to biomedical research? Contrast a fully explicated model (one that includes all interaction terms) versus one that has no interaction terms. How do logistic regression, linear regression, and Cox proportional hazards survival regression analyses differ in their outcome variables and output? What are some of the assumptions and limitations associated with each type of regression analysis?

Before we contemplate multivariable regression in layman's terms, let us cut our teeth on univariate regression concepts. Regression is a mathematical process that examines to what extent knowledge about one characteristic improves speculation about the value of a different characteristic. To illustrate these principles, we use a “guess a person's weight” example. Imagine that you are informed that we have obtained accurate weights on each individual in a 1,000-person population. You are told that you are going to guess the weights of randomly selected persons from this population and that your goal is to minimize your total deviation from their actual weights. You are given incentive to do this by a very large cash reward for the person who has the lowest total squared deviation from the true weights.

At the beginning, the only information you are provided is the mean weight of the entire population (160 lbs), the mean weight of the men in the population (200 lbs), and the mean weight of the women in the population (120 lbs). It should be obvious that your best strategy is to guess the overall mean (160 lbs) if you are told nothing about the person whose weight you are guessing. But what if you were provided some information about the individual? Would that information cause you to change your guess to a value that would increase the likelihood of being closer to the true value? For example, if you were told that the person was a woman, you might offer the mean weight for women (120 lbs) as your guess, rather than the overall average (160 lbs). Over a series of guesses, this strategy would lead to less deviation from the actual values than guessing the overall mean.

This example can be extended to continuous information such as height. Although we illustrate this concept with a linear regression, in all forms of regression we assume the nature of the relationship among the variables and we could just as well have assumed a fractional polynomial relationship instead of a linear relationship.³⁷ We start with a scatterplot of height (x axis) versus weight (y axis) and use least-squares regression techniques to draw a line that minimizes the overall squared deviations from the data.^⁎ We can then ask whether the slope of this line differs from 0 in an important way. If the slope is close to 0, it means that knowledge of height tells us little about how we should modify our guess about weight (Figure 1A). If, on the other hand, the slope is far from zero, it makes sense that one would modify the guess of weight according to the weight=m*height+b equation because that would decrease the overall error of one's guesses compared with simply guessing the mean weight. This is illustrated in Figure 2.

• 
  • View Large Image
  • Download to PowerPoint

• Figure 1. 

  Linear regression of weight by height. A, Knowledge of length provides no information about the weight. B, There is a relationship between the 2 variables, and knowledge of height can improve predictions about weight.

• 
  • View Large Image
  • Download to PowerPoint

• Figure 2. 

  The 4 panels show how more complex models can decrease the overall error between predictions and reality. The red bars indicate the error for each measurement, and the sum of the length of these bars is signified by ∑ |error|. ∑ error² is calculated as the sum of the square of each of these red bars. A, The example is equivalent to guessing the group mean for each and every subject. B, The sex-specific mean is used. C, Regression to account for the value of an independent variable (x axis), but ignoring the subject's sex. D, A model that accounts for both the sex and the value of the x axis variable of each subject.

So if we knew that we were dealing with a situation akin to that illustrated in Figure 1B and we were told the person's height, we would use the regression equation to predict the person's weight. Thus, regression can be seen as a means of improving our predictions about the value of a variable, using our knowledge of the status of other variables. Up to this point, we have considered 2 univariate linear regressions, one using a categorical-binary input variable (sex) and one using a continuous variable (height). Before we consider multivariate regression, let us differentiate linear regression from logistic regression and proportional-hazards regression. The main difference between these is the range of the output (dependent) variable. With linear regression, the mathematical expectation is that the dependent variable can take on any continuous value between -∞ and ∞, though in practice, biology usually dictates that the range be considerably narrower (eg, adult weight from 50 to 1,000 lbs). Contrast this with a binary outcome measure such as whether a person weighs more than a Maytag Series 7 clothes dryer (139 lbs), for which only 2 values (eg, 0 for no and 1 for yes) are permitted. We could use some type of quantum function that used input variables to produce output of exactly 0 or 1 (predictions of weight less or more than the dryer), or we might choose a mathematical function that produced values between 0 and 1 so that we can interpret these values as an indicator of the person's likely status. The logistic function does that. It takes the general form:

When height is infinite, the second part of the main denominator resolves to 0 and the probability that the person weighs more than the Maytag is 1. When height is 0, the probability resolves to 1/2, and when height is extremely negative the denominator resolves to ∞, so the probability is 0 (note than in a real application, one would replace “height” with “height–(mean height)” so that the input values range from negative to positive and the probabilities will vary from close to zero to close to 1). To summarize, logistic models are used when the outcome is binary and logistic regression naturally produces ORs as its output.

Linear models are additive. From the equation weight=m×height+b, we see that extra units of height add m×Δ height to the estimate of weight. In contrast, Cox proportional hazards models are multiplicative. Additional units of height might each double or halve our weight estimate (this might seem strange, but it makes more sense if the input was pack-years of smoking and the outcome were risk of lung cancer). In any case, regression of this kind uses mathematics of the general form weight=e^(m×height) so that the addition of one unit to height multiplies the weight estimate by m (ie, if m=2, the weight doubles for each additional unit of height). To summarize, proportional hazards models are used when the outcome is believed to relate to the inputs in a multiplicative (proportional) way.

We now extend the univariate linear model to a multivariable situation. Returning to our “guess the weight game,” what if you were told the subject's sex and height? A common approach would be to use a regression equation such as:

where sex=0 for females and 1 for males and m_s and m_h are the coefficients for the 2 input variables. Let us consider what this equation implies. For female subjects, the equation becomes: because sex=0 for female subjects. For male subjects, the equation becomes:

Subtracting equation (2) from equation (3), we get the difference between male and female subjects of the same height, which is m_s. In other words, regardless of the subject's height, our regression model adds m_s pounds to the prediction if the subject is male. Also recognize that regardless of the subject's sex, the equation adds m_h pounds for every extra unit of height. Said another way, the equation assumes that the relationship between height and weight is exactly the same for men and women. This is a very strong assumption.

We now consider a fully explicated model of the form:

Compared with model (1), this model contains the extra term m_sh×sex×height, which allows the relationship between height and weight to be different for each sex. For women, we still have equation (2) because sex=0 in the 2 other terms, and m_h can be interpreted as the slope of the relationship between height and weight for women. For men, equation (4) becomes:

and can be interpreted as the slope of the relationship between height and weight for women. Also observe that the difference between the equations for a man and woman of the same height is m_s+m_sh×height so that, in contradistinction to simple model (1), the value is no longer the same regardless of the person's height.

We have gone through all of this in some detail to make clear the difference between the fully explicated model (4) and the more commonly used model (1). They differ by the term (m_sh×sex×height) or, said another way, model 1 assumes that m_sh=0. This strong assumption—that the relationship between height and weight is identical for men and women—is hidden in equation (1). The adage “the most important thing about a model is what's left out” pertains here because it is easy to forget that each missing element translates to a strong assumption about relationships of the variables.

Why do investigators not always use fully explicated models? There are at least 4 reasons. First, simple models are so commonplace that some investigators may not understand the benefit of including cross-product (aka “interaction”) terms. Second, one needs more data to fit a model with interaction terms, and those data may not be available. Third, even if there are sufficient data to fit the model, confidence levels of coefficients are likely wider and P values likely higher for the more complex model, and investigators may not like this. Finally, fully explicated models are more complex and interpretation of their coefficients is less straightforward. The basic equation for a model with 3 independent variables is:

and the number of interaction terms becomes quite large as the number of variables increases. A compromise strategy is to eliminate those interaction terms for which there is strong a priori evidence that the interaction is likely to be small.

Finally, we address some limitations of modeling. There are entire books written on this topic; we touch briefly on 2 important points and encourage readers to seek additional information if they are curious. As we allude to above, one of the strongest assumptions is that the model form (the shape of the relationship among the variables) is correct. We often hear statements that “the model reaches statistical significance, proving that the relationship is linear.” Nothing could be further from the truth. The fact that one runs a linear regression and gets a statistically significant result tells one nothing about whether the relationship among the data is linear. The result is based on the assumption that it is linear. One should not perform any type of regression without first graphing the data and gaining a visual understanding of the nature of the relationships.

Second, for simplicity we have omitted error terms in the equations presented above. However, a consideration of error terms is essential if one is to understand the assumptions of the mathematics of regression. Readers should understand that every model, be it linear, logistic, proportional, or other, makes assumptions about the form of these residuals (errors) and that there are situations for which violations of these assumptions can lead to biased estimates. One should never report a regression without checking the residuals.

In summary, regression modeling can be a very useful tool for understanding relationships among variables in a data set and for making predictions about future cases. However, modeling requires many assumptions about the form of the model, the distribution of the data, and the nature of the interactions among variables. Makers and users of models should think carefully about these aspects of the modeling process.

Q3.c In the “Limitations” section, the authors mention that they used a fixed-effect model to account for the “likely variation in field care, hospital care, and injury characteristics between sites, EMS agencies, and hospitals.” Define what a fixed-effect model is and how it differs from a random-effects model.

Two sources of variance must be considered in cluster design studies, variability of patients within the cluster and variability among clusters.³⁸ The combined increase in variance results in widened CIs and increased P values compared with that in a study of individual patients.³⁸ These authors acknowledged that variation in out-of-hospital and hospital care might bias their conclusions. Newgard et al² used a fixed-effect model with the 10 sites as clusters to account for this possibility. A fixed-effect model assumes that the clusters are homogenous; therefore, the weight of each cluster to the overall conclusion is based on the inverse of the variance of the study sites within the cluster.³⁹ Large clusters with smaller variances will have a larger effect on the overall results.³⁹ A random-effect model assumes variability among clusters and subsequently adds variance to the site's effect in proportion to the variability of the individual sites within the cluster.³⁹ Random-effect models assume less homogeneity among clusters and are therefore more conservative models. These authors analyzed their data with both a fixed-effect and random-effect model, without a qualitative change in their outcome.

Back to Article Outline

Answer 4 

Q4. In your opinion, what are the most important conclusions from this article? How might these conclusions affect EMS transport of trauma patients in urban and rural settings? How might the limitations mentioned by the authors affect your medical control recommendations for patient transport by medical helicopter versus ground transport with or without lights and sirens?

Newgard et al² conclude there was no association between EMS intervals and patient mortality among injured patients with physiologic abnormality who were prospectively sampled from a diverse group of sites and EMS agencies across North America. The patients in the analysis were a severely injured group, as demonstrated by the 22% mortality rate, most dying on the first day of hospital presentation.² The results support the concept of transporting severely injured trauma patients to a trauma center even if this may delay the initiation of hospital-based trauma care by bypassing closer facilities. This study included 146 advanced ground and air EMS agencies that transport to Level I and II trauma centers. The study does include rural EMS providers, but transport times were still rapid, with most agencies spending 10 minutes or less en route to a trauma center. The conclusions of the authors should not be generalized to very rural areas in which volunteer EMS services provide most of the out-of-hospital care, with much longer transport intervals.

The authors mention several limitations to their study. These include the lack of detailed hospital-based information, including measures of injury severity, the loss of long-term follow-up or functional outcomes, the substantial variability in intervals among sites and heterogeneity in the patient population, and variability in field care, hospital care, and injury characteristics among sites, EMS agencies, and hospitals. Despite these limitations, this was a very well-designed and conducted multisite out-of-hospital study that questions the importance of rapid transport to a hospital, given the risks associated with “lights and sirens.” When providing medical control to incoming ground or air EMS agencies, the physician must be familiar with the transporting unit's protocols, level of training, and scope of practice. Familiarity with these issues will result in more informed recommendations regarding the transfer of critically ill trauma patients. The data from this large Epistry, combined with many of the referenced studies, should encourage local EMS directors to review their protocols for activating helicopter or emergency “lights and sirens” ground units when transporting severely injured trauma patients. Does the questionable improved patient survival by rapid transport to a trauma center outweigh the risks to the patient, crew, and innocent bystanders? We hope that this well-designed out-of-hospital study encourages more informed transport decisions that consider both the patient's welfare and the safety of the patient, transport team, and general public.

Back to Article Outline

References 

1. Burton A, Altman DG. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. Br J Cancer. 2004;91:4–8
   • View In Article
   • MEDLINE
   • CrossRef
2. Newgard CD, Schmicker RH, Hedges JR, et al. Emergency medical services intervals and survival in trauma: assessment of the “golden hour” in a North American prospective cohort. Ann Emerg Med. 2010;55:235–246e234
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (741 KB)
   • CrossRef
3. Donders AR, van der Heijden GJ, Stijnen T, et al. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59:1087–1091
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (101 KB)
   • CrossRef
4. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer; 2009;
   • View In Article
5. Allison PD. Missing data. In:  Lewis-Beck MS editors. Sage University Papers Series on Quantitative Applications in the Social Sciences: No. 07-136. Thousand Oaks, CA: Sage Publications; 2001;p. 1–12
   • View In Article
6. Harrell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer; 2001;
   • View In Article
7. Schafer JL. Analysis of Incomplete Multivariate Data. New York, NY: Chapman & Hall; 1997;
   • View In Article
8. de Groot JA, Janssen KJ, Zwinderman AH, et al. Multiple imputation to correct for partial verification bias revisited. Stat Med. 2008;27:5880–5889
   • View In Article
9. Janssen KJ, Vergouwe Y, Donders AR, et al. Dealing with missing predictor values when applying clinical prediction models. Clin Chem. 2009;55:994–1001
   • View In Article
   • CrossRef
10. van der Heijden GJ, Donders AR, Stijnen T, et al. Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example. J Clin Epidemiol. 2006;59:1102–1109
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (126 KB)
    • CrossRef
11. Moylan JA. Impact of helicopters on trauma care and clinical results. Ann Surg. 1988;208:673–678
    • View In Article
    • MEDLINE
    • CrossRef
12. Neel SH. Helicopter evacuation in Korea. U S Armed Forces Med J. 1955;6:691–702
    • View In Article
    • MEDLINE
13. Neel S. Army aeromedical evacuation procedures in Vietnam: implications for rural America. JAMA. 1968;204:309–313
    • View In Article
    • MEDLINE
14. Law DK, Law JK, Brennan R, et al. Trauma operating room in conjunction with an air ambulance system: indications, interventions, and outcomes. J Trauma. 1982;22:759–765
    • View In Article
    • MEDLINE
15. Cleveland HC, Bigelow DB, Dracon D, et al. A civilian air emergency service: a report of its development, technical aspects, and experience. J Trauma. 1976;16:452–463
    • View In Article
    • MEDLINE
16. Topol EJ, Fung AY, Kline E, et al. Safety of helicopter transport and out-of-hospital intravenous fibrinolytic therapy in patients with evolving myocardial infarction. Cathet Cardiovasc Diagn. 1986;12:151–155
    • View In Article
    • MEDLINE
17. Abernethy M, Bledsoe B, Carrison D. Can emergency physicians fix the helicopter EMS system?. Emergency Physicians Monthly. 2010;Vol 17:http://www.epmonthly.com/features/current-features/can-eps-fix-the-helicopter-ems-system?/Accessed April 19, 2010
    • View In Article
18. EMS Committee. Appropriate utilization of air medical transport in the out-of-hospital setting. Ann Emerg Med. 1999;34:420
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (38 KB)
    • CrossRef
19. Air ambulance medical transport advertising and marketing. Ann Emerg Med. 2008;52:580–581
    • View In Article
    • Full Text
    • Full-Text PDF (61 KB)
    • CrossRef
20. Talving P, Teixeira PG, Barmparas G, et al. Helicopter evacuation of trauma victims in Los Angeles: does it improve survival?. World J Surg. 2009;33:2469–2476
    • View In Article
    • CrossRef
21. Ringburg AN, Spanjersberg WR, Frankema SP, et al. Helicopter emergency medical services (HEMS): impact on on-scene times. J Trauma. 2007;63:258–262
    • View In Article
22. Biewener A, Aschenbrenner U, Rammelt S, et al. Impact of helicopter transport and hospital level on mortality of polytrauma patients. J Trauma. 2004;56:94–98
    • View In Article
    • MEDLINE
23. Buntman AJ, Yeomans KA. The effect of air medical transport on survival after trauma in Johannesburg, South Africa. S Afr Med J. 2002;92:807–811
    • View In Article
    • MEDLINE
24. Di Bartolomeo S, Sanson G, Nardi G, et al. HEMS vs. Ground-BLS care in traumatic cardiac arrest. Prehosp Emerg Care. 2005;9:79–84
    • View In Article
    • MEDLINE
    • CrossRef
25. Newgard CD, McConnell KJ, Hedges JR, et al. The benefit of higher level of care transfer of injured patients from nontertiary hospital emergency departments. J Trauma. 2007;63:965–971
    • View In Article
26. Peth HA. The Emergency Medical Treatment and Active Labor Act (EMTALA): guidelines for compliance. Emerg Med Clin North Am. 2004;22:225–240
    • View In Article
    • Full Text
    • Full-Text PDF (233 KB)
    • CrossRef
27. Centers for Medicare & Medicaid Services. State operations manual appendix V—interpretive guidelines—responsibilities of Medicare participating hospitals in emergency cases. http://www.cms.hhs.gov/manuals/Downloads/som107ap_v_emerg.pdfAccessed December 22, 2009
    • View In Article
28. McConnell KJ, Newgard CD, Mullins RJ, et al. Mortality benefit of transfer to level I versus level II trauma centers for head-injured patients. Health Serv Res. 2005;40:435–457
    • View In Article
    • MEDLINE
    • CrossRef
29. Cudnik MT, Newgard CD, Sayre MR, et al. Level I versus level II trauma centers: an outcomes-based assessment. J Trauma. 2009;66:1321–1326
    • View In Article
30. Koval KJ, Tingey CW, Spratt KF. Are patients being transferred to level-I trauma centers for reasons other than medical necessity?. J Bone Joint Surg Am. 2006;88:2124–2132
    • View In Article
    • MEDLINE
    • CrossRef
31. Spain DA, Bellino M, Kopelman A, et al. Requests for 692 transfers to an academic level I trauma center: implications of the emergency medical treatment and active labor act. J Trauma. 2007;62:63–67discussion 67-68
    • View In Article
    • MEDLINE
32. Cooper RJ. Misleading negative chest radiographs: should we ADHERE to the conclusions?. Ann Emerg Med. 2006;47:19–21
    • View In Article
    • Full Text
    • Full-Text PDF (69 KB)
    • CrossRef
33. Izquierdo JN, Schoenbach VJ. The potential and limitations of data from population-based state cancer registries. Am J Public Health. 2000;90:695–698
    • View In Article
    • MEDLINE
    • CrossRef
34. Gilbert EH, Lowenstein SR, Koziol-McLain J, et al. Chart reviews in emergency medicine research: where are the methods?. Ann Emerg Med. 1996;27:305–308
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (355 KB)
    • CrossRef
35. Newgard CD, Sears GK, Rea TD, et al. The Resuscitation Outcomes Consortium Epistry–Trauma: design, development, and implementation of a North American epidemiologic prehospital trauma registry. Resuscitation. 2008;78:170–178
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (347 KB)
    • CrossRef
36. Barrett TW, Schriger DL Annals of Emergency Medicine Journal Club. Measures of emergency department crowding, odds ratios, and the dangers of making continuous data categorical (Answers to January 2008 journal club questions). Ann Emerg Med. Jun 2008;51:782–789
    • View In Article
    • Full Text
    • Full-Text PDF (577 KB)
    • CrossRef
37. Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. Journal of the Royal Statistical Society. Series C (Applied Statistics). 1994;43:429–467
    • View In Article
38. Wears RL. Advanced statistics: statistical methods for analyzing cluster and cluster-randomized data. Acad Emerg Med. 2002;9:330–341
    • View In Article
    • MEDLINE
    • CrossRef
39. Grady D, Hearst N. Utilizing existing databases. In:  Hulley S,  Cummings S, Browner , et al. editor. Designing Clinical Research. 3rd ed.. Philadelphia, PA: Lippincott Williams & Wilkins; 2007;p. 220
    • View In Article

• ⁎ There are many excellent explanations of the mathematics of these techniques in textbooks and online and we will not review them here.