Evaluation of a Modified Prediction Instrument to Identify Significant Pediatric Intracranial Injury After Blunt Head Trauma

Introduction 

Goals of This Investigation 

We assessed a modified version of the University of California–Davis Rule that used stricter definitions of “headache” and “vomiting.” The pediatric subset of the National Emergency X-Radiography Utilization Study II (NEXUS II) includes prospective data on a large cohort of children who received head CT scan after blunt head injury. Because NEXUS II did not collect the same elements used for the University of California–Davis rule, we had to modify 2 predictors in NEXUS II, including presence of “high-risk” vomiting and “severe” headache, in place of any vomiting or headache as used in the University of California–Davis derivation set. Additionally, the wording of some other questions also differed between the 2 studies, as shown in Table 1.

Table 1. Candidate predictors and definitions.

	Variable	University of California–Davis Definition	NEXUS II Definition
	Scalp hematoma	Scalp hematoma	Refers to swelling felt to be a result of hematoma formation over any portion of the bony calvarium. Injuries that do not involve the calvarium, including hematomas limited to the face and neck, are not considered scalp hematomas
	Skull fracture	Includes palpable skull fracture, retroauricular bruising, periorbital bruising, hemotympanum, or cerebrospinal fluid otorrhea or rhinorrhea	Includes but not limited to any signs of basilar skull fracture, such as periorbital or periauricular ecchymoses, hemotympanum, and drainage of clear fluid from ears and nose; and signs of depressed skull fracture including a palpable step-off of skull
	Abnormal mental status	Positive for any of the following:	Abnormal mental status. Positive for any of the following:
		GCS score <15 (absence of spontaneous eye opening, normal orientation, ability to follow commands) Pediatric GCS score <15 in preverbal children (absence of spontaneous eye opening, appropriate coos, normal spontaneous movement) Patient is confused, somnolent, repetitive, or slow to respond to verbal communication	GCS score <15 (absence of spontaneous eye opening, normal orientation, ability to follow commands) Abnormal alertness. Includes but not limited to delayed or inappropriate response, excessive somnolence, to inability remember 3 objects at 5 minutes; or perseverating speech Abnormal behavior. Includes but not limited to excessive agitation, inconsolability, and violent activity
	Vomiting	History of vomiting	Includes but not limited to recurrent, projectile, or forceful emesis after trauma, or vomiting associated with altered sensorium
	Headache	Headache	Includes but not limited to any head pain deemed by the patient to be severe or progressive in nature

Materials and Methods 

Outcome Measures 

Clinically important intracranial injury was defined through an expert panel process as any injury that may require neurosurgical intervention (craniotomy, intracranial pressure monitoring, mechanical ventilation), lead to rapid clinical deterioration, or result in significant long-term neurologic impairment24 (Figure 1).

View Large Image Download to PowerPoint

Figure 1. CT findings representing clinically important intracranial injury.

Head CTs were interpreted by clinical radiologists at each site. Copies of all final radiographic readings were collected and abstracted to determine the presence or absence of clinically important intracranial injury. Investigators determined final injury classification while blinded to information about clinical variables.

Results 

Of the overall NEXUS II cohort of 13,728 patients, 1,666 were younger than 18 years and composed the analytic cohort for this study. The age distribution of the pediatric cohort is presented in Appendix E1 (available online at http://www.annemergmed.com). Baseline characteristics of the pediatric study population are presented in Table 2. There were 138 (8.3%) patients with clinically important intracranial injury on head CT. There were no clinically important differences in age, sex, and race/ethnicity between children with and without clinically important intracranial injury. On univariate testing, the presence of abnormal mental status, signs of skull fracture, high-risk vomiting, and presence of at least 1 of the 5 high-risk predictors were highly associated with presence of clinically important intracranial injury. Data were missing for these predictors in 2% to 12% of patients. Specific intracranial injuries are summarized in Table 3.

Table 2. Study population demographics.

	Characteristic	Absent Intracranial Injury (n=1,528)	Present Intracranial Injury (n=138)	Relative Risk Ratio (95% CI)
	Age (IQR)	10.3(4.4,16)	9.6(4.8,15.1)
	Male	64%	63%	0.95(0.7–1.3)
	Race/ethnicity, %
	White	49	50
	Black	16	9
	Hispanic	20	23
	Other/unknown	15	17
	Presence of high-risk predictors, %⁎
	Abnormal mental status	38	75	4.4(3.0–6.3)
	Signs of skull fracture	3	28	6.9(5.0–9.7)
	High risk vomiting	11	24	2.2(1.6–3.2)
	Scalp hematoma in children ≤2 y†	39	57	2.0(0.5–8.7)
	Severe or progressive headache	15	20	1.4(0.8–2.4)
	Any high-risk predictor	61	91	6.4(3.5–12.0)

⁎ Frequency of missing data: abnormal mental status (7.4%), skull fracture (4.8%), high-risk vomiting (1.6%), scalp hematoma in children ≤2 years (2.5%), severe headache (20.5%). † Of patients ≤2 years of age, 201 had absent intracranial injury and 7 had present intracranial injury.

Table 3. Significant intracranial injuries on head CT.

	Type of Injury	Patients With Significant Intracranial Injury, No. (%)⁎
	Cerebral contusion	68(49.3)
	Subarachnoid hemorrhage	41(29.7)
	Subdural hematoma	38(27.5)
	Depressed skull fracture	30(21.7)
	Epidural hematoma	27(19.6)
	Pneumocephalus	26(18.8)
	Diffuse cerebral edema	19(13.8)
	Intraventricular hemorrhage	10(7.2)
	Other significant cerebral injuries	22(15.9)

⁎ One hundred thirty-eight patients accounted for a total of 281 significant traumatic brain injuries.

Interrater reliability of the measurements was moderate, and κ values ranged between 0.54 to 0.75 for each of the 5 high-risk predictors and 0.53 for the presence of any high-risk predictor (Table 4).

Table 4. Interobserver agreement on presence or absence of predictors (n=265).

	Predictor	Subgroup No.	Agreement, %	κ (95% CI)
	Abnormal mental status	264	79.2	0.56(0.5–0.7)
	Signs of skull fracture	233	97.8	0.75(0.5–1.0)
	High-risk vomiting	256	93.4	0.73(0.6–0.9)
	Scalp hematoma in a child ≤2 y	169	87.6	0.71(0.6–0.8)
	Severe or progressive headache	199	85.9	0.54(0.4–0.7)
	Any high-risk predictor⁎	202	83.6	0.53(0.4–0.7)

⁎ Observation was dropped if either rater did not complete all elements of the modified University of California–Davis Rule and no abnormal findings were present.

The test characteristics of the modified University of California–Davis Rule are presented in Table 5. In the base case analysis, 13 (9%) patients with clinically important intracranial injury had none of the 5 high-risk predictors. The test characteristics of the modified prediction rule were minimally sensitive to different assumptions for handling missing data. None of the misclassified patients were younger than 2 years, and the test characteristics of the modified prediction rule were minimally affected by excluding patients younger than 2 years. Figure 2 describes the injuries of patients who were misclassified as “low risk.”

Table 5. Performance of modified prediction rule to identify significant traumatic intracranial injury.

	Assumption	Sensitivity (95% CI)	Specificity (95% CI)	Positive Predictive Value (95% CI)	Negative Predictive Value (95% CI)
	Base case	90.4(85.5–95.4)	42.7(40.1–45.3)	13.2(11.0–15.3)	97.9(96.8–99.0)
	Missing predictors set to “present”	90.6(85.7–95.5)	39.6(37.1–42)	11.9(9.9–13.9)	97.9(96.7–99.0)
	Missing predictors set to “absent”	89.1(83.9–94.3)	46.9(44.4–49.4)	13.2(11.0–15.3)	98.0(96.9–99.0)
	Age 2–18, base case assumptions	89.4(84.0–94.9)	46.1(43.3–48.9)	14.2(11.8–16.7)	97.8(96.5–99.0)

View Large Image Download to PowerPoint

Figure 2. Patients misclassified as “low risk” in base-case analysis.

Limitations 

There are a number of limitations to our study. Not all patients with blunt head injury received head CT. However, we performed a single center substudy of 1,266 patients with blunt head injury who did not receive head CT; none of these patients subsequently died, were diagnosed with an intracranial injury, or required a neurosurgical procedure at 3-month follow-up.23 Thus, we believe the potential effect of selection bias on our reported sensitivity is minimal.

We did not include the need for acute neurosurgical intervention as an outcome. The frequency of neurosurgical procedures in pediatric blunt head trauma is approximately 0.4% to 1.5%,3, 26, 27 and derivation of a reliable decision rule to identify neurosurgical intervention would likely require a study with more than 30,000 children. Thus, NEXUS II was not sufficiently powered to assess neurosurgical interventions as an outcome. Furthermore, the definition of acute intervention in the original University of California–Davis study included “hospitalization for 2 or more nights” and “anti-epileptic medication for more than 1 week.” Prolonged hospitalization and treatment with antiepileptic medications accounted for 69% of patients requiring acute intervention in the original University of California–Davis study, and it is possible that these outcomes are subject to variations in clinical judgment. Finally, many emergency physicians regard a missed traumatic brain injury that could have been identified on CT scan as unacceptable.28 Therefore, we believe that CT evidence of intracranial injury is a clinically relevant and important outcome.

Although the definitions for “abnormal mental status” were similar between NEXUS II and the original University of California–Davis study, NEXUS II did not collect data on pediatric GCS score in preverbal patients. However, none of the misclassified patients were younger than 2 years, and excluding children younger than 2 years from our analysis had no important effects on our overall findings. We suspect that children with an abnormal pediatric GCS score were appropriately classified by physicians as having abnormal alertness or behavior, which were both components of “abnormal mental status” in the NEXUS II study.

There may be important physiologic differences among infants, preadolescents, and teenagers, and a decision rule to identify pediatric head injury may have various test characteristics that are age dependent. In a post hoc sensitivity analysis, we did not find qualitatively important test differences of the modified University of California–Davis Rule in children aged 2 to 12 years (sensitivity 92%, 95% CI 85% to 98%; specificity 38%, 95% CI 34% to 42%) and aged 12 to 18 years (sensitivity 90%, 95% CI 81% to 98%; specificity 47%, 95% CI 43% to 51%) compared to the overall cohort. Test characteristics of the modified University of California–Davis Rule were more sensitive and less specific in children younger than 2 years (sensitivity 100%, 95% CI 72% to 100%; specificity 11%, 95% CI 6% to 16%); however, the generalizability of this finding is limited, given the small number of children in this subgroup (n=208 with 7 significant intracranial injuries).

The outcomes definitions were different between the 2 studies. The original University of California–Davis Rule considered any intracranial hemorrhage, hematoma, or cerebral edema to be a serious outcome, whereas NEXUS II used a more stringent set of definitions for significant intracranial injury (see Figure 1). However, in a post hoc sensitivity analysis that considered any intracranial hemorrhage, hematoma, or cerebral edema to be a serious outcome (excluding isolated, nondepressed skull fracture), there were no important changes in sensitivity (90.5%; 95% CI 85.8% to 95.3%) or specificity (43.0%; 95% CI 40.4% to 45.6%) of the modified University of California–Davis Rule.

Most important, we were unable to assess the precise University of California–Davis Rule because the data in the 2 studies were differently defined at collection, and thus we can comment directly only on a modified version of that decision instrument.

Discussion 

NEXUS II is a multicenter study that to our knowledge includes the largest published cohort of pediatric blunt head trauma patients for whom head CT scanning was done. The study design maximizes generalizability and allows us to assess the performance of any combination of clinical variables prospectively assessed for this study. Because we did not query 2 of the precise variables used to derive the University of California–Davis Rule, however (and other questions varied slightly in their wording), we can only assess a modified version of that decision instrument. We demonstrate that a modified version of the original University of California–Davis Rule results in misclassification of a significant proportion of children with clinically important intracranial injury in the NEXUS II cohort.

Identification of a reliable set of clinical predictors to identify clinically important intracranial injury in children might safely reduce use of head CT, and the University of California–Davis Rule is an important advance in this endeavor. The original derivation study prospectively collected data on 2,043 children with nontrivial blunt head injury, and a head CT was obtained in 1,271 (62%) patients. The 2 outcome measures included traumatic brain injury identified on CT imaging (excluding isolated skull fractures) and the need for acute intervention (defined as a neurosurgical procedure, persistent neurologic deficit at discharge, treatment with antiepileptic medications for more than 1 week, or hospitalization for 2 or more nights). The 5 clinical predictors of abnormal mental status, signs of skull fracture, scalp hematoma in children 2 years of age or younger, vomiting, or headache demonstrated high sensitivity for both outcomes.

In the original derivation study, the University of California–Davis Rule had a sensitivity of 99% (95% CI 94.4% to 100%) and specificity of 26% (95% CI 23.3% to 28.4%) for identifying children with any traumatic head injury on head CT.3 In the NEXUS II cohort, a modification of the University of California–Davis Rule demonstrated lower sensitivity (90.4%, 95% CI 85.5% to 95.4%) and higher specificity (42.7%, 95% CI 40.1% to 45.3%) to identify clinically significant intracranial injury.

The NEXUS II measurements of “high risk” vomiting and “severe” headache, rather than any vomiting or headache, may have contributed to lower sensitivity and higher specificity of the modified University of California–Davis rule than in the original study. Because NEXUS II did not include information about the presence of any vomiting or headache, we were unable to assess the exact University of California–Davis Rule. Although these differences may seem subtle, they may have an important impact on the accurate identification of patients with clinically important intracranial injury.

As for any such decision instrument, the University of California–Davis Rule should be applied with careful attention to the exact definitions of clinical predictors. Although we were unable to perform an external validation of the precise original University of California–Davis Rule, such studies are necessary before widespread application of this decision instrument. How much changes in definitions affect performance is not known. These caveats are particularly important in this instance because the University of California–Davis derivation study is now required reading for all board-certified emergency physicians through the American Board of Emergency Medicine's Life Long Self-Assessment program.21

In a separate report, we assessed other potential predictors of pediatric head injury in the NEXUS II cohort.2 A combination of criteria, including evidence of skull fracture, altered level of alertness, neurologic deficit, persistent vomiting, presence of scalp hematoma in any age, abnormal behavior, and coagulopathy, identified significant pediatric intracranial injury with a sensitivity of 98.6% (95% CI 94.9% to 99.8%) and specificity of 15.1% (95% CI 13.3% to 16.9%). These findings will require validation in an external cohort.