Nontraumatic Subarachnoid Hemorrhage in the Setting of Negative Cranial Computed Tomography Results: External Validation of a Clinical and Imaging Prediction Rule

      Study objective

      Clinical variables can reliably exclude a diagnosis of nontraumatic subarachnoid hemorrhage in patients with negative cranial computed tomography (CT) results. We externally validated 2 decision rules with 100% reported sensitivity for a diagnosis of subarachnoid hemorrhage, among patients undergoing lumbar puncture after a negative cranial CT result: (1) clinical rule: presence of any combination of age 40 years and older, neck pain or stiffness, loss of consciousness, or headache onset during exertion; and (2) imaging rule: cranial CT performed within 6 hours of headache onset.


      This was a matched case-control study of patients presenting to 21 emergency departments between 2000 and 2011. Patients with a diagnosis of subarachnoid hemorrhage as determined by lumbar puncture after a negative cranial CT result were screened for inclusion. A matched control cohort was selected among patients with a diagnosis of headache after negative cranial CT and lumbar puncture results.


      Fifty-five cases of subarachnoid hemorrhage meeting inclusion criteria were identified, 34 (62%) of which were attributed to cerebral aneurysms. External validation of the clinical rule demonstrated a sensitivity of 97.1% (95% confidence interval [CI] 88.6% to 99.7%), a specificity of 22.7% (95% CI 16.6% to 29.8%), and a negative likelihood ratio of 0.13 (95% CI 0.03 to 0.61) for a diagnosis of subarachnoid hemorrhage. External validation of the imaging rule revealed that 11 of 55 subarachnoid hemorrhage cases (20%) had negative cranial CT results for tests performed within 6 hours of headache onset.


      The clinical rule demonstrated useful Bayesian test characteristics when retrospectively validated against this patient cohort. The imaging rule, however, failed to identify 20% of subarachnoid hemorrhage patients with a negative cranial CT result.
      To read this article in full you will need to make a payment
      ACEP Member Login
      ACEP Members, full access to the journal is a member benefit. Use your society credentials to access all journal content and features.
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Purchase one-time access:

      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ramirez-Lassepas M.
        • Espinosa C.E.
        • Cicero J.J.
        • et al.
        Predictors of intracranial pathologic findings in patients who seek emergency care because of headache.
        Arch Neurol. 1997; 54: 1506-1509
        • Perry J.J.
        • Stiell I.
        • Wells G.
        • et al.
        Diagnostic test utilization in the emergency department for alert headache patients with possible subarachnoid hemorrhage.
        CJEM. 2002; 4: 333-337
        • Edlow J.A.
        • Caplan L.R.
        Avoiding pitfalls in the diagnosis of subarachnoid hemorrhage.
        N Engl J Med. 2000; 342: 29-36
        • Leicht M.J.
        Non-traumatic headache in the emergency department.
        Ann Emerg Med. 1980; 9: 404-409
        • Morgenstern L.B.
        • Luna-Gonzales H.
        • Huber Jr, J.C.
        • et al.
        Worst headache and subarachnoid hemorrhage: prospective, modern computed tomography and spinal fluid analysis.
        Ann Emerg Med. 1998; 32: 297-304
        • Nishioka H.
        • Torner J.C.
        • Graf C.J.
        • et al.
        Cooperative study of intracranial aneurysms and subarachnoid hemorrhage: a long-term prognostic study. II. Ruptured intracranial aneurysms managed conservatively.
        Arch Neurol. 1984; 41: 1142-1146
        • Nibbelink D.W.
        • Torner J.C.
        • Henderson W.G.
        Intracranial aneurysms and subarachnoid hemorrhage—report on a randomized treatment study. IV-A. Regulated bed rest.
        Stroke. 1977; 8: 202-218
        • Kassell N.F.
        • Boarini D.J.
        • Adams Jr, H.P.
        • et al.
        Overall management of ruptured aneurysm: comparison of early and late operation.
        Neurosurgery. 1981; 9: 120-128
        • Nibbelink D.W.
        • Knowler L.A.
        Cooperative study of intracranial aneurysms and subarachnoid hemorrhage. Report on a randomized treatment study. II. Objectives and design of randomized aneurysm study.
        Stroke. 1974; 5: 552-556
        • Mayer P.L.
        • Awad I.A.
        • Todor R.
        • et al.
        Misdiagnosis of symptomatic cerebral aneurysm. Prevalence and correlation with outcome at four institutions.
        Stroke. 1996; 27: 1558-1563
        • Kowalski R.G.
        • Claassen J.
        • Kreiter K.T.
        • et al.
        Initial misdiagnosis and outcome after subarachnoid hemorrhage.
        JAMA. 2004; 291: 866-869
        • Molyneux A.J.
        • Kerr R.S.
        • Birks J.
        • et al.
        Risk of recurrent subarachnoid haemorrhage, death, or dependence and standardised mortality ratios after clipping or coiling of an intracranial aneurysm in the International Subarachnoid Aneurysm Trial (ISAT): long-term follow-up.
        Lancet Neurol. 2009; 8: 427-433
        • Vermeulen M.J.
        • Schull M.J.
        Missed diagnosis of subarachnoid hemorrhage in the emergency department.
        Stroke. 2007; 38: 1216-1221
        • Vannemreddy P.
        • Nanda A.
        • Kelley R.
        • et al.
        Delayed diagnosis of intracranial aneurysms: confounding factors in clinical presentation and the influence of misdiagnosis on outcome.
        South Med J. 2001; 94: 1108-1111
        • Ritz R.
        • Reif J.
        Comparison of prognosis and complications after warning leaks in subarachnoidal hemorrhage—experience with 214 patients following aneurysm clipping.
        Neurol Res. 2005; 27: 620-624
        • Jakobsson K.E.
        • Saveland H.
        • Hillman J.
        • et al.
        Warning leak and management outcome in aneurysmal subarachnoid hemorrhage.
        J Neurosurg. 1996; 85: 995-999
        • Neil-Dwyer G.
        • Lang D.
        • Smith P.
        • et al.
        Outcome after aneurysmal subarachnoid haemorrhage: the use of a graphical model in the assessment of risk factors.
        Acta Neurochir (Wien). 1998; 140: 1019-1027
        • Kassell N.F.
        • Torner J.C.
        • Jane J.A.
        • et al.
        The International Cooperative Study on the Timing of Aneurysm Surgery. Part 2: surgical results.
        J Neurosurg. 1990; 73: 37-47
        • Perry J.J.
        • Spacek A.
        • Forbes M.
        • et al.
        Is the combination of negative computed tomography result and negative lumbar puncture result sufficient to rule out subarachnoid hemorrhage?.
        Ann Emerg Med. 2008; 51: 707-713
        • Perry J.J.
        • Stiell I.G.
        • Sivilotti M.L.
        • et al.
        Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study.
        BMJ. 2011; 343: d4277
        • Cortnum S.
        • Sorensen P.
        • Jorgensen J.
        Determining the sensitivity of computed tomography scanning in early detection of subarachnoid hemorrhage.
        Neurosurgery. 2010; 66 (discussion 903): 900-902
        • Carstairs S.D.
        • Tanen D.A.
        • Duncan T.D.
        • et al.
        Computed tomographic angiography for the evaluation of aneurysmal subarachnoid hemorrhage.
        Acad Emerg Med. 2006; 13: 486-492
        • Byyny R.L.
        • Mower W.R.
        • Shum N.
        • et al.
        Sensitivity of noncontrast cranial computed tomography for the emergency department diagnosis of subarachnoid hemorrhage.
        Ann Emerg Med. 2008; 51: 697-703
        • Lourenco A.P.
        • Mayo-Smith W.W.
        • Tubbs R.J.
        • et al.
        Does 16-detector computed tomography improve detection of non-traumatic subarachnoid hemorrhage in the emergency department?.
        J Emerg Med. 2009; 36: 171-175
        • Gee C.
        • Dawson M.
        • Bledsoe J.
        • et al.
        Sensitivity of newer-generation computed tomography scanners for subarachnoid hemorrhage: a Bayesian analysis.
        J Emerg Med. 2012; 43: 13-18
        • Edlow J.A.
        • Wyer P.C.
        Evidence-based emergency medicine/clinical question. How good is a negative cranial computed tomographic scan result in excluding subarachnoid hemorrhage?.
        Ann Emerg Med. 2000; 36: 507-516
        • Sidman R.
        • Connolly E.
        • Lemke T.
        Subarachnoid hemorrhage diagnosis: lumbar puncture is still needed when the computed tomography scan is normal.
        Acad Emerg Med. 1996; 3: 827-831
        • Sames T.A.
        • Storrow A.B.
        • Finkelstein J.A.
        • et al.
        Sensitivity of new-generation computed tomography in subarachnoid hemorrhage.
        Acad Emerg Med. 1996; 3: 16-20
        • van der Wee N.
        • Rinkel G.J.
        • Hasan D.
        • et al.
        Detection of subarachnoid haemorrhage on early CT: is lumbar puncture still needed after a negative scan?.
        J Neurol Neurosurg Psychiatry. 1995; 58: 357-359
        • Boesiger B.M.
        • Shiber J.R.
        Subarachnoid hemorrhage diagnosis by computed tomography and lumbar puncture: are fifth generation CT scanners better at identifying subarachnoid hemorrhage?.
        J Emerg Med. 2005; 29: 23-27
        • Schievink W.I.
        Intracranial aneurysms.
        N Engl J Med. 1997; 336: 28-40
        • Perry J.J.
        • Sivilotti M.L.
        • Stiell I.G.
        • et al.
        Should spectrophotometry be used to identify xanthochromia in the cerebrospinal fluid of alert patients suspected of having subarachnoid hemorrhage?.
        Stroke. 2006; 37: 2467-2472
        • MacDonald A.
        • Mendelow A.D.
        Xanthochromia revisited: a re-evaluation of lumbar puncture and CT scanning in the diagnosis of subarachnoid haemorrhage.
        J Neurol Neurosurg Psychiatry. 1988; 51: 342-344
        • Arora S.
        • Swadron S.P.
        • Dissanayake V.
        Evaluating the sensitivity of visual xanthochromia in patients with subarachnoid hemorrhage.
        J Emerg Med. 2010; 39: 13-16
        • Heasley D.C.
        • Mohamed M.A.
        • Yousem D.M.
        Clearing of red blood cells in lumbar puncture does not rule out ruptured aneurysm in patients with suspected subarachnoid hemorrhage but negative head CT findings.
        AJNR Am J Neuroradiol. 2005; 26: 820-824
        • Vermeulen M.
        • van Gijn J.
        The diagnosis of subarachnoid haemorrhage.
        J Neurol Neurosurg Psychiatry. 1990; 53: 365-372
        • Perry J.J.
        • Eagles D.
        • Clement C.M.
        • et al.
        An international study of emergency physicians' practice for acute headache management and the need for a clinical decision rule.
        CJEM. 2009; 11: 516-522
        • Perry J.J.
        • Stiell I.G.
        • Sivilotti M.L.
        • et al.
        High risk clinical characteristics for subarachnoid haemorrhage in patients with acute headache: prospective cohort study.
        BMJ. 2010; 341: c5204
        • Allison J.J.
        • Wall T.C.
        • Spettell C.M.
        • et al.
        The art and science of chart review.
        Jt Comm J Qual Improv. 2000; 26: 115-136
        • Raina K.D.
        • Callaway C.
        • Rittenberger J.C.
        • et al.
        Neurological and functional status following cardiac arrest: method and tool utility.
        Resuscitation. 2008; 79: 249-256
        • Zaroff J.G.
        • Leong J.
        • Kim H.
        • et al.
        Cardiovascular predictors of long-term outcomes after non-traumatic subarachnoid hemorrhage.
        Neurocrit Care. 2011;
        • Barnard J.
        • Meng X.L.
        Applications of multiple imputation in medical studies: from AIDS to NHANES.
        Stat Methods Med Res. 1999; 8: 17-36
      1. Cassell DL. Don't be loopy: re-sampling and simulation the SAS way. SAS Global Forum. Orlando, FL; April 16-19, 2007. Paper 183-2007.

      2. Miller DP. Bootstrap 101: obtain robust confidence intervals for any statistic. SUGI 29. Montreal, Canada; May 9-12, 2004. Paper 193-29.

        • Wood M.J.
        • Dimeski G.
        • Nowitzke A.M.
        CSF spectrophotometry in the diagnosis and exclusion of spontaneous subarachnoid haemorrhage.
        J Clin Neurosci. 2005; 12: 142-146
        • Linn F.H.
        • Wijdicks E.F.
        • van der Graaf Y.
        • et al.
        Prospective study of sentinel headache in aneurysmal subarachnoid haemorrhage.
        Lancet. 1994; 344: 590-593
        • Adams Jr, H.P.
        • Kassell N.F.
        • Torner J.C.
        • et al.
        CT and clinical correlations in recent aneurysmal subarachnoid hemorrhage: a preliminary report of the Cooperative Aneurysm Study.
        Neurology. 1983; 33: 981-988
        • Perry J.J.
        • Stiell I.G.
        • Wells G.A.
        • et al.
        Attitudes and judgment of emergency physicians in the management of patients with acute headache.
        Acad Emerg Med. 2005; 12: 33-37
        • Dorsch N.W.
        Cerebral aneurysms and the “missed” hemorrhage.
        Aust N Z J Med. 1986; 16: 486-490
        • Leblanc R.
        The minor leak preceding subarachnoid hemorrhage.
        J Neurosurg. 1987; 66: 35-39
        • Verweij R.D.
        • Wijdicks E.F.
        • van Gijn J.
        Warning headache in aneurysmal subarachnoid hemorrhage. A case-control study.
        Arch Neurol. 1988; 45: 1019-1020
        • Polmear A.
        Sentinel headaches in aneurysmal subarachnoid haemorrhage: what is the true incidence? a systematic review.
        Cephalalgia. 2003; 23: 935-941
        • Adams Jr, H.P.
        • Jergenson D.D.
        • Kassell N.F.
        • et al.
        Pitfalls in the recognition of subarachnoid hemorrhage.
        JAMA. 1980; 244: 794-796
        • van Gijn J.
        Slip-ups in diagnosis of subarachnoid haemorrhage.
        Lancet. 1997; 349: 1492
        • Adams Jr, H.P.
        • Kassell N.F.
        • Torner J.C.
        Usefulness of computed tomography in predicting outcome after aneurysmal subarachnoid hemorrhage: a preliminary report of the Cooperative Aneurysm Study.
        Neurology. 1985; 35: 1263-1267
        • Moreau A.
        • Carol L.
        • Dedianne M.C.
        • et al.
        What perceptions do patients have of decision making (DM)? toward an integrative patient-centered care model. A qualitative study using focus-group interviews.
        Patient Educ Couns. 2012; 87: 206-211
        • Hess E.P.
        • Knoedler M.A.
        • Shah N.D.
        • et al.
        The chest pain choice decision aid: a randomized trial.
        Circ Cardiovasc Qual Outcomes. 2012; 5: 251-259

      Linked Article

      • Clinical Suspicion of Subarachnoid Hemorrhage and Negative Head Computed Tomographic Scan Performed Within 6 Hours of Headache Onset—No Need for Lumbar Puncture
        Annals of Emergency MedicineVol. 61Issue 4
        • Preview
          With interest we read the publication of Mark et al,1 in which it was concluded that head computed tomography (CT) performed within 6 hours of headache onset failed to identify 20% of patients with subarachnoid hemorrhage. This finding contrasts with 2 recent publications showing that third-generation CT scans have 100% negative predictive value for excluding subarachnoid hemorrhage in patients with acute headache.2,3 We have a few comments.
        • Full-Text
        • PDF
      • Pragmatic Interpretation of the Study of Nontraumatic Subarachnoid Hemorrhage in the Setting of Negative Cranial Computed Tomography Results: External Validation of a Clinical and Imaging Prediction Rule
        Annals of Emergency MedicineVol. 62Issue 4
        • Preview
          We commend the efforts of Mark et al1 to validate our clinical decision rules for subarachnoid hemorrhage. Nevertheless, important limitations temper direct comparisons with our work,2,3 namely, (1) including patients with isolated cranial nerve palsies; (2) no rereads of reportedly normal computed tomography (CT) scan results; (3) studying only patients with normal CT results, precluding an estimate of CT sensitivity; and (4) the retrospective nature, which resulted in loss of information (eg, “onset during exertion” missing in 1 in 3 patients).
        • Full-Text
        • PDF
      • The Challenge of Analyzing and Interpreting NHAMCS
        Annals of Emergency MedicineVol. 62Issue 1
        • Preview
          In a recent study and a commentary, Drs. Green and Cooper cast doubt on the accuracy of the National Hospital Ambulatory Medical Care Survey (NHAMCS).1,2 We are concerned about this message and fear that readers will be quick to outright reject the many important insights provided by this valuable resource. Readers must appreciate that NHAMCS has many unique strengths, including its national representation, efficient sampling design, consistent data structure, and longevity (since 1992). NHAMCS has provided some of the most important and valuable insights of emergency department (ED) care in the United States.
        • Full-Text
        • PDF
      • Sensitivity of Computed Tomography for Subarachnoid Hemorrhage
        Annals of Emergency MedicineVol. 62Issue 4
        • Preview
          In their recent study, Mark et al1 retrospectively reviewed medical records from 21 emergency departments (EDs) during a 10-year period, looking for cases of subarachnoid hemorrhage (SAH) that had negative head computed tomography (CT) results. They identified 55 cases, of which 11 patients had negative CTs performed within 6 hours of headache onset. This appears to contradict 2 recent studies by Perry et al2 and Backes et al3 that found 100% sensitivity of CT within 6 hours. However, these results are not contradictory.
        • Full-Text
        • PDF