Interobserver variability in the interpretation of computed tomography following aneurysmal subarachnoid hemorrhage

Clinical article

Restricted access


Numerous abnormal findings may be evident on CT scans after aneurysmal subarachnoid hemorrhage (SAH). Here, the authors assess the interobserver variability in the radiological interpretation of the initial CT scan following SAH.


Two experienced reviewers, a neurosurgeon and a neuroradiologist, independently prospectively reviewed the initial CT scans of 413 patients enrolled in the CONSCIOUS-1 trial. Measured variables included SAH, intraventricular hemorrhage, intracerebral hemorrhage, subdural hematoma, chronic infarction, midline shift, and hydrocephalus. To assess interobserver variability, weighted kappa values and intraclass correlation coefficients (ICCs) were calculated and Bland-Altman analysis was performed.


Moderate to substantial agreement was found for most of the CT scanning findings. There was fair to moderate interobserver agreement between reviewers when determining the extent of SAH based on a descriptive categorical classification (kappa 0.41; 95% CI 0.33–0.49), and better agreement when a semiquantitative scale was used (ICC 0.56; 95% CI 0.49–0.62). There was poor agreement between reviewers for the presence of hydrocephalus (kappa 0.34; 95% CI 0.20–0.48), but substantial to near perfect agreement on ventriculocranial ratio measurements (ICC 0.77; 95% CI 0.72–0.81).


The authors' findings suggest that there is considerable interobserver variability in the interpretation of CT scans after SAH. Quantitative measures may reduce interobserver variability in comparison with qualitative or categorical scales. Variability in interpretation of CT scans has implications for patient care and conduct of clinical trials. It may be beneficial to develop standardized assessments to ensure consistent evaluation of measured variables.

Abbreviations used in this paper: EDH = epidural hematoma; ICC = intraclass correlation coefficient; ICH = intracerebral hemorrhage; IVH = intraventricular hemorrhage; SAH = subarachnoid hemorrhage; SDH = subdural hematoma; VCR = ventriculocranial ratio.

Article Information

Address correspondence to: R. Loch Macdonald, M.D., Ph.D., St. Michael's Hospital, 30 Bond Street, Toronto, Ontario M5B 1W8, Canada. email:

Please include this information when citing this paper: published online August 5, 2011; DOI: 10.3171/2011.7.JNS11725.

© AANS, except where prohibited by US copyright law.



  • View in gallery

    A: Scatter diagram of Hijdra score for 2 reviewers in 413 patients. B: Bland-Altman plot of interobserver variability in interpretation of subarachnoid clot burden using the Hijdra score. Upper and lower bars represent 95% limits of agreement, and the center bar represents the mean. C and D: Scatter diagram (C) and Bland-Altman analysis (D) for intraventricular clot burden measured using the modified Graeb score. E and F: Scatter diagram (E) and Bland-Altman analysis (F) for hydrocephalus measured using the VCR.



Adams HP JrKassell NFTorner JCHaley EC Jr: Predicting cerebral ischemia after aneurysmal subarachnoid hemorrhage: influences of clinical condition, CT results, and antifibrinolytic therapy. A report of the Cooperative Aneurysm Study. Neurology 37:158615911987


Bhattathiri PSGregson BPrasad KSMitchell PSoh CMitra D: Reliability assessment of computerized tomography scanning measurements in intracerebral hematoma. Neurosurg Focus 15:4E62003


Bland JMAltman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:3073101986


Bullock MRChesnut RGhajar JGordon DHartl RNewell DW: Surgical management of acute subdural hematomas. Neurosurgery 58:3 SupplS16S242006


Claassen JBernardini GLKreiter KBates JDu YECopeland D: Effect of cisternal and ventricular blood on risk of delayed cerebral ischemia after subarachnoid hemorrhage: the Fisher scale revisited. Stroke 32:201220202001


Fisher CMKistler JPDavis JM: Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery 6:191980


Frontera JAClaassen JSchmidt JMWartenberg KETemes RConnolly ES Jr: Prediction of symptomatic vasospasm after subarachnoid hemorrhage: the modified Fisher scale. Neurosurgery 59:21272006


Graeb DARobertson WDLapointe JSNugent RAHarrison PB: Computed tomographic diagnosis of intraventricular hemorrhage. Etiology and prognosis. Radiology 143:91961982


Hijdra ABrouwers PJVermeulen Mvan Gijn J: Grading the amount of blood on computed tomograms after subarachnoid hemorrhage. Stroke 21:115611611990


Kistler JPCrowell RMDavis KRHeros ROjemann RGZervas T: The relation of cerebral vasospasm to the extent and location of subarachnoid blood visualized by CT scan: a prospective study. Neurology 33:4244361983


Landis JRKoch GG: The measurement of observer agreement for categorical data. Biometrics 33:1591741977


Leonardi MFerro SAgati RFiorani LRighini ACristina E: Interobserver variability in CT assessment of brain atrophy. Neuroradiology 36:17191994


LeRoux PDHaglund MMNewell DWGrady MSWinn HR: Intraventricular hemorrhage in blunt head trauma: an analysis of 43 cases. Neurosurgery 31:6786851992


Macdonald RLKassell NFMayer SRuefenacht DSchmiedek PWeidauer S: Clazosentan to overcome neurological ischemia and infarction occurring after subarachnoid hemorrhage (CONSCIOUS-1): randomized, double-blind, placebo-controlled phase 2 dose-finding trial. Stroke 39:301530212008


Mesiwala AHAvellino AMEllenbogen RG: The diagonal ventricular dimension: a method for predicting shunt malfunction on the basis of changes in ventricular size. Neurosurgery 50:124612522002


Naidich TPSchott LHBaron RL: Computed tomography in evaluation of hydrocephalus. Radiol Clin North Am 20:1431671982


O'Hayon BBDrake JMOssip MGTuli SClarke M: Frontal and occipital horn ratio: a linear estimate of ventricular size for multiple imaging modalities in pediatric hydrocephalus. Pediatr Neurosurg 29:2452491998


Shimizu H: Re: Subarachnoid hemorrhage on computed tomography scanning and the development of cerebral vasospasm: the Fisher grade revisited (Smith ML et al. Surg Neurol 2005;63:229–35). Surg Neurol 66:5515522006. (Letter)


Shrout PEFleiss JL: Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:4204281979


Smith MLAbrahams JMChandela SSmith MJHurst RWLe Roux PD: Subarachnoid hemorrhage on computed tomography scanning and the development of cerebral vasospasm: the Fisher grade revisited. Surg Neurol 63:2292352005


Svensson EStarmark JEEkholm Svon Essen CJohansson A: Analysis of interobserver disagreement in the assessment of subarachnoid blood and acute hydrocephalus on CT scans. Neurol Res 18:4874941996


Sze RWGhioni VWeinberger ESeidel KDEllenbogen RG: Rapid computed tomography technique to measure ventricular volumes in the child with suspected ventriculoperitoneal shunt failure II. Clinical application. J Comput Assist Tomogr 27:6686732003


van der Jagt MHasan DBijvoet HWPieterman HKoudstaal PJAvezaat CJ: Interobserver variability of cisternal blood on CT after aneurysmal subarachnoid hemorrhage. Neurology 54:215621582000


van der Worp HBClaus SPBär PRRamos LMAlgra Avan Gijn J: Reproducibility of measurements of cerebral infarct volume on CT scans. Stroke 32:4244302001


van Gijn JHijdra AWijdicks EFVermeulen Mvan Crevel H: Acute hydrocephalus after aneurysmal subarachnoid hemorrhage. J Neurosurg 63:3553621985


van Norden AGvan Dijk GWvan Huizen MDAlgra ARinkel GJ: Interobserver agreement and predictive value for outcome of two rating scales for the amount of extravasated blood after aneurysmal subarachnoid haemorrhage. J Neurol 253:121712202006


van Zagten MKessels FBoiten JLodder J: Interobserver agreement in the assessment of cerebral atrophy on CT using bicaudate and sylvian-fissure ratios. Neuroradiology 41:2612651999


Vos MJUitdehaag BMBarkhof FHeimans JJBaayen HCBoogerd W: Interobserver variability in the radiological assessment of response to chemotherapy in glioma. Neurology 60:8268302003


Woods BTDouglass AGescuk B: Is the VBR still a useful measure of changes in the cerebral ventricles?. Psychiatry Res 40:1101991




All Time Past Year Past 30 Days
Abstract Views 58 58 13
Full Text Views 102 102 10
PDF Downloads 88 88 3
EPUB Downloads 0 0 0


Google Scholar