Interobserver variability in the interpretation of computed tomography following aneurysmal subarachnoid hemorrhage

Clinical article

George M. Ibrahim M.D.1,2, Stefan Weidauer M.D.3, and R. Loch Macdonald M.D., Ph.D.1,2
View More View Less
  • 1 Division of Neurosurgery, St. Michael's Hospital, Labatt Family Centre of Excellence in Brain Injury and Trauma Research, Keenan Research Centre of the Li Ka Shing Knowledge Institute of St. Michael's Hospital, Toronto;
  • | 2 Department of Surgery, University of Toronto, Ontario, Canada; and
  • | 3 Institute of Neuroradiology, Johann Wolfgang Goethe-Universität Frankfurt, Germany
Restricted access

Purchase Now

USD  $45.00

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $515.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $612.00
Print or Print + Online

Object

Numerous abnormal findings may be evident on CT scans after aneurysmal subarachnoid hemorrhage (SAH). Here, the authors assess the interobserver variability in the radiological interpretation of the initial CT scan following SAH.

Methods

Two experienced reviewers, a neurosurgeon and a neuroradiologist, independently prospectively reviewed the initial CT scans of 413 patients enrolled in the CONSCIOUS-1 trial. Measured variables included SAH, intraventricular hemorrhage, intracerebral hemorrhage, subdural hematoma, chronic infarction, midline shift, and hydrocephalus. To assess interobserver variability, weighted kappa values and intraclass correlation coefficients (ICCs) were calculated and Bland-Altman analysis was performed.

Results

Moderate to substantial agreement was found for most of the CT scanning findings. There was fair to moderate interobserver agreement between reviewers when determining the extent of SAH based on a descriptive categorical classification (kappa 0.41; 95% CI 0.33–0.49), and better agreement when a semiquantitative scale was used (ICC 0.56; 95% CI 0.49–0.62). There was poor agreement between reviewers for the presence of hydrocephalus (kappa 0.34; 95% CI 0.20–0.48), but substantial to near perfect agreement on ventriculocranial ratio measurements (ICC 0.77; 95% CI 0.72–0.81).

Conclusions

The authors' findings suggest that there is considerable interobserver variability in the interpretation of CT scans after SAH. Quantitative measures may reduce interobserver variability in comparison with qualitative or categorical scales. Variability in interpretation of CT scans has implications for patient care and conduct of clinical trials. It may be beneficial to develop standardized assessments to ensure consistent evaluation of measured variables.

Abbreviations used in this paper:

EDH = epidural hematoma; ICC = intraclass correlation coefficient; ICH = intracerebral hemorrhage; IVH = intraventricular hemorrhage; SAH = subarachnoid hemorrhage; SDH = subdural hematoma; VCR = ventriculocranial ratio.

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $515.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $612.00
  • 1

    Adams HP Jr, , Kassell NF, , Torner JC, & Haley EC Jr: Predicting cerebral ischemia after aneurysmal subarachnoid hemorrhage: influences of clinical condition, CT results, and antifibrinolytic therapy. A report of the Cooperative Aneurysm Study. Neurology 37:15861591, 1987

    • Search Google Scholar
    • Export Citation
  • 2

    Bhattathiri PS, , Gregson B, , Prasad KS, , Mitchell P, , Soh C, & Mitra D, et al.: Reliability assessment of computerized tomography scanning measurements in intracerebral hematoma. Neurosurg Focus 15:4 E6, 2003

    • Search Google Scholar
    • Export Citation
  • 3

    Bland JM, & Altman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307310, 1986

  • 4

    Bullock MR, , Chesnut R, , Ghajar J, , Gordon D, , Hartl R, & Newell DW, et al.: Surgical management of acute subdural hematomas. Neurosurgery 58:3 Suppl S16S24, 2006

    • Search Google Scholar
    • Export Citation
  • 5

    Claassen J, , Bernardini GL, , Kreiter K, , Bates J, , Du YE, & Copeland D, et al.: Effect of cisternal and ventricular blood on risk of delayed cerebral ischemia after subarachnoid hemorrhage: the Fisher scale revisited. Stroke 32:20122020, 2001

    • Search Google Scholar
    • Export Citation
  • 6

    Fisher CM, , Kistler JP, & Davis JM: Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery 6:19, 1980

    • Search Google Scholar
    • Export Citation
  • 7

    Frontera JA, , Claassen J, , Schmidt JM, , Wartenberg KE, , Temes R, & Connolly ES Jr, et al.: Prediction of symptomatic vasospasm after subarachnoid hemorrhage: the modified Fisher scale. Neurosurgery 59:2127, 2006

    • Search Google Scholar
    • Export Citation
  • 8

    Graeb DA, , Robertson WD, , Lapointe JS, , Nugent RA, & Harrison PB: Computed tomographic diagnosis of intraventricular hemorrhage. Etiology and prognosis. Radiology 143:9196, 1982

    • Search Google Scholar
    • Export Citation
  • 9

    Hijdra A, , Brouwers PJ, , Vermeulen M, & van Gijn J: Grading the amount of blood on computed tomograms after subarachnoid hemorrhage. Stroke 21:11561161, 1990

    • Search Google Scholar
    • Export Citation
  • 10

    Kistler JP, , Crowell RM, , Davis KR, , Heros R, , Ojemann RG, & Zervas T, et al.: The relation of cerebral vasospasm to the extent and location of subarachnoid blood visualized by CT scan: a prospective study. Neurology 33:424436, 1983

    • Search Google Scholar
    • Export Citation
  • 11

    Landis JR, & Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:159174, 1977

  • 12

    Leonardi M, , Ferro S, , Agati R, , Fiorani L, , Righini A, & Cristina E, et al.: Interobserver variability in CT assessment of brain atrophy. Neuroradiology 36:1719, 1994

    • Search Google Scholar
    • Export Citation
  • 13

    LeRoux PD, , Haglund MM, , Newell DW, , Grady MS, & Winn HR: Intraventricular hemorrhage in blunt head trauma: an analysis of 43 cases. Neurosurgery 31:678685, 1992

    • Search Google Scholar
    • Export Citation
  • 14

    Macdonald RL, , Kassell NF, , Mayer S, , Ruefenacht D, , Schmiedek P, & Weidauer S, et al.: Clazosentan to overcome neurological ischemia and infarction occurring after subarachnoid hemorrhage (CONSCIOUS-1): randomized, double-blind, placebo-controlled phase 2 dose-finding trial. Stroke 39:30153021, 2008

    • Search Google Scholar
    • Export Citation
  • 15

    Mesiwala AH, , Avellino AM, & Ellenbogen RG: The diagonal ventricular dimension: a method for predicting shunt malfunction on the basis of changes in ventricular size. Neurosurgery 50:12461252, 2002

    • Search Google Scholar
    • Export Citation
  • 16

    Naidich TP, , Schott LH, & Baron RL: Computed tomography in evaluation of hydrocephalus. Radiol Clin North Am 20:143167, 1982

  • 17

    O'Hayon BB, , Drake JM, , Ossip MG, , Tuli S, & Clarke M: Frontal and occipital horn ratio: a linear estimate of ventricular size for multiple imaging modalities in pediatric hydrocephalus. Pediatr Neurosurg 29:245249, 1998

    • Search Google Scholar
    • Export Citation
  • 18

    Shimizu H: Re: Subarachnoid hemorrhage on computed tomography scanning and the development of cerebral vasospasm: the Fisher grade revisited (Smith ML et al. Surg Neurol 2005;63:229–35). Surg Neurol 66:551552, 2006. (Letter)

    • Search Google Scholar
    • Export Citation
  • 19

    Shrout PE, & Fleiss JL: Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420428, 1979

  • 20

    Smith ML, , Abrahams JM, , Chandela S, , Smith MJ, , Hurst RW, & Le Roux PD: Subarachnoid hemorrhage on computed tomography scanning and the development of cerebral vasospasm: the Fisher grade revisited. Surg Neurol 63:229235, 2005

    • Search Google Scholar
    • Export Citation
  • 21

    Svensson E, , Starmark JE, , Ekholm S, , von Essen C, & Johansson A: Analysis of interobserver disagreement in the assessment of subarachnoid blood and acute hydrocephalus on CT scans. Neurol Res 18:487494, 1996

    • Search Google Scholar
    • Export Citation
  • 22

    Sze RW, , Ghioni V, , Weinberger E, , Seidel KD, & Ellenbogen RG: Rapid computed tomography technique to measure ventricular volumes in the child with suspected ventriculoperitoneal shunt failure II. Clinical application. J Comput Assist Tomogr 27:668673, 2003

    • Search Google Scholar
    • Export Citation
  • 23

    van der Jagt M, , Hasan D, , Bijvoet HW, , Pieterman H, , Koudstaal PJ, & Avezaat CJ: Interobserver variability of cisternal blood on CT after aneurysmal subarachnoid hemorrhage. Neurology 54:21562158, 2000

    • Search Google Scholar
    • Export Citation
  • 24

    van der Worp HB, , Claus SP, , Bär PR, , Ramos LM, , Algra A, & van Gijn J, et al.: Reproducibility of measurements of cerebral infarct volume on CT scans. Stroke 32:424430, 2001

    • Search Google Scholar
    • Export Citation
  • 25

    van Gijn J, , Hijdra A, , Wijdicks EF, , Vermeulen M, & van Crevel H: Acute hydrocephalus after aneurysmal subarachnoid hemorrhage. J Neurosurg 63:355362, 1985

    • Search Google Scholar
    • Export Citation
  • 26

    van Norden AG, , van Dijk GW, , van Huizen MD, , Algra A, & Rinkel GJ: Interobserver agreement and predictive value for outcome of two rating scales for the amount of extravasated blood after aneurysmal subarachnoid haemorrhage. J Neurol 253:12171220, 2006

    • Search Google Scholar
    • Export Citation
  • 27

    van Zagten M, , Kessels F, , Boiten J, & Lodder J: Interobserver agreement in the assessment of cerebral atrophy on CT using bicaudate and sylvian-fissure ratios. Neuroradiology 41:261265, 1999

    • Search Google Scholar
    • Export Citation
  • 28

    Vos MJ, , Uitdehaag BM, , Barkhof F, , Heimans JJ, , Baayen HC, & Boogerd W, et al.: Interobserver variability in the radiological assessment of response to chemotherapy in glioma. Neurology 60:826830, 2003

    • Search Google Scholar
    • Export Citation
  • 29

    Woods BT, , Douglass A, & Gescuk B: Is the VBR still a useful measure of changes in the cerebral ventricles?. Psychiatry Res 40:110, 1991

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 658 161 17
Full Text Views 149 12 0
PDF Downloads 139 14 0
EPUB Downloads 0 0 0