Interrater and intrarater reliability of the Knosp scale for pituitary adenoma grading

Restricted access

OBJECTIVE

The goal of this study was to determine the interrater and intrarater reliability of the Knosp grading scale for predicting pituitary adenoma cavernous sinus (CS) involvement.

METHODS

Six independent raters (3 neurosurgery residents, 2 pituitary surgeons, and 1 neuroradiologist) participated in the study. Each rater scored 50 unique pituitary MRI scans (with contrast) of biopsy-proven pituitary adenoma. Reliabilities for the full scale were determined 3 ways: 1) using all 50 scans, 2) using scans with midrange scores versus end scores, and 3) using a dichotomized scale that reflects common clinical practice. The performance of resident raters was compared with that of faculty raters to assess the influence of training level on reliability.

RESULTS

Overall, the interrater reliability of the Knosp scale was “strong” (0.73, 95% CI 0.56–0.84). However, the percent agreement for all 6 reviewers was only 10% (26% for faculty members, 30% for residents). The reliability of the middle scores (i.e., average rated Knosp Grades 1 and 2) was “very weak” (0.18, 95% CI −0.27 to 0.56) and the percent agreement for all reviewers was only 5%. When the scale was dichotomized into tumors unlikely to have intraoperative CS involvement (Grades 0, 1, and 2) and those likely to have CS involvement (Grades 3 and 4), the reliability was “strong” (0.60, 95% CI 0.39–0.75) and the percent agreement for all raters improved to 60%. There was no significant difference in reliability between residents and faculty (residents 0.72, 95% CI 0.55–0.83 vs faculty 0.73, 95% CI 0.56–0.84). Intrarater reliability was moderate to strong and increased with the level of experience.

CONCLUSIONS

Although these findings suggest that the Knosp grading scale has acceptable interrater reliability overall, it raises important questions about the “very weak” reliability of the scale's middle grades. By dichotomizing the scale into clinically useful groups, the authors were able to address the poor reliability and percent agreement of the intermediate grades and to isolate the most important grades for use in surgical decision making (Grades 3 and 4). Authors of future pituitary surgery studies should consider reporting Knosp grades as dichotomized results rather than as the full scale to optimize the reliability of the scale.

ABBREVIATIONS CS = cavernous sinus; ICA = internal carotid artery.
Article Information

Contributor Notes

INCLUDE WHEN CITING Published online July 1, 2016; DOI: 10.3171/2016.3.JNS153044.Correspondence Andrew S. Little, c/o Neuroscience Publications, Barrow Neurological Institute, St. Joseph's Hospital and Medical Center, 350 W Thomas Rd., Phoenix, AZ 85013. email: neuropub@dignityhealth.org.Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

© AANS, except where prohibited by US copyright law.

Headings
References
  • 1

    Ahmadi JNorth CMSegall HDZee CSWeiss MH: Cavernous sinus invasion by pituitary adenomas. AJR Am J Roentgenol 146:2572621986

  • 2

    Ames CPSmith JSEastlack RBlaskiewicz DJShaffrey CISchwab F: Reliability assessment of a novel cervical spine deformity classification system. J Neurosurg Spine 23:6736832015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Bonneville JFCattin FRacle ABouchareb MBoulard DPotelon P: Dynamic CT of the laterosellar extradural venous spaces. AJNR Am J Neuroradiol 10:5355421989

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Ceylan SKoc KAnik I: Endoscopic endonasal transsphenoidal approach for pituitary adenomas invading the cavernous sinus. J Neurosurg 112:991072010

  • 5

    Chabot JDChakraborty SImbarrato GDehdashti AR: Evaluation of outcomes after endoscopic endonasal surgery for large and giant pituitary macroadenoma: a retrospective review of 39 consecutive patients. World Neurosurg 84:9789882015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Cicchetti D: The effect of number of rating scale categories on levels of interrater reliability: a Monte Carlo investigation. Appl Psychol Meas 9:31361985

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7

    Cordova JSSchreibmann EHadjipanayis CGGuo YShu HKShim H: Quantitative tumor segmentation for evaluation of extent of glioblastoma resection to facilitate multisite clinical trials. Transl Oncol 7:40472014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8

    Dallapiazza RFGrober YStarke RMLaws ER JrJane JA Jr: Long-term results of endonasal endoscopic transsphenoidal resection of nonfunctioning pituitary macroadenomas. Neurosurgery 76:42532015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Davis PCHoffman JC JrMalko JATindall GTTakei YAvruch L: Gadolinium-DTPA and MR imaging of pituitary adenoma: a preliminary report. AJNR Am J Neuroradiol 8:8178231987

    • Search Google Scholar
    • Export Citation
  • 10

    Dehdashti ARGanna AKarabatsou KGentili F: Pure endoscopic endonasal approach for pituitary adenomas: early surgical results in 200 patients and comparison with previous microsurgical series. Neurosurgery 62:100610172008

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11

    Dickerman RDOldfield EH: Basis of persistent and recurrent Cushing disease: an analysis of findings at repeated pituitary surgery. J Neurosurg 97:134313492002

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Fahlbusch RBuchfelder M: Transsphenoidal surgery of parasellar pituitary adenomas. Acta Neurochir (Wien) 92:93991988

  • 13

    Frisoli FALang SSVossough ACahill AMHeuer GGDahmoush HM: Intrarater and interrater reliability of the pediatric arteriovenous malformation compactness score in children. J Neurosurg Pediatr 11:5475512013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Goel ANadkarni TMuzumdar DDesai KPhalke USharma P: Giant pituitary tumors: a study based on surgical treatment of 118 cases. Surg Neurol 61:4364462004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15

    Gordon ASWestrick ACFalola MIShannon CNWalters BCFisher WS: Reliability of postoperative photographs in assessment of facial nerve function after vestibular schwannoma resection. J Neurosurg 117:8608632012

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16

    Griessenauer CJMiller JHAgee BSFisher WS IIICuré JKChapman PR: Observer reliability of arteriovenous malformations grading scales using current imaging modalities. J Neurosurg 120:117911872014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17

    Harrop JSVaccaro ARHurlbert RJWilsey JTBaron EMShaffrey CI: Intrarater and interrater reliability and validity in the assessment of the mechanism of injury and integrity of the posterior ligamentous complex: a novel injury severity scoring system for thoracolumbar injuries. Invited submission from the Joint Section Meeting On Disorders of the Spine and Peripheral Nerves, March 2005. J Neurosurg Spine 4:1181222006

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18

    Jho HD: Endoscopic transsphenoidal surgery. J Neurooncol 54:1871952001

  • 19

    Jiménez-Roldán LAlén JFGómez PALobato RDRamos AMunarriz PM: Volumetric analysis of subarachnoid hemorrhage: assessment of the reliability of two computerized methods and their comparison with other radiographic scales. J Neurosurg 118:84932013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Kabil MSEby JBShahinian HK: Fully endoscopic endonasal vs. transseptal transsphenoidal pituitary surgery. Minim Invasive Neurosurg 48:3483542005

  • 21

    Knosp EKitz KSteiner EMatula C: Pituitary adenomas with parasellar invasion. Acta Neurochir Suppl (Wien) 53:65711991

  • 22

    Knosp ESteiner EKitz KMatula C: Pituitary adenomas with invasion of the cavernous sinus space: a magnetic resonance imaging classification compared with surgical findings. Neurosurgery 33:6106181993

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Kulkarni AVRiva-Cambrin JBrowd SR: Use of the ETV Success Score to explain the variation in reported endoscopic third ventriculostomy success rates among published case series of childhood hydrocephalus. J Neurosurg Pediatr 7:1431462011

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24

    Li WZhang YZhang MHuang GZhang Q: Wnt4 is over-expressed in human pituitary adenomas and is associated with tumor invasion. J Clin Neurosci 21:1371412014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25

    Micko ASWöhrer AWolfsberger SKnosp E: Invasion of the cavernous sinus space in pituitary adenomas: endoscopic verification and its correlation with an MRI-based classification. J Neurosurg 122:8038112015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26

    Miki YMatsuo MNishizawa SKuroda YKeyaki AMakita Y: Pituitary adenomas and normal pituitary tissue: enhancement patterns on gadopentetate-enhanced MR imaging. Radiology 177:35381990

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Nishioka HFukuhara NHoriguchi KYamada S: Aggressive transsphenoidal resection of tumors invading the cavernous sinus in patients with acromegaly: predictive factors, strategies, and outcomes. J Neurosurg 121:5055102014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 28

    Preston CCColman AM: Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol (Amst) 104:1152000

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 29

    Scotti GYu CYDillon WPNorman DColombo NNewton TH: MR imaging of cavernous sinus involvement by pituitary adenomas. AJR Am J Roentgenol 151:7998061988

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30

    Shou XFLi SQWang YFZhao YJia PFZhou LF: Treatment of pituitary adenomas with a transsphenoidal approach. Neurosurgery 56:2492562005

  • 31

    Steiner EImhof HKnosp E: Gd-DTPA enhanced high resolution MR imaging of pituitary adenomas. Radiographics 9:5875981989

  • 32

    Steiner EKnosp EHerold CJKramer JStiglbauer RStaniszewski K: Pituitary adenomas: findings of postoperative MR imaging. Radiology 185:5215271992

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33

    Thaler MLechner RGstöttner MLuegmair MLiebensteiner MNogler M: Interrater and intrarater reliability of the Kuntz et al new deformity classification system. Neurosurgery 71:47572012

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34

    Vachhrajani SSen ANSatyan KKulkarni AVBirchansky SBJea A: Estimation of normal computed tomography measurements for the upper cervical spine in the pediatric age group. J Neurosurg Pediatr 14:4254332014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 35

    Wu ZBYu CJSu ZPZhuge QCWu JSZheng WM: Bromocriptine treatment of invasive giant prolactinomas involving the cavernous sinus: results of a long-term follow up. J Neurosurg 104:54612006

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 36

    Yamada SFukuhara NHoriguchi KYamaguchi-Okada MNishioka HTakeshita A: Clinicopathological characteristics and therapeutic outcomes in thyrotropin-secreting pituitary adenomas: a single-center study of 90 cases. J Neurosurg 121:146214732014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
TrendMD
Cited By
Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 540 540 28
PDF Downloads 332 332 12
EPUB Downloads 0 0 0
PubMed
Google Scholar