Improving patient safety during introduction of novel medical devices through cumulative summation analysis

Restricted access


The aim of this study was to implement cumulative summation (CUSUM) analysis as an early-warning detection and quality assurance system for preclinical testing of the iSYS1 novel robotic trajectory guidance system.


Anatomically accurate 3D-printed skull phantoms were created for 3 patients who underwent implantation of 21 stereoelectroencephalography electrodes by surgeons using the current standard of care (frameless technique). Implantation schema were recreated using the iSYS1 system, and paired accuracy measures were compared with the previous frameless implantations. Entry point, target point, and implantation angle accuracy were measured on postimplantation CT scans. CUSUM analysis was undertaken prospectively.


The iSYS1 trajectory guidance system significantly improved electrode entry point accuracies from 1.90 ± 0.96 mm (mean ± SD) to 0.76 ± 0.57 mm (mean ± SD) without increasing implantation risk. CUSUM analysis was successful as a continuous measure of surgical performance and acted as an early-warning detection system. The surgical learning curve, although minimal, showed improvement after insertion of the eighth electrode.


The iSYS1 trajectory guidance system did not show any increased risk during phantom preclinical testing when used by neurosurgeons who had no experience with its use. CUSUM analysis is a simple technique that can be applied to all stages of the IDEAL (idea, development, exploration, assessment) framework as an extra patient safety mechanism. Further clinical trials are required to prove the efficacy of the device.

ABBREVIATIONS CUSUM = cumulative summation; IDEAL = idea, development, exploration, assessment, long-term follow-up; SEEG = stereoelectroencephalography.

Article Information

Correspondence Vejay N. Vakharia: University College London, London, United Kingdom.

INCLUDE WHEN CITING Published online February 16, 2018; DOI: 10.3171/2017.8.JNS17936.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

© AANS, except where prohibited by US copyright law.



  • View in gallery

    Photograph of implantation using the iSYS1 robotic trajectory guidance system on a phantom skull created to replicate SEEG implantation in patients. Figure is available in color online only.

  • View in gallery

    Schematic of implantation accuracy metrics including entry point, projected target point, and angle error to skull. The solid line (diamond) indicates the planned electrode and the dashed line (circle) indicates the bolt axis trajectory. Entry point (a) and projected target point (b) error were measured as lateral deviation from the plan.

  • View in gallery

    A: Comparison of entry point, target point, and entry angle deviations from plan with manual patient and iSYS1 phantom implantations. B: Suggested grading system for clinical relevance of implantation error and proportion of manual patient and iSYS1 phantom electrodes within error tolerances. EP = entry point; TP = target point. Figure is available in color online only.

  • View in gallery

    CUSUM analyses for entry point error (A), target point error (B), and angle error (C). A negative gradient implies that the intervention (iSYS1) is more beneficial than the control (frameless) implantation. Any change to a positive gradient should alert investigators to an increase in potential risk of the implantation (as can be seen between electrodes 6 and 8 in panel B). The end of the learning curve is taken as the point where the positive gradient becomes negative or where the gradient becomes most negative. In panel A, the learning curve for entry point error becomes most negative after the implantation of electrode 8. In panel B, the target point error becomes negative after the implantation of electrode 9. In panel C, the angle error learning curve becomes negative after the implantation of electrode 12, but there is poor correlation between the intervention and angle error. Figure is available in color online only.



Balanescu BFranklin RCiurea JMindruta IRasina ABobulescu RC: A personalized stereotactic fixture for implantation of depth electrodes in stereoelectroencephalography. Stereotact Funct Neurosurg 92:1171252014


Balsyte DSchäffer LBurkhardt TWisser JZimmermann RKurmanavicius J: Continuous independent quality control for fetal ultrasound biometry provided by the cumulative summation technique. Ultrasound Obstet Gynecol 35:4494552010


Cardinale FCossu MCastana LCasaceli GSchiariti MPMiserocchi A: Stereoelectroencephalography: surgical methodology, safety, and stereotactic application accuracy in 500 procedures. Neurosurgery 72:3533662013


Collins JWTyritzis SNyberg TSchumacher MCLaurin OAdding C: Robot-assisted radical cystectomy (RARC) with intracorporeal neobladder - what is the effect of the learning curve on outcomes? BJU Int 113:1001072014


Day CSPark DJRozenshteyn FSOwusu-Sarpong NGonzalez A: Analysis of FDA-approved orthopaedic devices and their recalls. J Bone Joint Surg Am 98:5175242016


Díaz CEFernández RArmada MGarcía F: A research review on clinical needs, technical requirements, and normativity in the design of surgical robots. Int J Med Robot. 2017


Dorfer CMinchev GCzech TStefanits HFeucht MPataraia E: A novel miniature robotic device for frameless implantation of depth electrodes in refractory epilepsy. J Neurosurg 126:162216282017


Guend HWidmar MPatel SNash GMPaty PBGuillem JG: Developing a robotic colorectal cancer surgery program: understanding institutional and individual learning curves. Surg Endosc 31:282028282017


Kim HJLee SHChang BSLee CKLim TOHoo LP: Monitoring the quality of robot-assisted pedicle screw fixation in the lumbar spine by using a cumulative summation test. Spine (Phila Pa 1976) 40:87942015


Marcus HJPayne CJHughes-Hallett AMarcus APYang GZDarzi A: Regulatory approval of new medical devices: cross sectional study. BMJ 353:i25872016


McCulloch PCook JAAltman DGHeneghan CDiener MK: IDEAL framework for surgical innovation 1: the idea and development stages. BMJ 346:f30122013


Minchev GKronreif GMartínez-Moreno MDorfer CMicko AMert A: A novel miniature robotic guidance device for stereotactic neurosurgical interventions: preliminary experience with the iSYS1 robot. J Neurosurg 126:9859962017


Mullin JPShriver MAlomar SNajm IBulacio JChauvel P: Is SEEG safe? A systematic review and meta-analysis of stereo-electroencephalography-related complications. Epilepsia 57:3864012016


Mullin JPSmithason SGonzalez-Martinez J: Stereo-electro-encephalo-graphy (SEEG) with robotic assistance in the presurgical evaluation of medical refractory epilepsy: a technical note. J Vis Exp (112): 2016


Novara GCatto JWFWilson TAnnerstedt MChan KMurphy DG: Systematic review and cumulative analysis of perioperative outcomes and complications after robot-assisted radical cystectomy. Eur Urol 67:3764012015


Nowell MRodionov RDiehl BWehner TZombori GKinghorn J: A novel method for implementation of frameless stereoEEG in epilepsy surgery. Neurosurgery 10 (Suppl 4):5255342014


Nowell MSparks RZombori GMiserocchi ARodionov RDiehl B: Comparison of computer-assisted planning and manual planning for depth electrode implantations in epilepsy. J Neurosurg 124:182018282016


Rodionov RVollmar CNowell MMiserocchi AWehner TMicallef C: Feasibility of multimodal 3D neuroimaging to guide implantation of intracranial EEG electrodes. Epilepsy Res 107:911002013


Serletis DBulacio JBingaman WNajm IGonzález-Martínez J: The stereotactic approach for mapping epileptic networks: a prospective study of 200 patients. J Neurosurg 121:123912462014


Sivaraman ASanchez-Salas RPrapotnich DYu KOlivier FSecin FP: Learning curve of minimally invasive radical prostatectomy: comprehensive evaluation and cumulative summation analysis of oncological outcomes. Urol Oncol 35:149.e1149.e62017


Sood AGhani KRAhlawat RModi PAbaza RJeong W: Application of the statistical process control method for prospective patient safety monitoring during the learning phase: robotic kidney transplantation with regional hypothermia (IDEAL phase 2a-b). Eur Urol 66:3713782014


Sparks RZombori GRodionov RNowell MVos SBZuluaga MA: Automated multiple trajectory planning algorithm for the placement of stereo-electroencephalography (SEEG) electrodes in epilepsy treatment. Int J CARS 12:1231362017


Vakharia VNSparks RO’Keeffe AGRodionov RMiserocchi AMcEvoy A: Accuracy of intracranial electrode placement for stereoencephalography: a systematic review and meta-analysis. Epilepsia 58:9219322017


Wang MMeng LCai YLi YWang XZhang Z: Learning curve for laparoscopic pancreaticoduodenectomy: a CUSUM analysis. J Gastrointest Surg 20:9249352016


Watson GJByth Kda Cruz M: Outcomes in stapedotomy surgery: the learning curve redefined. Otol Neurotol 36:160116032015


Young AMiller JPAzarow K: Establishing learning curves for surgical residents using cumulative summation (CUSUM) analysis. Curr Surg 62:3303342005


Zuluaga MARodionov RNowell MAchhala SZombori GMendelson AF: Stability, structure and scale: improvements in multi-modal vessel extraction for SEEG trajectory planning. Int J CARS 10:122712372015




All Time Past Year Past 30 Days
Abstract Views 74 74 28
Full Text Views 62 62 24
PDF Downloads 55 55 18
EPUB Downloads 0 0 0


Google Scholar