Real-time navigation guidance with intraoperative CT imaging for pedicle screw placement using an augmented reality head-mounted display: a proof-of-concept study

View More View Less
  • 1 Pickup Family Neurosciences Institute, Hoag Memorial Hospital Presbyterian Newport Beach; and
  • | 2 Disc Comfort, Inc., Newport Beach, California;
  • | 3 Neuroscience Institute, Hackensack Meridian JFK Medical Center, Edison; and
  • | 4 Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey;
  • | 5 Department of Orthopedic Surgery, Columbia University; and
  • | 6 Department of Neurological Surgery, NewYork-Presbyterian/Allen Hospital, New York, New York;
  • | 7 George Washington University School of Medicine, Washington, DC; and
  • | 8 Surgical Theater, Inc., Cleveland, Ohio
Free access

OBJECTIVE

Augmented reality (AR) has the potential to improve the accuracy and efficiency of instrumentation placement in spinal fusion surgery, increasing patient safety and outcomes, optimizing ergonomics in the surgical suite, and ultimately lowering procedural costs. The authors sought to describe the use of a commercial prototype Spine AR platform (SpineAR) that provides a commercial AR head-mounted display (ARHMD) user interface for navigation-guided spine surgery incorporating real-time navigation images from intraoperative imaging with a 3D-reconstructed model in the surgeon's field of view, and to assess screw placement accuracy via this method.

METHODS

Pedicle screw placement accuracy was assessed and compared with literature-reported data of the freehand (FH) technique. Accuracy with SpineAR was also compared between participants of varying spine surgical experience. Eleven operators without prior experience with AR-assisted pedicle screw placement took part in the study: 5 attending neurosurgeons and 6 trainees (1 neurosurgical fellow, 1 senior orthopedic resident, 3 neurosurgical residents, and 1 medical student). Commercially available 3D-printed lumbar spine models were utilized as surrogates of human anatomy. Among the operators, a total of 192 screws were instrumented bilaterally from L2–5 using SpineAR in 24 lumbar spine models. All but one trainee also inserted 8 screws using the FH method. In addition to accuracy scoring using the Gertzbein-Robbins grading scale, axial trajectory was assessed, and user feedback on experience with SpineAR was collected.

RESULTS

Based on the Gertzbein-Robbins grading scale, the overall screw placement accuracy using SpineAR among all users was 98.4% (192 screws). Accuracy for attendings and trainees was 99.1% (112 screws) and 97.5% (80 screws), respectively. Accuracy rates were higher compared with literature-reported lumbar screw placement accuracy using FH for attendings (99.1% vs 94.32%; p = 0.0212) and all users (98.4% vs 94.32%; p = 0.0099). The percentage of total inserted screws with a minimum of 5° medial angulation was 100%. No differences were observed between attendings and trainees or between the two methods. User feedback on SpineAR was generally positive.

CONCLUSIONS

Screw placement was feasible and accurate using SpineAR, an ARHMD platform with real-time navigation guidance that provided a favorable surgeon-user experience.

ABBREVIATIONS

AR = augmented reality; ARHMD = AR head-mounted display; DRF = dynamic reference frame; FH = freehand; SpineAR = spine AR platform; UEQ = user experience questionnaire.

OBJECTIVE

Augmented reality (AR) has the potential to improve the accuracy and efficiency of instrumentation placement in spinal fusion surgery, increasing patient safety and outcomes, optimizing ergonomics in the surgical suite, and ultimately lowering procedural costs. The authors sought to describe the use of a commercial prototype Spine AR platform (SpineAR) that provides a commercial AR head-mounted display (ARHMD) user interface for navigation-guided spine surgery incorporating real-time navigation images from intraoperative imaging with a 3D-reconstructed model in the surgeon's field of view, and to assess screw placement accuracy via this method.

METHODS

Pedicle screw placement accuracy was assessed and compared with literature-reported data of the freehand (FH) technique. Accuracy with SpineAR was also compared between participants of varying spine surgical experience. Eleven operators without prior experience with AR-assisted pedicle screw placement took part in the study: 5 attending neurosurgeons and 6 trainees (1 neurosurgical fellow, 1 senior orthopedic resident, 3 neurosurgical residents, and 1 medical student). Commercially available 3D-printed lumbar spine models were utilized as surrogates of human anatomy. Among the operators, a total of 192 screws were instrumented bilaterally from L2–5 using SpineAR in 24 lumbar spine models. All but one trainee also inserted 8 screws using the FH method. In addition to accuracy scoring using the Gertzbein-Robbins grading scale, axial trajectory was assessed, and user feedback on experience with SpineAR was collected.

RESULTS

Based on the Gertzbein-Robbins grading scale, the overall screw placement accuracy using SpineAR among all users was 98.4% (192 screws). Accuracy for attendings and trainees was 99.1% (112 screws) and 97.5% (80 screws), respectively. Accuracy rates were higher compared with literature-reported lumbar screw placement accuracy using FH for attendings (99.1% vs 94.32%; p = 0.0212) and all users (98.4% vs 94.32%; p = 0.0099). The percentage of total inserted screws with a minimum of 5° medial angulation was 100%. No differences were observed between attendings and trainees or between the two methods. User feedback on SpineAR was generally positive.

CONCLUSIONS

Screw placement was feasible and accurate using SpineAR, an ARHMD platform with real-time navigation guidance that provided a favorable surgeon-user experience.

Pedicle screws are routinely used in spinal fusion procedures to treat patients with spinal degeneration, instability, or deformity. Accurate screw placement is critical for avoiding complications such as dural tears, neurological injury from nerve root or spinal cord impingement, and vascular injury. However, reported accuracy of the conventional freehand (FH) method of pedicle screw placement based on anatomical landmarks varies widely, ranging from 27.6% to 96.5% in the thoracic spine, and 60% to 97.5% in the lumbar spine.1 The variability in reported accuracy rates may be attributed to inconsistencies in experience levels and training using the FH method. As a result, various image-guided techniques have been developed to minimize screw misplacement. Specifically, computer-assisted navigation has been shown to improve screw placement accuracy compared with the FH technique.1–4 Navigation-guided screw placement has also resulted in fewer postoperative revision surgeries than those performed FH.5 Furthermore, combining intraoperative 3D imaging with navigation has been shown to significantly increase placement accuracy compared with the FH technique, 2D navigation, or preoperative CT navigation.6–8 However, navigation-guided techniques are limited by the surgeon's need to direct attention away from the operative field to an external navigation screen for guidance, which hampers workflow and strains ergonomics. In fact, attention shift, or context switching, has been associated with increased cognitive load and performance error.9,10

Augmented reality (AR) allows projection of virtual 3D images onto the user's real-time view of the surgical field and is a rapidly emerging technology in spine surgery.11,12 While current research on AR-guided navigation for spine surgery remains limited, several studies have explored the application of AR for pedicle screw fixation with promising results.13–25 A systematic review found that current AR platforms for navigated spine surgery yielded superior workflow and noninferior accuracy when compared with the conventional FH method.23 All studies involving monitor-based AR platforms that display a video feed of the surgical field with the AR overlay on an external monitor reported favorable results for pedicle screw placement accuracy with AR. Notably, a prospective clinical trial of 20 patients reported significantly higher clinical accuracy of a video-based AR system providing navigation guidance with intraoperative 3D imaging compared with the FH method (93.9% vs 89.6%; p < 0.05).13 While superior accuracy was achieved and real-time operative guidance was available without the increased radiation exposure of fluoroscopy, the surgeon's attention required diversion to an external screen for guidance. An AR head-mounted display (ARHMD) has the potential to help surgeons maintain focus on the operative field by eliminating the need for context switching back and forth to a monitor, and to improve ergonomics, while increasing surgical efficiency and accuracy.

We evaluated a commercial prototype of a Spine AR platform (SpineAR) that utilizes an ARHMD-based user interface for navigation-guided spine fusion surgery using real-time navigation images from intraoperative imaging displayed in 3D in the surgeon's field of view. SpineAR utilizes the Surgical Navigation Advanced Platform (Surgical Theater, Inc.) integrated with a standard spinal navigation system and a commercially available ARHMD to guide pedicle screw placement and other procedures (Fig. 1A). SpineAR augments the patient-specific 360° AR model generated from volumetric imaging along with a customizable floating DICOM window displaying any combination of planes onto the user's field of view to provide intraoperative guidance without workflow disruption. We hypothesized that the comprehensive display of vertebral 2D and 3D navigation data in the user's field of view may enhance intraoperative guidance for screw placement accuracy.

FIG. 1.
FIG. 1.

Laboratory setup. A: SpineAR integrated with standard navigation system and ARHMD. B: A spine model in a foam mold secured with straps to a Jackson flat spine bed. C: Navigation instruments used by operators for screw placement.

Methods

Study Design

In this proof-of-concept study, screw placement accuracy was evaluated in a laboratory setting in which pedicle screws were inserted into commercial 3D-printed lumbar spine models by surgeons and trainees with different levels of experience. Accuracy was assessed and compared with literature-reported data of the FH screw placement technique. Accuracy was also compared between participants of varying spine surgical experience. For trainees, accuracy using the AR platform was compared with that of FH placement.

Eleven participants with no prior experience in AR-assisted pedicle screw placement took part in this multicenter study: 5 attending neurosurgeons (D.S.Y., B.M.O., R.G.L., Y.S., and A.I.) and 6 trainees (1 neurosurgical fellow [R.R.I.], 1 orthopedic surgery resident [V.B.], 3 neurosurgical residents [P.D.P., R.J., and M.C.], and 1 medical student [A.H.R.]). The study took place over 3 days. Each user participated in a 15- to 30-minute training session on the SpineAR platform during which they had the option of placing 2 pedicle screws in a thoracolumbosacral spine model (Synbone). Two attendings and all 6 trainees practiced screw placement. The spine models used for this training session were not included in the analysis for this study. The AR platform evaluated in this study is not yet FDA approved, and IRB approval was not required because phantom models were used.

Twenty-nine models were each instrumented with 8 screws, 5.5 mm in diameter and 40 mm long, from L2 to L5. A total of 232 screws were inserted: 192 using SpineAR and 40 using the FH method. All aspects of the standardized spines and foam molds were kept constant to eliminate variance between participants. Each model was placed on a Jackson flat spine bed during screw placement and secured with straps (Fig. 1B). Table 1 lists the experience level and number of screws placed by each participant.

TABLE 1.

Participant experience level and number of screws placed

Participant No.Surgical ExperienceAR ExperienceNo. of Screws, SpineARNo. of Screws, FH
1Attending, 15 yrsNew user240
2Attending, 13 yrsNew user240
3Attending, 7 yrsNew user320
4Attending, 5 yrsNew user240
5Attending, 4 yrsNew user80
6Neurosurgery fellowNew user248
7Orthopedic surgery resident, PGY-3New user248
8Neurosurgery resident, PGY-6New user80
9Neurosurgery resident, PGY-3New user168
10Neurosurgery resident, PGY-3New user88
11Medical studentNew user88
Total19240

PGY = postgraduate year.

A StealthStation 8 (Medtronic) spine surgical navigation system coupled with an O-arm version 2 (Medtronic) intraoperative scanner was used to initially register each 3D-printed spine model prior to connectivity to the AR platform. Registration reference frames were mounted on each spine via a clamp placed on a thoracic spinous process or via a percutaneous pin. Each spine model underwent 3D volumetric, high-definition CT scans with the scanner set to “small” for patient size and “high density” for the scan. Scan slice thickness was at most 0.83 mm. Previous testing had shown that these settings allowed for the most optimal visualization of the spine models. The volumetric scans were then exported from the scanner and imported into the AR platform for a 360° AR model creation. Briefly, the original scans in DICOM format were rescaled to fit in the software's working intensity range (between 0 and 5000) and then were automatically reconstructed into a volumetric dataset to create a 3D model.26 The transfer functions, based on pixel intensities, were adjusted to highlight the bony anatomy based on the operator's preference. Each 360° AR model construction took approximately 5 minutes. SpineAR retrieved data and coordinates from the navigation system for accurate emulation of tracked instruments in a virtual environment. The virtual reconstructions were then placed within the ARHMD during screw placement (Fig. 2). Figures 2B, 2C, and 3 show examples of what the operator saw through the ARHMD—the 2D DICOM window above the 360° AR model, both positioned based on the operator's preference. The room was kept under low light conditions to minimize any visual cues coming from the models. Figure 3 illustrates the 360° AR model at each stage in the workflow.

FIG. 2.
FIG. 2.

SpineAR visualization deck as seen with ARHMD. A: Operator inserting a pedicle screw. B and C: Examples of visualization displayed in the ARHMD. The navigated instrument is tracked on both the DICOM and the 360° AR model (yellow).

FIG. 3.
FIG. 3.

360° AR model during preinstrumentation, instrumentation with AR guidance, and postinstrumentation. Navigated instruments (e.g., probe [gray], and tap [yellow]) are tracked during instrumentation.

Pedicle Screw Placement Workflow

Following registration, each participant, donning an ARHMD (Magic Leap, Inc.), briefly reviewed the 360° AR model and customized their SpineAR visualization deck—a projection of the 360° AR model along with a DICOM window and integrated tools (e.g., navigation probe, tap, and driver) in the same field of view as the phantom model (Figs. 1C and 2). The opacity of the 360° AR model and the positions of both floating model and DICOM windows were adjusted based on the user's preference. The DICOM window was customized to display any combination of planes. Most users chose the axial and sagittal planes. Using a navigated awl sharp tip, each participant physically marked the entry point on each pedicle. A navigated awl-tip tap was then used to prepare the pedicles for screw placement. Lastly, navigated 5.5-mm drivers with attached 5.5-mm-diameter × 40-mm-long screws were used to complete placement for each pair of pedicles from L2 to L5. After placement of all screws, postinstrumentation CT scans were obtained of each model using the O-arm.

User Experience Questionnaire

Each participant provided feedback on their experience with the AR platform by answering 5- and 7-point Likert scale questions after they completed the tasks. The 26 7-point Likert scale questions were from a published user experience questionnaire (UEQ; https://www.ueq-online.org/).27 Using the UEQ data analysis tool (version 8), user feedback on the 7-point Likert scale questions was rescaled to the range of −3 (completely agree with negative term) to +3 (completely agree with positive term). The 26 user experience measures were grouped into 6 categories: attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty. The mean scores from this study were compared with existing results from a benchmark data set containing data from 20,190 participants from 452 studies on different products. The comparisons were then graded on a scale from bad ("The evaluated product is among the worst 25% of results") to excellent ("The evaluated product is among the best 10% of results").

Accuracy Grading

Accuracy of the postinstrumentation O-arm CT scans was assessed by one independent attending neuroradiologist reviewer using the Gertzbein-Robbins scale of 2-mm increments, as outlined in Table 2.28 Grades 0 and 1 were considered accurate. Pedicle widths were correlated with each assigned grade. Axial trajectory was also assessed as an additional parameter. Optimal axial trajectory of each screw was set to be a minimum of 5° medial angulation. To eliminate grading bias, the type of surgical technique and participant were blinded to the reviewer.

TABLE 2.

The Gertzbein-Robbins grading scale

GradeDescription
0Screw w/in the pedicle, no cortical breach
10- to 2-mm minor cortical breach
2>2- to 4-mm moderate cortical breach
3>4-mm severe cortical breach

Accuracy grading for pedicle screw placement based on that provided in Gertzbein SD, Robbins SE. Accuracy of pedicular screw placement in vivo. Spine (Phila Pa 1976). 1990;15(1):11-14.

Statistical Analysis

Descriptive summary statistics were expressed as frequency for each grade and percentage accuracy. Using the Prism version 8 (GraphPad) software, Fisher's exact test was performed to evaluate SpineAR accuracy compared with a historical control (94.32%, n = 7431; patient and cadaver2) that used the FH method, and variances between participants based on surgical experience; p < 0.05 was considered statistically significant.

Results

Screw Placement Accuracy

Based on the Gertzbein-Robbins grading scale, the overall screw placement accuracy using SpineAR among all users was 98.4% (192 screws; Table 3). Accuracy for attendings and trainees was 99.1% (112 screws) and 97.5% (80 screws), respectively, and did not significantly differ (p = 0.2346). The accuracy with SpineAR for all groups was similar to the accuracy for the FH technique (100%; 40 screws). Compared with literature-reported lumbar screw placement accuracy using FH (94.32%, n = 7431; FH pedicle screw placement in patients and cadavers2), statistical significance was observed for SpineAR accuracy rates of attendings (p = 0.0212) and all users (p = 0.0099). The percentage of total inserted screws that met the criteria for appropriate axial trajectory was 100% for all groups (Table 3). Of note, the study was not adequately powered for statistical comparisons regarding the FH technique.

TABLE 3.

Screw placement accuracy according to the Gertzbein-Robbins grading scale

Gertzbein-Robbins GradeTotal No. of ScrewsAccuracyp ValueAxial Trajectory*
0123
FH
 Trainees3730040100.0%>0.55, vs all groups100%
SpineAR
 Attendings10651011299.1%0.5718, vs trainees100%
0.0212, vs HC
 Trainees6810208097.5%0.3269, vs HC100%
 All users174153019298.4%0.0099, vs HC100%

HC = historical control.

Axial trajectory is presented as the percentage of screws that had a minimum of 5° medial angulation.

Literature-reported lumbar screw placement accuracy using freehand (94.32%, n = 7431; patient and cadaver2).

User Feedback

All participants agreed that the AR platform was easy to use and was useful in guiding screw placement (Fig. 4A). Nine participants agreed that operating with SpineAR helped them maintain their focus and improved ergonomics. The mean UEQ scores were in the range of the 10% best results of the benchmark dataset in the following categories: attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty (Fig. 4B).

FIG. 4.
FIG. 4.

User feedback. A: Feedback specific to SpineAR. B: The mean UEQ results compared with existing values from a benchmark dataset in 6 categories. Scores ranged from −3 (completely agree with negative term) to +3 (completely agree with positive term). All mean scores were in the “excellent” range, which is interpreted as “in the range of the 10% best results” of the benchmark data set.

Discussion

Use of AR during spinal surgery has the potential to transform the surgical process, enhance safety, help surgeons maintain focus on the surgical field, alleviate surgeon fatigue, speed procedure time, lower occupational exposure to ionizing radiation, increase surgical accuracy, and, ultimately, enhance clinical outcomes.11,12,23 Here, we demonstrated accuracy of pedicle screw placement in lumbar spine models using an AR-based platform that integrates with a standard surgical navigation platform and Magic Leap, a commercially available ARHMD. By rendering voxel information in DICOM format and streaming it to an ARHMD, SpineAR can maintain the high-resolution, high-fidelity information from the DICOM images in the form of an interactive 360° AR model without losing valuable data. Other navigated AR systems use smoothing algorithms that create lighter files to be displayed in the headset but yield images with lower resolution. The surgeon's field of view when donning the ARHMD consists of the 360° AR model floating right above the operative field, and a customizable floating DICOM navigation window to provide live navigation data showing the instrument's position and trajectory. The positions of the 360° AR model and DICOM windows within the ARHMD can be customized to allow for enhanced ergonomics and comfort. The ability to visualize individually tailored pertinent information improves screen management and surgeon comfort without disrupting workflow. Based on surgeon feedback, SpineAR provided high-resolution visualization of navigation data without obscuring the surgical field or distracting the operator.

Most reported ARHMD-based platforms use HoloLens (Microsoft) and rely on direct surface tracking with manual registration.20–22,24,25,29 The surface tracking technique involves images taken of the patient's surface anatomy or exposed spinal anatomy which are algorithmically analyzed to output reference points for instrument tracking via optical systems.30 These images are fused with either preoperative or intraoperative imaging to allow for 3D navigation. While radiation exposure to the patient could potentially be reduced, surgeons looking to perform minimally invasive surgery cannot use this patient tracking method due to the lack of distinguishable bony surface anatomy. In this study, intraoperative O-arm imaging with a dynamic reference frame (DRF) attached to the spinous process was utilized for automatic coregistration. This scan can be the sole source of images to generate the patient-specific 360° AR model or fused to existing preoperative MRI and/or CT scans to provide a more comprehensive visualization of the patient anatomy for minimally invasive surgery or more complex procedures. While another HoloLens-based platform used DRF for patient tracking, fluoroscopy was utilized for the intraoperative imaging technique, which the authors reported as time consuming.25 Furthermore, occupational radiation exposure is higher in fluoroscopy-guided procedures than in CT-guided procedures.31

Currently, xvision (Augmedics) is a commercially available platform that uses DRF patient tracking with intraoperative CT imaging to provide an ARHMD navigation system with wireless infrared-based optical tracking cameras inside the headset.16–18 A noninferiority trial comparing xvision's accuracy with literature-reported mean accuracy rates for FH, manual computer-navigated, and robot-assisted computer-navigated techniques found thoracolumbar screw placement accuracy of the ARHMD system to be superior to FH and noninferior to the computer-navigated techniques when assessed using the Gertzbein-Robbins grading system in a cadaveric study.16 In a follow-up study, the authors reported an overall clinical accuracy of 99.1% when the ARHMD was utilized for percutaneous pedicle screw insertion.17 Similar to xvision, SpineAR also integrates with navigation allowing for real-time tracking of tools and screw placement. However, SpineAR integrates with standard surgical navigation systems and is compatible with a commercially available ARHMD, which streams the high-resolution interactive 360° AR model along with corresponding 2D DICOM imaging that is customizable. The tip of the integrated tap can be tracked directly or as a projection with visualization of the appropriate trajectory down the bony corridor of the pedicle. As the tap is advanced, the AR model automatically clips to the depth of the tip to show a cross-sectional reconstruction of the relevant anatomy at that level. After pairing the screw in the driver, the screw tip can be navigated and visualized on both the AR model and the DICOM images. The automatic clipping feature allows the user to inspect the 3D anatomy and visualize down the barrel of the pedicle on the AR image as they advance the tap or insert the screw.

Pedicle screw placement accuracy with SpineAR among the attendings (99.1%), trainees (97.5%), and all users (98.4%) was not significantly different (Table 3). Additionally, accuracy rates were higher compared with literature-reported lumbar screw placement accuracy using FH for attendings (99.1% vs 94.32%; p = 0.0212) and all users (98.4% vs 94.32%; p = 0.0099). In a systematic review chosen as the historical control, Aoude et al. found a 94.32% accuracy rate from 7431 screws placed in lumbar pedicles of patients and cadavers.2 Further verification with nonphantom models and a noninferiority trial comparing SpineAR accuracy with accuracy rates of FH and other computer-assisted navigation techniques will be performed in future studies for further validation. As with any new technology, there was a learning curve for new adopters, especially those with limited experience with extended reality environments.12 The participants in this study had no prior experience with AR-assisted pedicle screw placement. However, these findings along with the positive user feedback suggest that SpineAR was easy to learn and that a brief tutorial was adequate for new users to adapt and be comfortable with the technology (Table 3 and Fig. 4). Dennler et al. reported no differences in precision of drilling pilot holes for pedicle screw insertion in phantom models between expert and novice surgeons when using AR, and a higher precision when using AR compared with FH in the novice group.22 This aligns well with our findings, suggesting the potential of AR in decreasing the effect of the user's experience level. Moreover, AR has shown promise as an effective tool for medical training.32 Specifically for pedicle screw placement, residents and fellows who rehearsed the procedure on an AR simulator with haptic feedback showed significant improvement in performance accuracy from practice to test sessions.19

The SpineAR accuracy rates were comparable with those of the FH method performed in this study (Table 3). The FH method was limited to trainees because of the simplicity of the model and the size of the screws. The chosen model was favorable for high accuracy despite the experience level of the operator, as shown by the 100% accuracy rates of the FH technique performed by the junior residents and the medical student. This data will be further analyzed in subsequent studies looking at more complex models and instrumentation. In this proof-of-concept study, the phantom models chosen did not include surrounding soft-tissue structures and neural elements, which can negatively affect registration and navigation accuracy. Screw placement was performed in the lumbar region instead of the thoracic region, which has smaller pedicles increasing difficulty of accurate screw placement. In addition to assessing screw placement accuracy using the Gertzbein-Robbins grading scale, we also evaluated axial trajectory. Nevertheless, screw placement should be expanded to include the whole thoracolumbar region. The average pedicle width of the Synbone models ranged from approximately 10 mm for L2 to 18 mm for L5 based on measurements performed on 10 of the 29 models. Since 6.5 mm is the more standard diameter in lumbar screw placement studies, utilization of 5.5-mm screws may provide a slight bias toward lessened cortical breaches by the criteria. We were also not able to evaluate screw length because only 40-mm screws were available. Screw placement time should also be analyzed after a learning curve. Additionally, the study was not adequately powered to show superiority over the FH method for the chosen model as it was not the intent of this preliminary study. Subsequent testing will involve more complex models such as cadavers and degenerate phantom spine models as well as a variability of tools (i.e., longer screws with a larger diameter), and SpineAR accuracy will be compared with that of the FH technique for trainees and attendings. Future studies will also compare different navigation systems and ARHMDs to confirm compatibility as well as evaluate efficacy in cadaveric models and subsequently patients. While the focus here is pedicle screw placement, future applications of SpineAR should be expanded to a wider scope of procedures including tumor resection and minimally invasive spinal surgery.

Conclusions

Findings from this preliminary study demonstrate that screw placement is viable with AR technology in a phantom spine model. Various ARHMD-based platforms such as SpineAR offer promise in accessing critical localizing information by displaying the AR image in the surgeon's field of view in real time rather than on a distant monitor. Combining the advantage of navigation with the 3D awareness of the 360° AR model may provide enhanced intraoperative guidance, which can potentially improve surgical safety, efficiency, ergonomics, costs, and patient outcomes.

Acknowledgments

We thank Alex Smith, Nick Quon, Stefannie Moak, Patrick ONeal, Alex Yefimov, Dennis Roy, Daniela Barbery, and Sukhpal Singh for their assistance with the laboratory study as well as Dr. Jason Handwerker for his work on the radiological evaluation.

Disclosures

Surgical Theater provided support for the laboratory study, which included financial support for the Synbone models, aid in the design and conduct of the study, and help with manuscript preparation. Dr. Yanni, Dr. Louis, and Dr. Ozgur are consultants for Surgical Theater. Dr. Yanni is also a consultant for Zimmer Biomet, Spinal Elements, and Cerenovus. Dr. Louis is also a consultant for Medtronic Karl Storz, BehaVR, and XR Health. Dr. Herur-Raman is a past employee of Surgical Theater. Dr. Dang is a Surgical Theater employee. Dr. Goldstein is a consultant for Zimmer Biomet Spine, Alphatec Spine, and Globus Medical; and reports receiving non–study-related clinical or research effort from AbbVie and In Vivo Therapeutics. Dr. Lenke reports personal fees from Medtronic, non-financial support from Broadwater, grants and non-financial support from Scoliosis Research Society, grants from EOS, grants from Setting Scoliosis Straight Foundation, other from Evans Family Donation, other from Fox Family Foundation, grants and non-financial support from AOSpine, personal fees from Abryx, personal fees from EOS Technologies, personal fees from Acuity Surgical.

Author Contributions

Conception and design: Yanni, Ozgur, Louis, Dang, Lenke. Acquisition of data: Yanni, Ozgur, Louis, Shekhtman, RR Iyer, Boddapati, A Iyer, Patel, Jani, Cummock, Herur-Raman. Analysis and interpretation of data: Yanni, Ozgur, Louis, Lenke. Drafting the article: Yanni, Ozgur, Louis, Dang. Critically revising the article: Yanni, Ozgur, Louis, Brant-Zawadzki, Lenke. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Yanni. Administrative/technical/material support: Yanni, Brant-Zawadzki, Steineke, Lenke. Study supervision: Steineke, Lenke.

References

  • 1

    Kosmopoulos V, Schizas C. Pedicle screw placement accuracy: a meta-analysis. Spine (Phila Pa 1976). 2007;32(3):E111E120.

  • 2

    Aoude AA, Fortin M, Figueiredo R, Jarzem P, Ouellet J, Weber MH. Methods to determine pedicle screw placement accuracy in spine surgery: a systematic review. Eur Spine J. 2015;24(5):9901004.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Tian NF, Huang QS, Zhou P, Zhou Y, Wu RK, Lou Y, Xu HZ. Pedicle screw insertion accuracy with different assisted methods: a systematic review and meta-analysis of comparative studies. Eur Spine J. 2011;20(6):846859.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Perdomo-Pantoja A, Ishida W, Zygourakis C, Holmes C, Iyer RR, Cottrill E, et al. Accuracy of current techniques for placement of pedicle screws in the spine: a comprehensive systematic review and meta-analysis of 51,161 screws. World Neurosurg. 2019;126:664678.e3.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Fichtner J, Hofmann N, Rienmüller A, Buchmann N, Gempt J, Kirschke JS, et al. Revision rate of misplaced pedicle screws of the thoracolumbar spine-comparison of three-dimensional fluoroscopy navigation with freehand placement: a systematic analysis and review of the literature. World Neurosurg. 2018;109:e24e32x.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Zhang W, Takigawa T, Wu Y, Sugimoto Y, Tanaka M, Ozaki T. Accuracy of pedicle screw insertion in posterior scoliosis surgery: a comparison between intraoperative navigation and preoperative navigation techniques. Eur Spine J. 2017;26(6):17561764.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7

    Liu H, Chen W, Liu T, Meng B, Yang H. Accuracy of pedicle screw placement based on preoperative computed tomography versus intraoperative data set acquisition for spinal navigation system. J Orthop Surg (Hong Kong). 2017;25(2):2309499017718901.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Peh S, Chatterjea A, Pfarr J, Schäfer JP, Weuster M, Klüter T, et al. Accuracy of augmented reality surgical navigation for minimally invasive pedicle screw insertion in the thoracic and lumbar spine with a new tracking device. Spine J. 2020;20(4):629637.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Gabbard JL, Mehra DG, Swan JE. Effects of AR display context switching and focal distance switching on human performance. IEEE Trans Vis Comput Graph. 2019;25(6):22282241.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Herrlich M, Tavakol P, Black D, Wenig D, Rieder C, Malaka R, Kikinis R. Instrument-mounted displays for reducing cognitive load during surgical navigation. Int J CARS. 2017;12(9):15991605.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Yoo JS, Patel DS, Hrynewycz NM, Brundage TS, Singh K. The utility of virtual reality and augmented reality in spine surgery. Ann Transl Med. 2019;7(5)(suppl 5):S171.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Sakai D, Joyce K, Sugimoto M, Horikita N, Hiyama A, Sato M, et al. Augmented, virtual and mixed reality in spinal surgery: a real-world experience. J Orthop Surg (Hong Kong). 2020;28(3):2309499020952698.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13

    Elmi-Terander A, Burström G, Nachabé R, Fagerlund M, Ståhl F, Charalampidis A, et al. Augmented reality navigation with intraoperative 3D imaging vs fluoroscopy-assisted free-hand surgery for spine fixation surgery: a matched-control study comparing accuracy. Sci Rep. 2020;10(1):707.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Elmi-Terander A, Burström G, Nachabe R, Skulason H, Pedersen K, Fagerlund M, et al. Pedicle screw placement using augmented reality surgical navigation with intraoperative 3D imaging: a first in-human prospective cohort study. Spine (Phila Pa 1976). 2019;44(7):517525.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15

    Elmi-Terander A, Skulason H, Söderman M, Racadio J, Homan R, Babic D, et al. Surgical navigation technology based on augmented reality and integrated 3D intraoperative imaging: a spine cadaveric feasibility and accuracy study. Spine (Phila Pa 1976). 2016;41(21):E1303E1311.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16

    Molina CA, Theodore N, Ahmed AK, Westbroek EM, Mirovsky Y, Harel R, et al. Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study. J Neurosurg Spine. 2019;31(1):139146.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17

    Molina CA, Phillips FM, Colman MW, Ray WZ, Khan M, Orru’ E, et al. A cadaveric precision and accuracy analysis of augmented reality-mediated percutaneous pedicle implant insertion. J Neurosurg Spine. 2021;34(2):316324.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18

    Molina CA, Sciubba DM, Greenberg JK, Khan M, Witham T. Clinical accuracy, technical precision, and workflow of the first in human use of an augmented-reality head-mounted display stereotactic navigation system for spine surgery. Oper Neurosurg (Hagerstown). 2021;20(3):300309.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19

    Luciano CJ, Banerjee PP, Bellotte B, Oh GM, Lemole M Jr, Charbel FT, Roitberg B. Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery. 2011;69(1 Suppl Operative):ons14ons19.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Gibby JT, Swenson SA, Cvetko S, Rao R, Javan R. Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography. Int J CARS. 2019;14(3):525535.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21

    Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, et al. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J CARS. 2019;14(7):11571165.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22

    Dennler C, Jaberg L, Spirig J, Agten C, Götschi T, Fürnstahl P, Farshad M. Augmented reality-based navigation increases precision of pedicle screw insertion. J Orthop Surg Res. 2020;15(1):174.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Burström G, Persson O, Edström E, Elmi-Terander A. Augmented reality navigation in spine surgery: a systematic review. Acta Neurochir (Wien). 2021;163(3):843852.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Liu H, Wu J, Tang Y, Li H, Wang W, Li C, Zhou Y. Percutaneous placement of lumbar pedicle screws via intraoperative CT image-based augmented reality-guided technology. J Neurosurg Spine. 2020;32(4):542547.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25

    ller F, Roner S, Liebmann F, Spirig JM, Fürnstahl P, Farshad M. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine J. 2020;20(4):621628.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26

    Luccichenti G, Cademartiri F, Pezzella FR, Runza G, Belgrano M, Midiri M, et al. 3D reconstruction techniques made easy: know-how and pictures. Eur Radiol. 2005;15(10):21462156.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Schrepp M, Hinderks A, Thomaschewski J. Construction of a benchmark for the User Experience Questionnaire (UEQ). IJIMAI. 2017;4(4):4044.

  • 28

    Gertzbein SD, Robbins SE. Accuracy of pedicular screw placement in vivo. Spine (Phila Pa 1976). 1990;15(1):1114.

  • 29

    Urakov TM, Wang MY, Levi AD. Workflow caveats in augmented reality-assisted pedicle instrumentation: cadaver lab. World Neurosurg. 2019;126:e1449e1455.

  • 30

    Manni F, Elmi-Terander A, Burström G, Persson O, Edström E, Holthuizen R, et al. Towards optical imaging for spine tracking without markers in navigated spine surgery. Sensors (Basel). 2020;20(13):3641.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31

    Dietrich TJ, Peterson CK, Zeimpekis KG, Bensler S, Sutter R, Pfirrmann CWA. Fluoroscopy-guided versus CT-guided lumbar steroid injections: comparison of radiation exposure and outcomes. Radiology. 2019;290(3):752759.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32

    Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc. 2016;30(10):41744183.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Laboratory setup. A: SpineAR integrated with standard navigation system and ARHMD. B: A spine model in a foam mold secured with straps to a Jackson flat spine bed. C: Navigation instruments used by operators for screw placement.

  • View in gallery

    SpineAR visualization deck as seen with ARHMD. A: Operator inserting a pedicle screw. B and C: Examples of visualization displayed in the ARHMD. The navigated instrument is tracked on both the DICOM and the 360° AR model (yellow).

  • View in gallery

    360° AR model during preinstrumentation, instrumentation with AR guidance, and postinstrumentation. Navigated instruments (e.g., probe [gray], and tap [yellow]) are tracked during instrumentation.

  • View in gallery

    User feedback. A: Feedback specific to SpineAR. B: The mean UEQ results compared with existing values from a benchmark dataset in 6 categories. Scores ranged from −3 (completely agree with negative term) to +3 (completely agree with positive term). All mean scores were in the “excellent” range, which is interpreted as “in the range of the 10% best results” of the benchmark data set.

  • 1

    Kosmopoulos V, Schizas C. Pedicle screw placement accuracy: a meta-analysis. Spine (Phila Pa 1976). 2007;32(3):E111E120.

  • 2

    Aoude AA, Fortin M, Figueiredo R, Jarzem P, Ouellet J, Weber MH. Methods to determine pedicle screw placement accuracy in spine surgery: a systematic review. Eur Spine J. 2015;24(5):9901004.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Tian NF, Huang QS, Zhou P, Zhou Y, Wu RK, Lou Y, Xu HZ. Pedicle screw insertion accuracy with different assisted methods: a systematic review and meta-analysis of comparative studies. Eur Spine J. 2011;20(6):846859.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Perdomo-Pantoja A, Ishida W, Zygourakis C, Holmes C, Iyer RR, Cottrill E, et al. Accuracy of current techniques for placement of pedicle screws in the spine: a comprehensive systematic review and meta-analysis of 51,161 screws. World Neurosurg. 2019;126:664678.e3.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Fichtner J, Hofmann N, Rienmüller A, Buchmann N, Gempt J, Kirschke JS, et al. Revision rate of misplaced pedicle screws of the thoracolumbar spine-comparison of three-dimensional fluoroscopy navigation with freehand placement: a systematic analysis and review of the literature. World Neurosurg. 2018;109:e24e32x.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Zhang W, Takigawa T, Wu Y, Sugimoto Y, Tanaka M, Ozaki T. Accuracy of pedicle screw insertion in posterior scoliosis surgery: a comparison between intraoperative navigation and preoperative navigation techniques. Eur Spine J. 2017;26(6):17561764.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7

    Liu H, Chen W, Liu T, Meng B, Yang H. Accuracy of pedicle screw placement based on preoperative computed tomography versus intraoperative data set acquisition for spinal navigation system. J Orthop Surg (Hong Kong). 2017;25(2):2309499017718901.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Peh S, Chatterjea A, Pfarr J, Schäfer JP, Weuster M, Klüter T, et al. Accuracy of augmented reality surgical navigation for minimally invasive pedicle screw insertion in the thoracic and lumbar spine with a new tracking device. Spine J. 2020;20(4):629637.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Gabbard JL, Mehra DG, Swan JE. Effects of AR display context switching and focal distance switching on human performance. IEEE Trans Vis Comput Graph. 2019;25(6):22282241.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Herrlich M, Tavakol P, Black D, Wenig D, Rieder C, Malaka R, Kikinis R. Instrument-mounted displays for reducing cognitive load during surgical navigation. Int J CARS. 2017;12(9):15991605.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Yoo JS, Patel DS, Hrynewycz NM, Brundage TS, Singh K. The utility of virtual reality and augmented reality in spine surgery. Ann Transl Med. 2019;7(5)(suppl 5):S171.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Sakai D, Joyce K, Sugimoto M, Horikita N, Hiyama A, Sato M, et al. Augmented, virtual and mixed reality in spinal surgery: a real-world experience. J Orthop Surg (Hong Kong). 2020;28(3):2309499020952698.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13

    Elmi-Terander A, Burström G, Nachabé R, Fagerlund M, Ståhl F, Charalampidis A, et al. Augmented reality navigation with intraoperative 3D imaging vs fluoroscopy-assisted free-hand surgery for spine fixation surgery: a matched-control study comparing accuracy. Sci Rep. 2020;10(1):707.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Elmi-Terander A, Burström G, Nachabe R, Skulason H, Pedersen K, Fagerlund M, et al. Pedicle screw placement using augmented reality surgical navigation with intraoperative 3D imaging: a first in-human prospective cohort study. Spine (Phila Pa 1976). 2019;44(7):517525.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15

    Elmi-Terander A, Skulason H, Söderman M, Racadio J, Homan R, Babic D, et al. Surgical navigation technology based on augmented reality and integrated 3D intraoperative imaging: a spine cadaveric feasibility and accuracy study. Spine (Phila Pa 1976). 2016;41(21):E1303E1311.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16

    Molina CA, Theodore N, Ahmed AK, Westbroek EM, Mirovsky Y, Harel R, et al. Augmented reality-assisted pedicle screw insertion: a cadaveric proof-of-concept study. J Neurosurg Spine. 2019;31(1):139146.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17

    Molina CA, Phillips FM, Colman MW, Ray WZ, Khan M, Orru’ E, et al. A cadaveric precision and accuracy analysis of augmented reality-mediated percutaneous pedicle implant insertion. J Neurosurg Spine. 2021;34(2):316324.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18

    Molina CA, Sciubba DM, Greenberg JK, Khan M, Witham T. Clinical accuracy, technical precision, and workflow of the first in human use of an augmented-reality head-mounted display stereotactic navigation system for spine surgery. Oper Neurosurg (Hagerstown). 2021;20(3):300309.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19

    Luciano CJ, Banerjee PP, Bellotte B, Oh GM, Lemole M Jr, Charbel FT, Roitberg B. Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery. 2011;69(1 Suppl Operative):ons14ons19.

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Gibby JT, Swenson SA, Cvetko S, Rao R, Javan R. Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography. Int J CARS. 2019;14(3):525535.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21

    Liebmann F, Roner S, von Atzigen M, Scaramuzza D, Sutter R, Snedeker J, et al. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J CARS. 2019;14(7):11571165.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22

    Dennler C, Jaberg L, Spirig J, Agten C, Götschi T, Fürnstahl P, Farshad M. Augmented reality-based navigation increases precision of pedicle screw insertion. J Orthop Surg Res. 2020;15(1):174.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Burström G, Persson O, Edström E, Elmi-Terander A. Augmented reality navigation in spine surgery: a systematic review. Acta Neurochir (Wien). 2021;163(3):843852.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Liu H, Wu J, Tang Y, Li H, Wang W, Li C, Zhou Y. Percutaneous placement of lumbar pedicle screws via intraoperative CT image-based augmented reality-guided technology. J Neurosurg Spine. 2020;32(4):542547.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25

    ller F, Roner S, Liebmann F, Spirig JM, Fürnstahl P, Farshad M. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine J. 2020;20(4):621628.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26

    Luccichenti G, Cademartiri F, Pezzella FR, Runza G, Belgrano M, Midiri M, et al. 3D reconstruction techniques made easy: know-how and pictures. Eur Radiol. 2005;15(10):21462156.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Schrepp M, Hinderks A, Thomaschewski J. Construction of a benchmark for the User Experience Questionnaire (UEQ). IJIMAI. 2017;4(4):4044.

  • 28

    Gertzbein SD, Robbins SE. Accuracy of pedicular screw placement in vivo. Spine (Phila Pa 1976). 1990;15(1):1114.

  • 29

    Urakov TM, Wang MY, Levi AD. Workflow caveats in augmented reality-assisted pedicle instrumentation: cadaver lab. World Neurosurg. 2019;126:e1449e1455.

  • 30

    Manni F, Elmi-Terander A, Burström G, Persson O, Edström E, Holthuizen R, et al. Towards optical imaging for spine tracking without markers in navigated spine surgery. Sensors (Basel). 2020;20(13):3641.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31

    Dietrich TJ, Peterson CK, Zeimpekis KG, Bensler S, Sutter R, Pfirrmann CWA. Fluoroscopy-guided versus CT-guided lumbar steroid injections: comparison of radiation exposure and outcomes. Radiology. 2019;290(3):752759.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32

    Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc. 2016;30(10):41744183.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1736 1090 88
PDF Downloads 1994 1306 70
EPUB Downloads 0 0 0