360° 3D virtual reality operative video for the training of residents in neurosurgery

View More View Less
  • 1 Department of Neurosurgery, Charité–Universitätsmedizin Berlin;
  • | 2 Cluster of Excellence: “Matters of Activity. Image Space Material,” Humboldt University, Berlin;
  • | 3 gamelab.berlin, Cluster of Excellence: “Matters of Activity. Image Space Material,” Humboldt University, Berlin;
  • | 4 Berlin Simulation and Training Center (BeST), Charité–Universitätsmedizin Berlin; and
  • | 5 Berlin Institute of Health, BIH Academy, Clinician Scientist Program, Charité–Universitätsmedizin Berlin, Germany
Free access

OBJECTIVE

Training of residents is an essential but time-consuming and costly task in the surgical disciplines. During the coronavirus disease 2019 pandemic, surgical education became even more challenging because of the reduced caseload due to the increased shift to corona care. In this context, augmented 360° 3D virtual reality (VR) videos of surgical procedures enable effective off-site training through virtual participation in the surgery. The goal of this study was to establish and evaluate 360° 3D VR operative videos for neurosurgical training.

METHODS

Using a 360° camera, the authors recorded three standard neurosurgical procedures: a lumbar discectomy, brain metastasis resection, and clipping of an aneurysm. Combined with the stereoscopic view of the surgical microscope, 7- to 10-minute 360° 3D VR videos augmented with annotations, overlays, and commentary were created. These videos were then presented to the neurosurgical residents at the authors’ institution using a head-mounted display. Before viewing the videos, the residents were asked to fill out a questionnaire indicating their VR experience and self-assessment of surgical skills regarding the specific procedure. After watching the videos, the residents completed another questionnaire to evaluate their quality and usefulness. The parameters were scaled with a 5-point Likert scale.

RESULTS

Twenty-two residents participated in this study. The mean years of experience of the participants in neurosurgery was 3.2 years, ranging from the 1st through the 7th year of training. Most participants (86.4%) had no or less than 15 minutes of VR experience. The overall quality of the videos was rated good to very good. Immersion, the feeling of being in the operating room, was high, and almost all participants (91%) stated that 360° VR videos provide a useful addition to the neurosurgical training. VR sickness was negligible in the cohort.

CONCLUSIONS

In this study, the authors demonstrated the feasibility and high acceptance of augmented 360° 3D VR videos in neurosurgical training. Augmentation of 360° videos with complementary and interactive content has the potential to effectively support trainees in acquiring conceptual knowledge. Further studies are necessary to investigate the effectiveness of their use in improving surgical skills.

ABBREVIATIONS

COVID-19 = coronavirus disease 2019; HMD = head-mounted display; OR = operating room; PGY = postgraduate year; S3D = stereoscopic 3D; VR = virtual reality.

OBJECTIVE

Training of residents is an essential but time-consuming and costly task in the surgical disciplines. During the coronavirus disease 2019 pandemic, surgical education became even more challenging because of the reduced caseload due to the increased shift to corona care. In this context, augmented 360° 3D virtual reality (VR) videos of surgical procedures enable effective off-site training through virtual participation in the surgery. The goal of this study was to establish and evaluate 360° 3D VR operative videos for neurosurgical training.

METHODS

Using a 360° camera, the authors recorded three standard neurosurgical procedures: a lumbar discectomy, brain metastasis resection, and clipping of an aneurysm. Combined with the stereoscopic view of the surgical microscope, 7- to 10-minute 360° 3D VR videos augmented with annotations, overlays, and commentary were created. These videos were then presented to the neurosurgical residents at the authors’ institution using a head-mounted display. Before viewing the videos, the residents were asked to fill out a questionnaire indicating their VR experience and self-assessment of surgical skills regarding the specific procedure. After watching the videos, the residents completed another questionnaire to evaluate their quality and usefulness. The parameters were scaled with a 5-point Likert scale.

RESULTS

Twenty-two residents participated in this study. The mean years of experience of the participants in neurosurgery was 3.2 years, ranging from the 1st through the 7th year of training. Most participants (86.4%) had no or less than 15 minutes of VR experience. The overall quality of the videos was rated good to very good. Immersion, the feeling of being in the operating room, was high, and almost all participants (91%) stated that 360° VR videos provide a useful addition to the neurosurgical training. VR sickness was negligible in the cohort.

CONCLUSIONS

In this study, the authors demonstrated the feasibility and high acceptance of augmented 360° 3D VR videos in neurosurgical training. Augmentation of 360° videos with complementary and interactive content has the potential to effectively support trainees in acquiring conceptual knowledge. Further studies are necessary to investigate the effectiveness of their use in improving surgical skills.

The training of residents and medical students in the surgical disciplines can be challenging since it is often both time-consuming and costly.1,2 This is becoming increasingly important because in modern-day surgical training the time spent in the operating room (OR) is limited by reduced working hours and an increasing volume of patients with a condensation of workflows.3 The global coronavirus disease 2019 (COVID-19) pandemic has now significantly exacerbated these problems. Hands-on courses and elective surgeries are kept to a minimum, limiting two of the key factors in surgical education: observation and practice.47

Conventional surgical videos are limited to the video of the surgical microscope. Attempts to provide a true 3D perspective of the surgeon’s view with head-mounted cameras to visualize the complex interactions in the OR have proved impractical.10,11 Currently, with rapidly evolving technologies, more tools like virtual reality (VR) are available to drive modern surgical training. VR allows for a multisensory immersive experience based on the stereoscopic 3D (S3D) view that has been successfully implemented in videos and simulators.9,12 The immersion and interactivity of VR have proved to increase learner’s motivation13 and understanding of spatial relationships8,14 for neuroanatomy, as well as operative performance, reducing operative time in laparoscopic surgery.15

A rather new development in surgical training is the implementation of 360° videos, which give traditional surgical videos a new dimension and enable all-round access to the OR.16 To create a 360° video, an S3D-capable 360° camera with ambisonic microphones, which is placed at a central point, captures the entire OR and enables the trainee to turn in all possible directions to see the complete environment. If the microscope video is also embedded in the 360° video, the trainee can not only watch the microsurgical video, but also observe the surgeon’s actions and interaction within the OR. Different surgical or procedural 360° videos have already been tested in various medical disciplines and setups1721 such as physical examination,22 laparoscopic surgery16,23 or gastrointestinal endoscopy.24 With regard to surgical training, Yoganathan et al., for example, showed in a randomized controlled trial in 2018 that the acquisition of knot-tying skills in surgery residents was better with a 360° 3D VR video than with a 2D video.25 The University of Leiden uses 360° 3D VR videos as part of an online course in transplant medicine.26 However, the major innovation that 360° videos provide is the possibility to superimpose interactive elements like illustrations or annotations. This is particularly helpful to not only highlight the complex intraoperative anatomy, but also explain the multiple processes, such as neuronavigation and neuromonitoring, that are essential, especially with neurosurgical procedures. Such augmented 360° videos provide a holistic approach and therefore are supposed to not only teach surgical skills, but also promote conceptual knowledge. However, to our knowledge, for complex neurosurgical or neurointerventional procedures so far, only S3D GoPro or 180° videos, but not augmented 360° 3D VR videos, exist.10,11,27,28 In this study, we wanted to demonstrate that augmented 360° 3D VR videos are feasible as training elements for complex neurosurgical approaches and to further evaluate whether they are regarded as useful by neurosurgical residents.

Methods

Study Design

In cooperation with gamelab.berlin, which is associated with Humboldt University (Berlin, Germany), we produced three augmented 360° 3D VR videos. The videos demonstrate an excision of a lumbar disc herniation via the interlaminar approach (lumbar disc), microsurgical resection of cerebral metastasis (tumor), and microsurgical clipping of an intracranial aneurysm (aneurysm) (Figs. 1 and 2). Then, in a prospective study, from June 2020 to January 2021 we showed the training videos to neurosurgical residents in postgraduate years (PGYs) 1–7 at our department using a head-mounted display (HMD). Questionnaires before and after watching each video were applied. This study was approved by the local ethics committee of Charité–Universitätsmedizin Berlin. Each patient and member of the involved surgical team was informed about the recording of the operation and gave consent. The detailed methods, including production, demonstration of 360° 3D VR videos, and questionnaires and statistics, are reported in Supplementary Methods.

FIG. 1.
FIG. 1.

Photograph of the intraoperative setting while recording the clipping of a middle cerebral artery bifurcation aneurysm with a 360° camera attached to the lighting system.

FIG. 2.
FIG. 2.

Screenshot of the augmented 360° 3D VR video of the resection of a cerebral metastasis. The stereoscopic operative microscope video is projected onto a virtual screen within the OR. Please contact the corresponding author for a one-time link to view the video using a smartphone VR headset, if needed.

Results

Participant Characteristics

In this study, 22 neurosurgical residents with a mean age (range) of 30.3 (25–35) years and a mean experience of 3.2 years, ranging from the 1st to the 7th year of training, were included. Most residents were in PGY-1 (n = 7, 31.8%) (Table 1). Majority of the participants were male (63.6%). Only 3 of the 22 trainees had more than 15 minutes of VR experience prior to this study, whereas most residents had no (40.9%) or less than 15 minutes (45.5%) of experience with VR (Table 1).

TABLE 1.

Characteristics of the participants including VR experience and self-assessment of satisfaction with the existing technical learning opportunities at our institution, independent of any particular procedure, using a 5-point Likert scale

CharacteristicValue
Total no. of participants22
Age, yrs30.27 (25–35)
Female/male sex36.4%/63.6%
Mean experience in neurosurgery, yrs3.16 (1–7)
Distribution over the years of training
 PGY-17 (31.8)
 PGY-24 (18.2)
 PGY-31 (4.5)
 PGY-43 (13.6)
 PGY-53 (13.6)
 PGY-63 (13.6)
 PGY-71 (4.5)
Experience with VR
 None9 (40.9)
 <15 mins10 (45.5)
 >15 mins3 (13.6)
I am satisfied with the technical learning opportunities at my institution2 [1.75–3.0]

Values are presented as number (%), mean (range), or median [IQR] unless otherwise indicated.

Self-Assessment of Surgical Skills Before the 360° 3D VR Videos

Fifty-five questionnaires were completed (22 lumbar disc, 17 tumor, 16 aneurysm). The quantification of residents’ experience and their self-assessment regarding the particular procedure are summarized in Table 2. Of the three given surgeries, the residents most often had performed a resection of a disc herniation partially or completely self-reliantly (mean [range] 11.2 [0–40] and 9.8 [0–40] times, respectively). The experience with clipping of aneurysms was limited to assistance during the procedure. Accordingly, the subjective perception of good preparation for the surgical procedure was highest in the case of disc herniation (mean [range] 4 [2–4]), with the highest estimate for the self-sustaining performance of the operation (mean [range] 59.5% [0%–100%]).

TABLE 2.

Evaluation of the standing for the individual operations before viewing of the 360° 3D VR videos

Lumbar Disc (n = 22)Tumor (n = 17)Aneurysm (n = 16)
Objective experience
 No. of times in which this operation was performed partially self-reliantly11.2 (0–40)4.4 (0–40)0
 No. of times in which this operation was performed totally self-reliantly9.8 (0–40)1.5 (0–10)0
No. of times assisted in this operation
 <107 (31.8)7 (41.2)10 (62.5)
 10–306 (27.3)6 (35.3)5 (31.25)
 >309 (40.9)4 (23.5)1 (6.25)
Subjective feeling of experience/preparation
 I feel well prepared for this operation4.0 (2.0–4.0)2.0 (2.0–4.0)2.5 (1.25–3.0)
 I have a lot of experience with this operation2.0 (1.0–3.0)2.0 (1.0–3.0)1.5 (1.0–2.0)
 I have substantial knowledge about the anatomy in the operation situs4.0 (3.0–4.0)4.0 (2.5–4.0)4.0 (2.5–4.0)
I can do _% of this operation by myself
 Mean (range)59.5 (0–100)40.8 (0–100)10.1 (0–30)
 Median [IQR]80 [10–95]20 [10–85]10 [0–18.8]

This table summarizes the objective and subjective experiences of the participants per surgical approach prior to viewing the 360° 3D VR videos. Values are presented as number (%), mean (range), or median [IQR].

Assessment of the Quality of the 360° 3D VR Videos

After watching the videos, the participants answered questions regarding their quality. The overall quality of the videos was rated similarly for each video, between very good and good, with a median (IQR) of 1.5 (1.0–2.0) according to German school grades: 1 (very good) to 6 (insufficient). The separate items regarding the quality of the 360° videos can be seen in Table 3. Briefly, the length of the video was reported to be neither too long nor too short. Mild VR sickness (feeling of dizziness/nausea) occurred only in 3 of 55 instances (5.45%); however, VR sickness did not cause interruption or pausing of the video in any of the cases. Participants mostly agreed that the important steps of the surgeries were included and described appropriately, and the crucial structures were easy to identify (median of 4 for both questions for all videos). The user interface and wearing comfort of the VR headset were satisfying. Audio overlay was rated very good in all the videos, while the quality of the resolution seemed to differ between the videos (mean [range] 3 [3–5] for disc herniation to mean 5 [3.5–5] for metastasis). Participants had the feeling of being in the OR for all videos as an indicator for immersion.

TABLE 3.

Assessment of the quality and content of the 360° 3D VR videos, as well as the user interface and convenience of the VR headset, using a 5-point Likert scale

ItemsLumbar Disc (n = 22)Tumor (n = 17)Aneurysm (n = 16)
This video is a suitable preparation for this operation5.0 [3.75–5.0]5.0 [4.5–5.0]5.0 [4.25–5.0]
I had the feeling of being in the OR4.0 [3.0–5.0]4.0 [4.0–5.0]4.0 [4.0–5.0]
I experienced nausea/dizziness/headaches while watching the video1.0 [1.0–2.0]1.0 [1.0–2.0]1.0 [1.0–2.0]
The video was too short2.0 [1.0–3.25] 2.0 [1.0–3.0]1.5 [1.0–2.75]
The video was too long1.0 [1.0–2.0]1.0 [1.0–2.5]1.5 [1.0–2.75]
The important structures were easy to identify3.0 [2.75–4.0]4.0 [4.0–5.0]4.0 [4.0–5.0]
The important steps were described appropriately4.0 [4.0–5.0]4.0 [3.5–5.0]4.0 [3.25–5.0]
The important segments of the operation were included4.0 [4.0–5.0]4.0 [4.0–5.0]4.0 [4.0–5.0]
The resolution was good3.0 [3.0–5.0]5.0 [3.5–5.0]4.5 [4.0–5.0]
The audio was good5.0 [4.0–5.0]5.0 [4.5–5.0]5.0 [4.0–5.0]
I am satisfied with the user interface4.5 [3.75–5.0]5.0 [4.0–5.0]4.5 [4.0–5.0]
I am satisfied with the wearing comfort of the VR headset4.0 [4.0–5.0]4.0 [4.0–5.0]4.0 [3.25–5.0]
Quality (German school grades)1.5 [1.0–2.0]1.5 [1.0–2.0]1.5 [1.0–2.0]

Values are presented as median [IQR].

Subjective Evaluation of the Benefits of 360° 3D VR Videos

In general, 20 of 22 trainees agreed or strongly agreed with the statement that 360° 3D VR videos are a good addition to surgical training (Fig. 3). To evaluate the subjective benefit of the participants, we assessed whether any open questions about the respective operation could be resolved. Here, we observed variable ratings for the lumbar disc video and rather good ratings for the tumor and aneurysm videos. In most cases, the participants felt more confident after viewing the videos. Overall, participants strongly agreed that the augmented 360° 3D VR videos are a suitable preparation for the particular operation (Fig. 3).

FIG. 3.
FIG. 3.

Subjective evaluation of the usefulness of the 360° 3D VR operative videos for each of three recorded surgeries: clipping of a middle cerebral artery bifurcation aneurysm, removal of a brain tumor (metastasis), and excision of a lumbar disc herniation via an interlaminar approach.

Correlation Analysis

For all three videos, there was a positive correlation of the rating of the resolution with the reported feeling of immersion (Spearman rank correlation coefficient [rs] = 0.562, p = 0.013; rs = 0.747, p = 0.002; and rs = 0.792, p = 0.001 for disc herniation, frontal metastasis, and aneurysm, respectively) (Table 4). However, there was no correlation between the experience with a certain operation and the rating of the usefulness of the respective video as a preparation (Table 4). For the disc herniation video, but not for the other videos, there was a strong and statistically significant correlation between the feeling of immersion and the evaluation of the usefulness for preparation for this operation (rs = 0.55, p = 0.008). However, even though this correlation could not be confirmed statistically for brain surgery videos, both the usefulness and immersion scored 4 to 5 on the Likert scale (Table 4).

TABLE 4.

Results of correlation analysis of key elements using the Spearman rank correlation coefficient

Question CorrelatedLumbar Disc (n = 22)Tumor (n = 17)Aneurysm (n = 16)
rsp Valuersp Valuersp Value
Feeling of immersion to rating of resolution0.5620.0130.7470.0020.7920.001
Feeling of immersion to rating of usefulness as a preparation tool0.550.0080.2190.4540.0890.926
No. of times in which this operation was performed partially self-reliantly to rating of usefulness0.4040.0880.2560.332NANA
No. of times in which this operation was performed self-reliantly to rating of usefulness0.2740.2530.251 0.385NANA

NA = not applicable.

Discussion

The main finding of our study is that augmented 360° 3D VR videos are feasible and provide a viable and useful tool to complement neurosurgical training, and the acceptance of such videos by neurosurgical trainees is high. We successfully recorded and created 360° 3D VR neurosurgical videos that combine a full 360° view of the OR with the stereoscopic microscope video of the surgical site. Further, these videos were augmented with didactic overlays, pictures of neuroimaging, and audio comments. Almost all interns and residents stated that the 360° videos are a good preparation for the respective operation and complement the surgical training on site well.

Surgical Training

A career in neurosurgery begins as early as medical school. Training, starting from the medical student stage through residency to qualified membership on a highly specialized interprofessional team, is an essential component of a modern university hospital. Surgical training in particular is demanding, since practicing and observing are essential, but the high workload and strict working time regulations significantly reduce the time spent in the OR.3 The trainees find themselves in a constant field of tension between patient care tasks, administrative tasks, teaching, training, and research, which leads to time pressure and stress, both of which have a negative effect on surgical training.29 There is growing concern that the next generation of surgeons will not be sufficiently qualified at the end of their training to perform operations independently.3 The COVID-19 pandemic exacerbated these problems, as case volume decreased and further diminished residents’ but also students’ neurosurgical exposure.6,30 Therefore, new and innovative teaching strategies were and are needed to maintain and improve the training quality of surgical residents. Given the critical role of spatial awareness,31 combined with sound conceptual and procedural knowledge of the pertinent steps in neurosurgical operations, the training material should aspire to an authentic and realistic representation of the anatomy and overall scenario. According to Heath and Cohen-Gadol, anatomical relations can be better understood by depiction of depth via 3D imaging, which enables trainees to comprehend the intraoperative surgical decision-making based on the surgical anatomy (e.g., a clip application for an aneurysm or tumor dissection technique).9 A recent review by van Beurden et al. analyzed the benefits of stereoscopic visualization and emphasized that all pertinent studies concluded a stereo advantage when used in training.32 More recent studies also highlighted the advantage of stereoscopy in neuroanatomy teaching.14,33,34 Importantly, S3D imaging can be helpful not only in a purely visual sense, but also for learning visuospatial tasks that involve sensing or grasping objects.9 However, since even the improved instructional videos with head-mounted cameras using stereoscopic visualization do not allow full visualization of the surgical procedure,10,11 stereoscopic 360° 3D VR videos could be a perfect addition to the training material in this regard. Such videos represent a holistic approach with additional external observation of every step. Consequently, the number of 360° 3D VR videos has been continuously increasing since the onset of the COVID-19 pandemic.1720,22,35

One of the major add-ons of 360° 3D VR videos is immersion, in this case the feeling of being in the OR. Immersion is crucial for VR applications in general. While viewing the 360° 3D VR videos, the user can observe the entire OR as if they were standing in the OR and follow the complex interactions between the surgeon, OR nurse, and anesthetist. In our study, the rating of resolution correlated positively with the feeling of immersion. In the lumbar disc video, immersion also correlated with usefulness in preparing for surgery, underscoring the importance of the 360° effect. However, this was not the case in the other two videos. Yet, these videos received only high Likert scale ratings (at least 4 of 5) for both immersion and usefulness. However, the lack of correlation with consistently high ratings and a relatively small sample do not rule out a positive correlation between immersion and usefulness. In fact, the exclusively positive ratings regarding immersion and usefulness support the importance of the 360° effect in brain surgery. In this study, we used Oculus Go HMDs with a resolution of 1280 × 1440 pixels per eye; however, because HMDs with a better resolution will likely provide an even more immersive experience, possibly providing a greater learning benefit, further evaluations should take this into account. This lines up with the findings of Gutiérrez et al. and Rupp et al., in which a high degree of immersion not only enhanced the viewing experience, but also improved learning performance and led “to a greater desire to learn about the video’s subject-matter.”36,37 In line with this, Chan et al. showed higher engagement in medical students learning anatomy with 360° videos compared with traditional 2D videos.35 In a randomized crossover study with 2D and 360° videos of a laparoscopic cholecystectomy, Harrington et al. also showed higher engagement, together with fewer thoughts unrelated to the task, indicating better concentration.23

However, there are also studies that did not show superior results for 360° videos. For instance, a 360° 3D VR video of a gentle cesarean section in addition to the standard curriculum did not lead to better knowledge retention in students.17 According to the authors, this could be because of the split-attention effect or an increased cognitive load.17 In this regard, Andersen et al. and Frederiksen et al. reported higher cognitive load for VR simulation and immersive VR videos compared with traditional simulators or 2D videos, possibly inhibiting performance and learning according to the cognitive load theory.3840 Our approach with augmentation of the 360° video, however, might address this issue using interactive elements that can be selected, played, or repeated separately, and more studies will be necessary to evaluate the effectiveness of augmented 360° videos and their impact on cognitive load. Another important point is the selection of the intervention for such training videos. In procedures with a complex intraoperative anatomy requiring interactive teamwork, 3D perception and a 360° overview of the OR are probably more advantageous. Likewise, more complex operations including a large surgical team interacting could benefit more from an immersive 360° approach than an intervention done by one person (e.g., central venous catheter implantation).19 Standardized operations with a regular workflow are especially suitable for teaching conceptual knowledge and the respective sequence of steps. Neurosurgical procedures often require elaborate and extensive intraoperative technologies such as neuromonitoring and neuronavigation and might therefore be more suitable.

Augmented 360° Videos in Neurosurgery

Recent studies have shown encouraging results for the usability of 360° videos in other surgical disciplines.18,24 However, in neurosurgery, only S3D videos captured with a head-mounted camera10 or 180° 3D videos, in which the surgeon’s point of view was augmented with anatomical overlays and CT scans,27,28 have been evaluated so far. The detailed view of the surgical microscope combined with a 360° view offers a new dimension to video-based learning. To our knowledge, this is the first study on the implementation of augmented 360° 3D VR videos in neurosurgery, although 3D imaging plays a huge role in neurosurgical research and education.41 As neurosurgery is a diversified and manifold field, we chose three different exemplary interventions covering three main areas: spine, tumor, and vascular. These operations also represent different stages of experience for the trainee. According to Stienen et al., supratentorial craniotomy (including the resection of a frontal cerebral metastasis) and dorsal noninstrumented spine surgery (including lumbar discectomy) are most frequently performed during neurosurgical residency in Europe.2 Microsurgical treatment of vascular pathologies such as the clipping of an aneurysm, however, is usually only done as an assistant during residency and is rarely performed supervised or even independently,2 as also observed in our cohort. Most of the participants agreed with the statement that the videos helped them feel more confident about performing the operation. Similar to our results, 360° videos improved the self-confidence of residents in orthognathic surgery for performing Le Fort I osteotomy.42,43 We decided to tailor the enhanced 360° videos specifically to the needs of beginners, and the study participants were consequently young residents. Interestingly, however, there was no significant correlation between the number of surgical procedures already performed independently by the trainees and their assessment of the quality and usefulness of the videos. This underlines the usefulness of 360° educational videos also for more experienced trainees.

The role of pre-education in the success of manual training for orthopedic surgery residents has already been shown to increase the intraoperative performance and knowledge retainment.44 We have limited the duration of the videos to 7.5–9.5 minutes so that they can be viewed quickly before performing a similar operation, and also to reduce the risk of VR sickness, which is a common problem in VR applications with symptoms such as nausea, disorientation, headache, dizziness, or eye strain.45 In the study of Arents et al., which analyzed the role of 360° videos in teaching the performance of a cesarean section to students, more than 50% of the participants reported VR sickness.17 According to a meta-analysis on factors associated with VR sickness with HMDs, a video length of less than 10 minutes, low visual stimulation, and matching of user-initiated movement to the environment were helpful in reducing the risk.45 Accordingly, our participants found the length of the videos appropriate.

Drawbacks of the 360° 3D VR Technology, Possible Future Implementations, and Generalizability

While 360° 3D VR videos provide an innovative and promising surgical training approach, the need for sophisticated equipment and advanced technical support is a drawback and might limit the general availability and widespread implementation in neurosurgical and other surgical programs. Such videos require a 360° camera and a specialized team for recording, editing, and stitching, which increases the costs considerably compared with conventional 2D videos.23 At our institution, the creation of one 360° 3D VR video costs around €3000. Therefore, the production of 360° VR videos might be reserved to centers with the necessary financial resources and technical means. At the same time, the videos can also be distributed to other hospitals and countries, necessitating only translation into different languages. This approach is also a way to support developing countries, as a simple VR headset is sufficient to use the videos for high-level teaching. Another positive aspect is the ability to tailor the same 360° VR video to different user groups, in order to provide different professional groups with educational videos with different focuses. For instance, Edwards et al. demonstrated the improved confidence, understanding, technical skills, and efficiency of OR nurses with 360° videos.43 Taken together, VR videos are substantially more sustainable and their benefits outweigh the high production costs. Furthermore, according to Benet et al., 3D imaging in neurosurgical research and education can help reduce overall healthcare costs and improve patient outcomes as a result of improved surgical skills and shortened learning curves.41 In our opinion, these advantages also apply to 360° 3D VR videos and should be further assessed in the future. The introduced setup consisting of one 360° camera with integration of the surgical microscope video is particularly suitable for primarily microsurgical procedures.

These first insights in 360° 3D VR videos encouraged us to implement a VR video library in our department with instructional videos for a variety of standard neurosurgical procedures. The HMD and videos are now readily accessible to interns and residents at any time and should be seen as an effective complement to self-study and surgical preparation (Fig. 4). Further studies are planned to confirm the benefits of augmented 360° 3D VR videos on comprehensive learning and cognitive load. As a future perspective, we also aim to adjust these videos for patient education, like previous studies demonstrated for radiation therapy planning.46

FIG. 4.
FIG. 4.

Representative image of the VR video library in our department. The television screen is on to demonstrate the content the participant is watching.

Limitations

As this was only a feasibility study at a single center, our study is subject to some well-known limitations. First, the sample size and the number of videos were rather small. Second, our study relies on subjective ratings of usefulness by the residents. However, the Likert scale represents a suitable tool to assess the subjective evaluation in a scientific manner and has been commonly used in similar evaluation studies.4749 Still, further studies that objectively assess knowledge retention and gain in surgical skills using augmented 360° 3D VR videos compared with conventional videos are warranted to assess whether experience with 360° VR significantly outperforms that with simple tools. In this study, the feasibility and usefulness of this approach were shown for the first time for neurosurgical training. In fact, our videos could only exploit some of the enormous potential of augmented 360° 3D VR operation videos. As a next step, our goal is to add interactive elements that allow the user to go into detail for different steps or categories such as anatomy, procedural knowledge, and interactions, thereby individualizing the videos to fit each user’s needs.

Conclusions

This initial evaluation provides encouraging results about the practicality and usefulness of 360° VR videos as a complement of neurosurgical training that warrants further investigations. Immersive VR videos can prepare students, interns, and residents for the OR, potentially shortening the learning curves. Furthermore, 360° 3D VR videos are sustainable and can also help address additional training challenges due to the pandemic. However, the impact on knowledge retention or skill acquisition needs further investigation in future studies.

Acknowledgments

Dr. Acker is a participant of the Berlin Institute of Health–Charité Clinician Scientist Program that is funded by the Charité–Universitätsmedizin Berlin and the Berlin Institute of Health. We acknowledge the support of the Cluster of Excellence: “Matters of Activity. Image Space Material” funded by the Deutsche Forschungsgemeinschaft (DFG; German Research Foundation) under Germany’s Excellence Strategy (EXC 2025-390648296). We also acknowledge the help of gamelab.berlin as well as Matías Bertoia, 360° video editor.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Acker, Bruening, Stein, Picht. Acquisition of data: Bruening, Fuellhase. Analysis and interpretation of data: Acker, Bruening. Drafting the article: Acker, Bruening, Truckenmueller. Critically revising the article: Acker, Truckenmueller, Stein, Fuellhase, Vajkoczy, Picht. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Acker. Statistical analysis: Bruening. Administrative/technical/material support: Stein, Fuellhase, Vajkoczy, Picht. Study supervision: Acker, Vajkoczy, Picht.

Supplemental Information

Online-Only Content

Supplemental material is available online.

Previous Presentations

Portions of this work were presented virtually in abstract form at the German Neurosurgical Society (Deutsche Gesellschaft für Neurochirurgie) Annual Conference on Neuro-Oncology, October 17, 2020; and at the German Neurosurgical Society (Deutsche Gesellschaft für Neurochirurgie) Annual Conference, June 9, 2021.

References

  • 1

    Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177(1):2832.

  • 2

    Stienen MN, Bartek J Jr, Czabanka MA, et al. Neurosurgical procedures performed during residency in Europe-preliminary numbers and time trends. Acta Neurochir (Wien). 2019;161(5):843853.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Griner D, Menon RP, Kotwall CA, Clancy TV, Hope WW. The eighty-hour workweek: surgical attendings’ perspectives. J Surg Educ. 2010;67(1):2531.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 4

    Iancu AM, Kemp MT, Alam HB. Unmuting medical students’ education: utilizing telemedicine during the COVID-19 pandemic and beyond. J Med Internet Res. 2020;22(7):e19667.

  • 5

    Preisser AM, Pieter J, Harth V. Classroom teaching at universities and colleges under the conditions of the SARS-CoV-2 pandemic. Article in German. Zentralbl Arbeitsmed Arbeitsschutz Ergon. 2021;71(2):4955.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    COVIDSurg Collaborative. Elective surgery cancellations due to the COVID-19 pandemic: global predictive modelling to inform surgical recovery plans. Br J Surg. 2020;107(11):14401449.

    • Search Google Scholar
    • Export Citation
  • 7

    Kapila AK, Farid Y, Kapila V, Schettino M, Vanhoeij M, Hamdi M. The perspective of surgical residents on current and future training in light of the COVID-19 pandemic. Br J Surg. 2020:107(9):e305.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8

    Held RT, Hui TT. A guide to stereoscopic 3D displays in medicine. Acad Radiol. 2011;18(8):10351048.

  • 9

    Heath MD, Cohen-Gadol AA. Intraoperative stereoscopic 3D video imaging: pushing the boundaries of surgical visualisation and applications for neurosurgical education. Br J Neurosurg. 2012;26(5):662667.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Lee B, Chen BR, Chen BB, Lu JY, Giannotta SL. Recording stereoscopic 3D neurosurgery with a head-mounted 3D camera system. Br J Neurosurg. 2015;29(3):371373.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11

    Park MS, Brock A, Mortimer V, Taussky P, Couldwell WT, Quigley E. GoPro Hero cameras for creation of a three-dimensional, educational, neurointerventional video. J Digit Imaging. 2017;30(5):561565.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12

    Bernardo A. Virtual reality and simulation in neurosurgical training. World Neurosurg. 2017;106: 10151029.

  • 13

    Stepan K, Zeiger J, Hanchuk S, et al. Immersive virtual reality as a teaching tool for neuroanatomy. Int Forum Allergy Rhinol. 2017;7(10):10061013.

  • 14

    Greuter L, De Rosa A, Cattin P, Croci DM, Soleman J, Guzman R. Randomized study comparing 3D virtual reality and conventional 2D on-screen teaching of cerebrovascular anatomy. Neurosurg Focus. 2021;51(2):E18.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15

    Alaker M, Wynn GR, Arulampalam T. Virtual reality training in laparoscopic surgery: a systematic review & meta-analysis. Int J Surg. 2016;29:8594.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16

    Huber T, Paschold M, Hansen C, Wunderling T, Lang H, Kneist W. New dimensions in surgical training: immersive virtual reality laparoscopic simulation exhilarates surgical staff. Surg Endosc. 2017;31(11):44724477.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17

    Arents V, de Groot PCM, Struben VMD, van Stralen KJ. Use of 360° virtual reality video in medical obstetrical education: a quasi-experimental design. BMC Med Educ. 2021;21(1):202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18

    Kantor J. Use of 3-dimensional 360-degree virtual reality video in dermatologic and reconstructive surgery and Mohs micrographic surgery. J Am Acad Dermatol. 2021;85(2):e67e68.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19

    Mah E, Yu J, Deck M, et al. Immersive video modeling versus traditional video modeling for teaching central venous catheter insertion to medical residents. Cureus. 2021;13(3):e13661.

    • Search Google Scholar
    • Export Citation
  • 20

    Petrica A, Lungeanu D, Ciuta A, Marza AM, Botea MO, Mederle OA. Using 360-degree video for teaching emergency medicine during and beyond the COVID-19 pandemic. Ann Med. 2021;53(1):15201530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21

    Patel D, Hawkins J, Chehab LZ, et al. Developing virtual reality trauma training experiences using 360-degree video: tutorial. J Med Internet Res. 2020;22(12):e22420.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22

    Chao YP, Chuang HH, Hsin LJ, et al. Using a 360° virtual reality or 2D video to learn history taking and physical examination skills for undergraduate medical students: pilot randomized controlled trial. JMIR Serious Games. 2021;9(4):e13124.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23

    Harrington CM, Kavanagh DO, Wright Ballester G, et al. 360° operative videos: a randomised cross-over study evaluating attentiveness and information retention. J Surg Educ. 2018;75(4):9931000.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Rodríguez-D’Jesús A, Uchima H. 360° video recording inside a GI endoscopy room: technical feasibility and its potential use for the acquisition of gastrointestinal endoscopy skills. Pilot experience. Gastroenterol Hepatol. 2021;44(3):245249.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25

    Yoganathan S, Finch DA, Parkin E, Pollard J. 360° virtual reality video for the acquisition of knot tying skills: a randomised controlled trial. Int J Surg. 2018;54(Pt A):24-27.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26

    Pieterse AD, Huurman VAL, Hierck BP, Reinders MEJ. Introducing the innovative technique of 360° virtual reality in kidney transplant education. Transpl Immunol. 2018;49:56.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 27

    Ros M, Trives JV, Lonjon N. From stereoscopic recording to virtual reality headsets: designing a new way to learn surgery. Neurochirurgie. 2017;63(1):15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 28

    Ros M, Debien B, Cyteval C, Molinari N, Gatto F, Lonjon N. Applying an immersive tutorial in virtual reality to learning a new technique. Neurochirurgie. 2020;66(4):212218.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29

    Flinn JT, Miller A, Pyatka N, Brewer J, Schneider T, Cao CG. The effect of stress on learning in surgical skill acquisition. Med Teach. 2016;38(9):897903.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 30

    Pelargos PE, Chakraborty A, Zhao YD, Smith ZA, Dunn IF, Bauer AM. An evaluation of neurosurgical resident education and sentiment during the coronavirus disease 2019 pandemic: a North American survey. World Neurosurg. 2020;140:e381e386.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31

    Hegarty M, Keehner M, Cohen C, Montello D. The role of spatial cognition in medicine: applications for selecting and training professionals. In: Allen GL, ed. Applied Spatial Cognition: From Research to Cognitive Technology. Lawrence Erlbaum Associates Publishers; 2007:285-315.

    • Search Google Scholar
    • Export Citation
  • 32

    van Beurden MHPH, Ijsselsteijn W, Juola J. Effectiveness of stereoscopic displays in medicine: a review. 3D Res. 2012;3:3

  • 33

    Cui D, Wilson TD, Rockhold RW, Lehman MN, Lynch JC. Evaluation of the effectiveness of 3D vascular stereoscopic models in anatomy instruction for first year medical students. Anat Sci Educ. 2017;10(1):3445.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 34

    Jacquesson T, Simon E, Dauleac C, Margueron L, Robinson P, Mertens P. Stereoscopic three-dimensional visualization: interest for neuroanatomy teaching in medical school. Surg Radiol Anat. 2020;42(6):719727.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 35

    Chan V, Larson ND, Moody DA, Moyer DG, Shah NL. Impact of 360° vs 2D videos on engagement in anatomy education. Cureus. 2021;13(4):e14260.

    • Search Google Scholar
    • Export Citation
  • 36

    Gutiérrez F, Pierce J, Vergara VM, et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform. 2007;125:155160.

    • Search Google Scholar
    • Export Citation
  • 37

    Rupp MA, Odette KL, Kozachuk J, Michaelis JR, Smither JA, McConnell DS. Investigating learning outcomes and subjective experiences in 360-degree videos. Comput Educ. 2019;128:256268.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 38

    Andersen SA, Mikkelsen PT, Konge L, Cayé-Thomasen P, Sørensen MS. Cognitive load in mastoidectomy skills training: virtual reality simulation and traditional dissection compared. J Surg Educ. 2016;73(1):4550.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 39

    Andersen SAW, Konge L, Sørensen MS. The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training. Med Teach. 2018;40(7):684689.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 40

    Frederiksen JG, Sørensen SMD, Konge L, et al. Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial. Surg Endosc. 2020;34(3):12441252.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 41

    Benet A, Tabani H, Griswold D, et al. Three-dimensional imaging in neurosurgical research and education. World Neurosurg. 2016;91:317325.

  • 42

    Pulijala Y, Ma M, Pears M, Peebles D, Ayoub A. Effectiveness of immersive virtual reality in surgical training—a randomized control trial. J Oral Maxillofac Surg. 2018;76(5):10651072.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 43

    Edwards TC, Patel A, Szyszka B, et al. Immersive virtual reality enables technical skill acquisition for scrub nurses in complex revision total knee arthroplasty. Arch Orthop Trauma Surg. 2021;141(12):23132321.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 44

    Tarchala M, Charbonneau M, Aduljabbar FH, Teles AR, Weber M. Pre-education enhances the success of manual training for orthopedic surgery residents. J Surg Educ. 2019;76(5):14331439.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 45

    Saredakis D, Szpak A, Birckhead B, Keage HAD, Rizzo A, Loetscher T. Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Front Hum Neurosci. 2020;14:96.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 46

    Johnson K, Liszewski B, Dawdy K, Lai Y, McGuffin M. Learning in 360 degrees: a pilot study on the use of virtual reality for radiation therapy patient education. J Med Imaging Radiat Sci. 2020;51(2):221226.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 47

    Clark LA, Watson D. Constructing validity: new developments in creating objective measuring instruments. Psychol Assess. 2019;31(12):14121427.

  • 48

    Jebb AT, Ng V, Tay L. A review of key Likert scale development advances: 1995–2019. Review. Front Psychol. 2021;12:637547.

  • 49

    Meulenbroeks R, van Joolingen WR. Students’ self-reported well-being under corona measures, lessons for the future. Heliyon. 2022;8(1):e08733.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery

    Photograph of the intraoperative setting while recording the clipping of a middle cerebral artery bifurcation aneurysm with a 360° camera attached to the lighting system.

  • View in gallery

    Screenshot of the augmented 360° 3D VR video of the resection of a cerebral metastasis. The stereoscopic operative microscope video is projected onto a virtual screen within the OR. Please contact the corresponding author for a one-time link to view the video using a smartphone VR headset, if needed.

  • View in gallery

    Subjective evaluation of the usefulness of the 360° 3D VR operative videos for each of three recorded surgeries: clipping of a middle cerebral artery bifurcation aneurysm, removal of a brain tumor (metastasis), and excision of a lumbar disc herniation via an interlaminar approach.

  • View in gallery

    Representative image of the VR video library in our department. The television screen is on to demonstrate the content the participant is watching.

  • 1

    Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177(1):2832.

  • 2

    Stienen MN, Bartek J Jr, Czabanka MA, et al. Neurosurgical procedures performed during residency in Europe-preliminary numbers and time trends. Acta Neurochir (Wien). 2019;161(5):843853.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3

    Griner D, Menon RP, Kotwall CA, Clancy TV, Hope WW. The eighty-hour workweek: surgical attendings’ perspectives. J Surg Educ. 2010;67(1):2531.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 4

    Iancu AM, Kemp MT, Alam HB. Unmuting medical students’ education: utilizing telemedicine during the COVID-19 pandemic and beyond. J Med Internet Res. 2020;22(7):e19667.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 5

    Preisser AM, Pieter J, Harth V. Classroom teaching at universities and colleges under the conditions of the SARS-CoV-2 pandemic. Article in German. Zentralbl Arbeitsmed Arbeitsschutz Ergon. 2021;71(2):4955.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6

    COVIDSurg Collaborative. Elective surgery cancellations due to the COVID-19 pandemic: global predictive modelling to inform surgical recovery plans. Br J Surg. 2020;107(11):14401449.

    • Search Google Scholar
    • Export Citation
  • 7

    Kapila AK, Farid Y, Kapila V, Schettino M, Vanhoeij M, Hamdi M. The perspective of surgical residents on current and future training in light of the COVID-19 pandemic. Br J Surg. 2020:107(9):e305.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Held RT, Hui TT. A guide to stereoscopic 3D displays in medicine. Acad Radiol. 2011;18(8):10351048.

  • 9

    Heath MD, Cohen-Gadol AA. Intraoperative stereoscopic 3D video imaging: pushing the boundaries of surgical visualisation and applications for neurosurgical education. Br J Neurosurg. 2012;26(5):662667.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 10

    Lee B, Chen BR, Chen BB, Lu JY, Giannotta SL. Recording stereoscopic 3D neurosurgery with a head-mounted 3D camera system. Br J Neurosurg. 2015;29(3):371373.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Park MS, Brock A, Mortimer V, Taussky P, Couldwell WT, Quigley E. GoPro Hero cameras for creation of a three-dimensional, educational, neurointerventional video. J Digit Imaging. 2017;30(5):561565.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12

    Bernardo A. Virtual reality and simulation in neurosurgical training. World Neurosurg. 2017;106: 10151029.

  • 13

    Stepan K, Zeiger J, Hanchuk S, et al. Immersive virtual reality as a teaching tool for neuroanatomy. Int Forum Allergy Rhinol. 2017;7(10):10061013.

  • 14

    Greuter L, De Rosa A, Cattin P, Croci DM, Soleman J, Guzman R. Randomized study comparing 3D virtual reality and conventional 2D on-screen teaching of cerebrovascular anatomy. Neurosurg Focus. 2021;51(2):E18.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15

    Alaker M, Wynn GR, Arulampalam T. Virtual reality training in laparoscopic surgery: a systematic review & meta-analysis. Int J Surg. 2016;29:8594.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16

    Huber T, Paschold M, Hansen C, Wunderling T, Lang H, Kneist W. New dimensions in surgical training: immersive virtual reality laparoscopic simulation exhilarates surgical staff. Surg Endosc. 2017;31(11):44724477.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17

    Arents V, de Groot PCM, Struben VMD, van Stralen KJ. Use of 360° virtual reality video in medical obstetrical education: a quasi-experimental design. BMC Med Educ. 2021;21(1):202.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18

    Kantor J. Use of 3-dimensional 360-degree virtual reality video in dermatologic and reconstructive surgery and Mohs micrographic surgery. J Am Acad Dermatol. 2021;85(2):e67e68.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19

    Mah E, Yu J, Deck M, et al. Immersive video modeling versus traditional video modeling for teaching central venous catheter insertion to medical residents. Cureus. 2021;13(3):e13661.

    • Search Google Scholar
    • Export Citation
  • 20

    Petrica A, Lungeanu D, Ciuta A, Marza AM, Botea MO, Mederle OA. Using 360-degree video for teaching emergency medicine during and beyond the COVID-19 pandemic. Ann Med. 2021;53(1):15201530.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21

    Patel D, Hawkins J, Chehab LZ, et al. Developing virtual reality trauma training experiences using 360-degree video: tutorial. J Med Internet Res. 2020;22(12):e22420.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22

    Chao YP, Chuang HH, Hsin LJ, et al. Using a 360° virtual reality or 2D video to learn history taking and physical examination skills for undergraduate medical students: pilot randomized controlled trial. JMIR Serious Games. 2021;9(4):e13124.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23

    Harrington CM, Kavanagh DO, Wright Ballester G, et al. 360° operative videos: a randomised cross-over study evaluating attentiveness and information retention. J Surg Educ. 2018;75(4):9931000.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Rodríguez-D’Jesús A, Uchima H. 360° video recording inside a GI endoscopy room: technical feasibility and its potential use for the acquisition of gastrointestinal endoscopy skills. Pilot experience. Gastroenterol Hepatol. 2021;44(3):245249.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 25

    Yoganathan S, Finch DA, Parkin E, Pollard J. 360° virtual reality video for the acquisition of knot tying skills: a randomised controlled trial. Int J Surg. 2018;54(Pt A):24-27.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 26

    Pieterse AD, Huurman VAL, Hierck BP, Reinders MEJ. Introducing the innovative technique of 360° virtual reality in kidney transplant education. Transpl Immunol. 2018;49:56.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 27

    Ros M, Trives JV, Lonjon N. From stereoscopic recording to virtual reality headsets: designing a new way to learn surgery. Neurochirurgie. 2017;63(1):15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 28

    Ros M, Debien B, Cyteval C, Molinari N, Gatto F, Lonjon N. Applying an immersive tutorial in virtual reality to learning a new technique. Neurochirurgie. 2020;66(4):212218.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29

    Flinn JT, Miller A, Pyatka N, Brewer J, Schneider T, Cao CG. The effect of stress on learning in surgical skill acquisition. Med Teach. 2016;38(9):897903.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 30

    Pelargos PE, Chakraborty A, Zhao YD, Smith ZA, Dunn IF, Bauer AM. An evaluation of neurosurgical resident education and sentiment during the coronavirus disease 2019 pandemic: a North American survey. World Neurosurg. 2020;140:e381e386.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31

    Hegarty M, Keehner M, Cohen C, Montello D. The role of spatial cognition in medicine: applications for selecting and training professionals. In: Allen GL, ed. Applied Spatial Cognition: From Research to Cognitive Technology. Lawrence Erlbaum Associates Publishers; 2007:285-315.

    • Search Google Scholar
    • Export Citation
  • 32

    van Beurden MHPH, Ijsselsteijn W, Juola J. Effectiveness of stereoscopic displays in medicine: a review. 3D Res. 2012;3:3

  • 33

    Cui D, Wilson TD, Rockhold RW, Lehman MN, Lynch JC. Evaluation of the effectiveness of 3D vascular stereoscopic models in anatomy instruction for first year medical students. Anat Sci Educ. 2017;10(1):3445.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 34

    Jacquesson T, Simon E, Dauleac C, Margueron L, Robinson P, Mertens P. Stereoscopic three-dimensional visualization: interest for neuroanatomy teaching in medical school. Surg Radiol Anat. 2020;42(6):719727.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 35

    Chan V, Larson ND, Moody DA, Moyer DG, Shah NL. Impact of 360° vs 2D videos on engagement in anatomy education. Cureus. 2021;13(4):e14260.

    • Search Google Scholar
    • Export Citation
  • 36

    Gutiérrez F, Pierce J, Vergara VM, et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform. 2007;125:155160.

    • Search Google Scholar
    • Export Citation
  • 37

    Rupp MA, Odette KL, Kozachuk J, Michaelis JR, Smither JA, McConnell DS. Investigating learning outcomes and subjective experiences in 360-degree videos. Comput Educ. 2019;128:256268.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 38

    Andersen SA, Mikkelsen PT, Konge L, Cayé-Thomasen P, Sørensen MS. Cognitive load in mastoidectomy skills training: virtual reality simulation and traditional dissection compared. J Surg Educ. 2016;73(1):4550.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 39

    Andersen SAW, Konge L, Sørensen MS. The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training. Med Teach. 2018;40(7):684689.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 40

    Frederiksen JG, Sørensen SMD, Konge L, et al. Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial. Surg Endosc. 2020;34(3):12441252.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 41

    Benet A, Tabani H, Griswold D, et al. Three-dimensional imaging in neurosurgical research and education. World Neurosurg. 2016;91:317325.

  • 42

    Pulijala Y, Ma M, Pears M, Peebles D, Ayoub A. Effectiveness of immersive virtual reality in surgical training—a randomized control trial. J Oral Maxillofac Surg. 2018;76(5):10651072.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 43

    Edwards TC, Patel A, Szyszka B, et al. Immersive virtual reality enables technical skill acquisition for scrub nurses in complex revision total knee arthroplasty. Arch Orthop Trauma Surg. 2021;141(12):23132321.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 44

    Tarchala M, Charbonneau M, Aduljabbar FH, Teles AR, Weber M. Pre-education enhances the success of manual training for orthopedic surgery residents. J Surg Educ. 2019;76(5):14331439.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 45

    Saredakis D, Szpak A, Birckhead B, Keage HAD, Rizzo A, Loetscher T. Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Front Hum Neurosci. 2020;14:96.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 46

    Johnson K, Liszewski B, Dawdy K, Lai Y, McGuffin M. Learning in 360 degrees: a pilot study on the use of virtual reality for radiation therapy patient education. J Med Imaging Radiat Sci. 2020;51(2):221226.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 47

    Clark LA, Watson D. Constructing validity: new developments in creating objective measuring instruments. Psychol Assess. 2019;31(12):14121427.

  • 48

    Jebb AT, Ng V, Tay L. A review of key Likert scale development advances: 1995–2019. Review. Front Psychol. 2021;12:637547.

  • 49

    Meulenbroeks R, van Joolingen WR. Students’ self-reported well-being under corona measures, lessons for the future. Heliyon. 2022;8(1):e08733.

    • Crossref
    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 213 213 213
PDF Downloads 216 216 216
EPUB Downloads 0 0 0