Informed consent, a fundamental tenet of surgical practice, is an opportunity for simultaneously building the patient-physician relationship and improving patient education and understanding of their illness. For preoperative care, informed consent is a complex process with untapped potential for enhancing the quality of care delivered and potentially improving outcomes for both patients and physicians. Conversations regarding informed consent necessarily rely on the rapport between the consenting patient and the surgeon. Previously published studies have demonstrated that patient understanding of a diagnosis and surgical procedure is poor when verbal communication is the sole means of education. 1 Patients appear to have a better understanding when other modes of communication are integrated into the education process. 2
Virtual reality (VR) and augmented reality platforms have recently garnered attention as practical and effective modes of enhancing education for medical students and residents. 3,4 In the field of neurological surgery, these platforms have primarily been utilized for anatomical education and demonstration of surgical approaches, such as simulating an intracranial ventriculostomy catheter or spinal pedicle screw placement. One such VR platform, Surgical Theater, a 3D, surgical rehearsal platform (SRP), is used for education and intraoperative stereotactic navigation. This technology allows for visualization of a surgical approach through the generation of a 3D interface where residents and faculty may rehearse the procedure prior to or during surgery. 5,6
While such platforms have demonstrated utility in education and surgical efficiency, 6 there are few published studies that have examined the use of these technologies to help clinicians communicate with patients and potentially facilitate improved patient understanding of planned surgical procedures. In relationship-centered care, physicians and patients must work together to pursue shared goals, including education, alignment of expectations, and treatment-relevant decision-making. Advances in VR technology provide an invaluable opportunity to bridge the gap in understanding between patients and physicians, further enhancing the quality and completeness of the informed consent process.
We investigated the utility of a VR platform in the surgical informed consent process in order to facilitate communication between neurosurgical residents and patients. We hypothesized that this enhanced consent process would improve patient satisfaction with their physician and improve measures of physician-patient alliance. For residents, we hypothesized that a VR platform would improve comfort and confidence with the consent process. A secondary hypothesis was that the increase in direct patient interaction with the virtual reality–facilitated informed consent (VR-IC) process would lead to a measurable decrease in resident burnout.
Methods
Patients
Patients were eligible for inclusion if they were inpatients and scheduled to undergo elective craniotomy for aneurysm or tumor resection at a tertiary academic medical center. Patients older than 18 years of age with intracranial tumors requiring surgical intervention were considered for inclusion. The final study cohort included those with diagnoses of low- and high-grade glioma, meningioma, vestibular schwannoma, and brain metastases. All consecutive patients who were scheduled for craniotomy for resection of intracranial tumor or clipping of an unruptured cerebral aneurysm between July 2018 and November 2019 were screened for study inclusion. In the event that the patient was unable to undergo the VR-IC process or complete the questionnaires due to neurological symptoms, family members were invited to participate. Patients were excluded if they met one of the following criteria: emergency surgery required, age less than 18 years, or an inability to complete questionnaires with no family members willing or able to participate. The study was approved by the local institutional review board.
Residents
The study included postgraduate year (PGY)–1 through PGY-7 neurosurgical residents in a neurosurgical residency training program at a single academic institution. All residents enrolled in the program between July 2018 and November 2019 elected to participate in this study on a volunteer basis. No residents refused to participate. Preconsent surveys for each resident were performed at the beginning of the training year. Postconsent responses were performed on a rolling basis over the study enrollment period. No other standard training was provided for obtaining patient consent for surgery. Resident anonymity was ensured through the use of surveys conducted through REDCap with analysis conducted using de-identified survey results.
VR-IC Intervention
Following patient enrollment, preoperative volumetric MR images were formatted in the SRP to create an interactive, 3D representation of each patient’s unique anatomy. Patients completed a pre–VR-IC survey to assess baseline satisfaction with their physician as well as their knowledge and subsequent understanding of the medical diagnosis and planned surgical procedure.
Patient participants were then provided a VR headset (Oculus Rift) or a transportable touch-screen VR monitor displaying the 3D representation of volumetric MRI data on which surgical anatomy and pathology were highlighted. All patients were guided through the process of informed consent led by a neurosurgical resident, including visualization of positioning during surgery, incision location, size and location of the craniotomy, the surgical working corridor, lesion-specific anatomy, and specific anatomical considerations associated with potential operative morbidity. At the conclusion of this process, patients were asked to sign the standard hospital informed procedural consent forms providing permission to proceed with surgery. Patients then completed postconsent surveys within 24 hours of the VR-IC encounter, prior to their scheduled surgery. All patients or family members who were enrolled completed postconsent surveys. Each participant who filled out the pre–VR-IC questionnaire was aware that they would undergo VR-IC after they filled out the survey.
Patient questionnaires were modeled after the Likert scale by design, with 1 being the lowest score (strongly disagree) and 5 being the highest score (strongly agree). Following each VR-IC encounter, residents were asked to complete postconsent surveys, which were compared with preconsent survey data obtained at study initiation. The surveys were stored securely on an institutional online research platform (REDCap). 7
Outcome Measures
Patient outcomes were measured using the Patient-Doctor Relationship Questionnaire (PDRQ-9). 8,9 Resident outcomes were assessed using the PDRQ-9, a modified Satisfaction with Simulation Experience Scale (SSES), and the Maslach Burnout Inventory. 8,10–12
The PDRQ-9 is a tool to assess the patient-physician alliance. It is a 9-item questionnaire that has been validated in the primary care setting to evaluate the concept of the “helping alliance” between the physician and patient. 8 For this study, the PDRQ-9 was modified to evaluate the alliance between the physician obtaining the informed consent and the patient both before and after VR-IC.
The SSES is a reliable and validated instrument developed to measure differences in perceived satisfaction with simulated training experiences or simulated learning environments. 11 This scale was originally developed and tested as a comparative analytical tool for multiple simulated training exercises performed by nursing students and has been shown to have adequate internal reliability and construct validity. 10,11 The SSES is an 18-item questionnaire using a 5-point Likert scale. A modified version of the SSES was used to assess resident satisfaction with the VR-IC simulated surgical consent process.
The Maslach Burnout Inventory is a validated scale that assesses three dimensions of burnout: emotional exhaustion, depersonalization, and personal accomplishment. 13 The two most validated questions from the Maslach Burnout Inventory were used to assess the degree of resident burnout both before and after VR-IC. 14
Statistical Analysis
Statistical analysis was performed using SAS 9.4 for Windows (SAS Institute). Descriptive statistics were generated for patient demographics, resident burnout characteristics, and resident satisfaction. Paired t-tests were used to assess pre- and postconsent survey responses for patients and residents. Fisher’s exact test was used to assess differences in responses to the Maslach’s Burnout Inventory. For all tests, a value of α < 0.05 was set for statistical significance.
Ethical Considerations
Iterative analyses were performed at the one-third and halfway points to determine patient perception of the VR-IC. It was determined that if the postconsent overall PDRQ-9 scores were significantly lower than the preconsent values, the study would be halted.
Results
Patient and Resident Characteristics
A total of 50 patients provided surgical consent by using VR-IC. The majority of the patients were Caucasian (88%) with a mean age of 57.5 years (SD 15.3 years). The patients were evenly divided between males (52%) and females (48%). The majority of individuals giving consent were patients undergoing surgery, although family members did perform VR-IC in 2 cases (Table 1).
Patient demographics and clinical characteristics
| Variable | Value |
|---|---|
| Mean age (SD), yrs | 57.5 (15.3) |
| Range | 22–84 |
| Race, n (%) | |
| Caucasian | 44 (88) |
| African American | 3 (6) |
| Asian | 0 (0) |
| Hispanic | 1 (2) |
| Unknown | 2 (4) |
| Sex, n (%)* | |
| Male | 25 (52) |
| Female | 23 (48) |
| Person giving consent, n (%) | |
| Patient | 48 (96) |
| Family member | 2 (4) |
| Pathology, n (%)† | |
| Supratentorial tumor | 38 (78) |
| Infratentorial tumor | 10 (20) |
| Supratentorial aneurysm | 1 (2) |
Data were available for 48 patients.
Data were available for 49 patients.
A total of 19 neurosurgical residents participated in this study. Each resident performed an average of 2.6 consents over the study enrollment period. Twenty-six consent procedures were performed by residents in the 1st year of training, and 24 were performed by residents during PGY 2–7.
Patient Outcomes
The mean overall score of the patient postconsent PDRQ-9 was significantly higher than that of the preconsent mean score (50.53 vs 45.79, p = 0.001; Table 2). The scores for 10 of 12 survey questions increased significantly after VR-IC was performed. The only scores that did not change significantly following the VR-IC intervention were the responses to “I think my surgeon finds me hard to deal with,” which had low scores on both pre- and postconsent surveys (2.08 vs 2.02, p = 0.656), and “My surgeon helps me” (4.04 vs 4.29, p = 0.090).
Comparison of pre- and postconsent patient PDRQ-9 responses
| Mean (SD)† | |||
|---|---|---|---|
| Survey Question* | Preconsent | Postconsent | p Value |
| My surgeon understands me. | 3.94 (0.98) | 4.41 (0.57) | 0.005 |
| I trust my surgeon. | 4.26 (0.80) | 4.59 (0.54) | 0.010 |
| I can talk to my surgeon. | 4.14 (0.88) | 4.46 (0.81) | 0.038 |
| I feel content with my surgeon’s treatment. | 4.08 (0.78) | 4.49 (0.78) | 0.003 |
| I think my surgeon finds me hard to deal with. | 2.08 (1.03) | 2.02 (1.17) | 0.656 |
| My surgeon helps me. | 4.04 (0.77) | 4.29 (0.87) | 0.090 |
| My surgeon has enough time for me. | 3.92 (0.85) | 4.25 (0.76) | 0.006 |
| I benefit from treatment from my surgeon. | 3.86 (0.90) | 4.26 (0.74) | 0.005 |
| My surgeon and I agree on the nature of my symptoms. | 4.08 (0.89) | 4.40 (0.64) | 0.033 |
| I find my surgeon easily accessible. | 3.66 (0.98) | 4.07 (0.85) | 0.013 |
| Thanks to my surgeon, I feel better. | 3.74 (0.90) | 4.12 (0.78) | 0.012 |
| Thanks to my surgeon, I understand my illness better. | 3.90 (0.95) | 4.53 (0.58) | <0.001 |
| Overall score | 45.79 (8.30) | 50.53 (5.93) | 0.001 |
Boldface type indicates statistical significance. Survey questions obtained from the PDRQ-9. 8
1 = strongly disagree; 5 = strongly agree.
Data were available for 50 patients at both time points.
Resident Outcomes
The results of resident pre- and postconsent PDRQ-9 are presented in Table 3. There was a significant difference in residents’ comfort with performing informed consent. The responses to both “I feel comfortable consenting” and “I am prepared to describe each step of the procedure during the consent process” were significantly higher after they had experience with VR-IC (p < 0.0001). Subgroup analysis showed that the largest change after VR-IC occurred in the PGY-1 residents, while the only significant change in the PGY 2–7 residents was the response to “I am confident in my ability to answer all questions posed by a patient or family member during the consent process.” For the PGY-1 residents, the only response that decreased significantly post–VR-IC was the response to “I feel comfortable interacting with patients and families” (4.77 vs 4.17, p = 0.013; Table 3).
Comparison of pre- and postconsent resident PDRQ-9 responses
| Overall | PGY 1 | PGY 2–7 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Survey Question | Preconsent (n = 50) | Postconsent (n = 46) | p Value | Preconsent (n = 26) | Postconsent (n = 24) | p Value | Preconsent (n = 24) | Postconsent (n = 22) | p Value |
| I feel comfortable consenting. | 3.56 (1.21) | 4.50 (0.51) | <0.0001 | 2.62 (0.85) | 4.25 (0.44) | <0.0001 | 4.58 (0.50) | 4.77 (0.43) | 0.213 |
| I am prepared to explain each step of the procedure during the consent process. | 3.46 (1.34) | 4.24 (0.79) | <0.0001 | 2.42 (0.99) | 3.71 (0.69) | <0.0001 | 4.58 (0.50) | 4.82 (0.39) | 0.096 |
| I feel comfortable interacting with patients and families. | 4.70 (0.46) | 4.48 (0.72) | 0.173 | 4.77 (0.43) | 4.17 (0.82) | 0.013 | 4.63 (0.49) | 4.82 (0.39) | 0.096 |
| I am confident in my ability to answer all questions posed by a patient or family member during the consent process. | 3.40 (1.39) | 4.13 (1.00) | 0.0002 | 2.31 (0.97) | 3.46 (0.93) | 0.001 | 4.58 (0.50) | 4.86 (0.35) | 0.030 |
n = number of resident encounters.
Values are presented as the mean (SD) unless indicated otherwise. Boldface type indicates statistical significance. Survey questions obtained from the PDRQ-9. 8
Resident responses to the Maslach Burnout Inventory also changed after VR-IC (Table 4). Prior to VR-IC, the most common response to “I feel burned out from my work” was “never” (16 encounters of residents administering informed consent), followed by “a few times a week” (11 cases). However, after VR-IC, the most common response was “a few times a month” (17 cases), followed by “once a week” (8 cases), although this was not statistically significant. The response to “Since taking this job, I am more callous towards people” changed significantly (p = 0.033). Prior to VR-IC, the most common response was “never” (15 cases), while in the postconsent surveys, the most common response was “a few times a month” (15 cases).
Resident responses to the Maslach Burnout Inventory
| No. of Resident Encounters | |||
|---|---|---|---|
| Survey Question | Preconsent | Postconsent | p Value* |
| I feel burned out from my work | 0.070 | ||
| Never | 16 | 1 | |
| A few times a year or less | 9 | 6 | |
| Once a month or less | 2 | 7 | |
| A few times a month | 5 | 17 | |
| Once a week | 7 | 8 | |
| A few times a week | 11 | 7 | |
| Every day | 0 | 0 | |
| Since taking this job, I am more callous towards people | 0.033 | ||
| Never | 15 | 2 | |
| A few times a year or less | 11 | 8 | |
| Once a month or less | 3 | 7 | |
| A few times a month | 4 | 15 | |
| Once a week | 5 | 6 | |
| A few times a week | 12 | 7 | |
| Every day | 0 | 1 | |
Boldface type indicates statistical significance. Survey questions obtained from Maslach Burnout Inventory. 13
Fisher’s exact test.
Residents were asked to complete a postconsent modified SSES following each VR-IC. The results of these postconsent surveys are shown in Table 5. On a scale of 1–5, the mean score for “This was a valuable learning experience” was 4.46, with all residents giving a response of 4 or 5 (Table 5).
Resident postconsent survey responses: modified SSES
| Survey Question | Mean Score (SD) |
|---|---|
| The surgical rehearsal platform (SRP) consent process improved my delivery skills during informed consent. | 4.32 (0.60) |
| The simulation helped me gain insight into patient understanding of the subject. | 4.39 (0.58) |
| This was a valuable learning experience. | 4.46 (0.50) |
| The simulation caused me to reflect on my ability to perform an effective informed consent. | 4.33 (0.56) |
| The simulation caused me to reflect on my ability to meet patient needs during informed consent. | 4.26 (0.65) |
| The simulation helped me to apply what I have learned throughout residency. | 4.35 (0.60) |
| The simulation helped me to recognize weaknesses in my informed consent delivery skills. | 4.15 (0.70) |
| Having used this technology, I now feel more comfortable in interacting with patients and family members. | 4.20 (0.69) |
Discussion
Facilitating informed consent is one of the most important steps in the preoperative period for patients undergoing surgical procedures. The process of obtaining consent is an opportunity to enhance the therapeutic alliance between physicians and patients. This proof-of-concept study assessed the effect of VR-IC on patient satisfaction and alliance with their surgeon, resident comfort with the consent process as measured by the PDRQ-9 and modified SSES, and resident burnout by the Maslach Burnout Inventory. Our findings have potentially important implications with respect to the role of VR-IC in improving the surgical informed consent process, the experience of patients undergoing surgical procedures, and improving competence and reducing burden among surgical residents.
Informed Consent in Surgery
Informed consent serves many purposes beyond simply obtaining the medical-legal paperwork to perform a surgery. Anticipation and concern about surgical procedures have been known to create feelings of anxiety and vulnerability in patients. 15 However, improved understanding and comprehension can lead to an improved sense of autonomy and a greater sense of control for the patient. In-depth discussions of the planned intraoperative and postoperative care periods between patients and surgeons have a direct impact on patient perceptions of the quality of healthcare services delivered. 16 Informed consent discussions therefore offer a natural opportunity for physicians to address both patient concerns and anxiety, as well as to improve patient understanding and experience.
Previous studies have demonstrated that patient recollections of informed consent discussions are inconsistent. 1,16 Krupp et al. reported that in as little as 2 hours following informed consent for surgical intervention in patients undergoing cranial and spinal surgery, on average, patients did not recall more than 2 of the 6 typical major risks of the procedure that had been discussed. 1 Prior studies have demonstrated that patients undergoing informed consent may benefit when verbal communication is supplemented by other modalities, such as illustrations or audiovisual presentations. 17–19 In a study of patients undergoing treatment of unruptured intracranial aneurysms, Park et al. found that patients were most likely to retain information regarding their pathology and risks of interventions when they received verbal information from their surgeon in addition to a supplemental video explaining the pathology and the array of procedures. 16 A systematic literature review that investigated informed consent in patients who underwent cardiac surgery reported that patient recall and understanding of procedures improved significantly with interactive multimedia presentations and illustrations. 20
VR Augments Patient Experience
VR platforms have proven to be beneficial in patient education efforts prior to surgery or radiation therapy. 21–24 In a study of patients with abdominal aortic aneurysms, Pandrangi et al. reported that patients who were shown pathological models using VR felt more informed. 21 Bekelis et al. found that an immersive preoperative VR experience decreased self-reported patient anxiety and increased patient satisfaction. 25
However, while VR experiences have shown efficacy in improving patient understanding of the pathological diagnosis, this technology has not been widely studied as an adjunct for informed consent discussions. In this study, patients’ satisfaction with their surgeons, as well as perception of their illness, as measured by the PDRQ-9, improved after the enhanced VR-IC process. These results suggest that VR-IC may decrease the burden of explanation that normally occurs in a standard informed consent process and may lead to an improvement in patient-reported measures of understanding in patients undergoing a variety of neurosurgical and general surgery procedures. Given the proven utility of VR in medical education, the poor retention of details among patients following standard informed consent, and the shift toward relationship-based care, the use of VR-IC could be beneficial in enhancing physician-patient discussions regarding complex patient pathophysiology and ensuing surgical procedures.
VR as an Aid to Improve Resident Communication and Experience
In addition to improvements in patient understanding and patient satisfaction, VR-IC may also have an impact on resident-physician satisfaction and perception of the consent discussion. In this study, residents who led the VR-IC discussions demonstrated increased confidence in their ability to consent patients over the study period. At teaching institutions, resident trainees are often expected to lead informed consent discussions. Furthermore, the Accreditation Council for Graduate Medical Education mandates that communication training be part of the surgical residency curriculum. 26
Comfort with obtaining informed consent varies across resident year, specialty, and institution. 27–29 Different strategies have been proposed to improve the informed consent process, including standardizing the process based on procedure and implementing educational programs for residents. 30,31 The results of our pilot study suggest that VR-IC is another method through which residents may gain confidence in their ability to perform informed consent procedures and may also allow them to more thoroughly address patient concerns. The majority of the residents obtaining VR-IC were in PGY-1, and their responses to the PDRQ-9, which gauged confidence with the consent process, improved significantly. This suggests that such a consent process may be particularly useful for early-career residents who are in the process of building patient-communication skills. This technology also has the benefit of widespread applicability across surgical subspecialties and locations, and, although this work was performed in a population of neurosurgical patients and neurosurgical residents, results would be expected to be similar over a wide range of surgical conditions, pathological processes, and surgical subspecialties.
Resident responses to the Maslach Burnout Inventory indicated an increase in burnout following VR-IC compared with preconsent survey results obtained at the beginning of the academic year (Table 4). This finding may be explained by a true progression in measures of burnout over the academic year. The preconsent surveys for each resident were performed at the beginning of the training year, while the postconsent responses were performed on a rolling basis over the study enrollment period. Alternatively, the findings in the study may be explained by an improvement in resident insight in identifying burnout and an improved awareness by resident trainees involved in this study. It should also be noted that no specific training on the process of informed consent was provided to resident trainees before or during this study period. Resident trainees were instructed on how to use the VR as an adjunct tool during informed consent, but no training exercises were conducted. The observed results of the study within this context may provide evidence that the use of VR-IC itself, and not a confounding training program, led to improved outcomes relative to comfort with performing informed consent.
Limitations and Future Directions
This study is limited by a small study population of patients at a single tertiary medical center in the inpatient setting. As a result, it is difficult to generalize these results in community settings or across other specialties. Additionally, as this was a pilot study examining the role of VR in informed consent discussions, we did not survey a control group who underwent the conventional informed consent process. Future studies should compare VR-IC with traditional informed consent approaches to determine situations where VR-IC is most beneficial. The study must also acknowledge the availability of other multimedia adjuncts that may be beneficial for centers that do not have access to VR.
Since this was a pilot study, we did not examine the effect of demographic factors such as patient education level or socioeconomic status on their satisfaction with VR. Future work should study the association of patient satisfaction with VR-IC with demographic factors to better elucidate how informed consent strategies can be tailored to individual patients. Finally, while patient satisfaction and patient understanding may often be correlated, further work must be done to understand the impact of VR-IC on patient understanding of their disease.
Conclusions
In this prospective study, VR-IC improved patient satisfaction, measures of physician-patient alliance, and trainee comfort and confidence. These findings are relevant to modern medical practice as increasing emphasis is placed on patient-reported outcomes and resident educational quality metrics. VR-IC is valuable in catalyzing effective patient-physician communication and alleviating patient anxiety. Most importantly, it fundamentally improves the ability to aid patients with the opportunity to provide truly informed consent.
These findings have significant implications across a wide range of critically important aspects of healthcare. Patient satisfaction and compliance, the patient-physician relationship, and physician burnout are interwoven with the quality of communication between patients and physicians. While neurosurgery accounts for only a small percentage of all surgical procedures that occur throughout the United States, the use of VR-IC can easily be scaled to many common surgical procedures. Additional research is needed to explore scalability and generalizability to other training and clinical settings and to determine the impact of pathology on the utility of VR-IC.
Acknowledgments
This work was funded by the Accreditation Council for Graduate Medical Education Back to Bedside Initiative ($20,000, PI J.M. Wright, 2018–2020).
We would like to thank Betsy Wilson for her help with the statistical analysis.
Disclosures
Dr. Sajatovic: consultant for Alkermes, Otsuka, Janssen, Neurocrine, Bracket, Health Analytics, and Frontline Medical Communications; royalties from Springer Press, Johns Hopkins University Press, Oxford Press, and UpToDate; clinical or research support for the study described from Nuromate, Otsuka, Alkermes, and the International Society for Bipolar Disorders (ISBD); and compensation for preparation of CME activities from the American Physician Institute, MCM Education, CMEology, the Potomac Center for Medical Education, Global Medical Education, Creative Educational Concepts, and the Psychopharmacology Institute. Dr. Selman: investor in and cofounder of Surgical Theater.
Author Contributions
Conception and design: JM Wright, CH Wright, Sajatovic, Selman. Acquisition of data: JM Wright, Raghavan. Analysis and interpretation of data: JM Wright, Raghavan, CH Wright, Shammassian, Sajatovic, Selman. Drafting the article: JM Wright, Raghavan, Shammassian, Duan. Critically revising the article: all authors. Reviewed submitted version of manuscript: JM Wright, Raghavan, CH Wright, Duan, Sajatovic, Selman. Approved the final version of the manuscript on behalf of all authors: JM Wright. Statistical analysis: JM Wright, Raghavan, CH Wright, Sajatovic. Administrative/technical/material support: JM Wright, Raghavan. Study supervision: JM Wright, Selman.
Supplemental Information
Online-Only Content
Supplemental material is available with the online version of the article.
-
Visual Abstract. https://thejns.org/doi/suppl/10.3171/2020.6.JNS201865.
References
- 1↑
Krupp W , Spanehl O , Laubach W , Seifert V. Informed consent in neurosurgery: patients’ recall of preoperative discussion . Acta Neurochir (Wien) . 2000 ;142 (3 ):233 –239 .
- 2↑
Armstrong AP , Cole AA , Page RE. Informed consent: are we doing enough? Br J Plast Surg . 1997 ;50 (8 ):637 –640 .
- 3↑
Lemole M , Banerjee PP , Luciano C , et al. Virtual ventriculostomy with ‘shifted ventricle’: neurosurgery resident surgical skill assessment using a high-fidelity haptic/graphic virtual reality simulator . Neurol Res . 2009 ;31 (4 ):430 –431 .
- 4↑
Banerjee PP , Luciano CJ , Lemole GM Jr , et al. Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback . J Neurosurg . 2007 ;107 (3 ):515 –521 .
- 5↑
Bambakidis NC , Selman WR , Sloan AE. Surgical rehearsal platform: potential uses in microsurgery . Neurosurgery . 2013 ;73 (1 )(suppl 1 ):122 –126 .
- 6↑
Chugh AJ , Pace JR , Singer J , et al. Use of a surgical rehearsal platform and improvement in aneurysm clipping measures: results of a prospective, randomized trial . J Neurosurg . 2017 ;126 (3 ):838 –844 .
- 7↑
Harris PA , Taylor R , Thielke R , et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support . J Biomed Inform . 2009 ;42 (2 ):377 –381 .
- 8↑
Van der Feltz-Cornelis CM , Van Oppen P , Van Marwijk HWJ , et al. A patient-doctor relationship questionnaire (PDRQ-9) in primary care: development and psychometric evaluation . Gen Hosp Psychiatry . 2004 ;26 (2 ):115 –120 .
- 9↑
Wood TJ , Marks M , Jabbour M. The development of a participant questionnaire to assess continuing medical education presentations . Med Educ . 2005 ;39 (6 ):568 –572 .
- 10↑
Williams B , Dousek S. The satisfaction with simulation experience scale (SSES): a validation study . J Nurs Educ Pract . 2012 ;2 (3 ):74 –80 .
- 11↑
Levett-Jones T , McCoy M , Lapkin S , et al. The development and psychometric testing of the Satisfaction with Simulation Experience Scale . Nurse Educ Today . 2011 ;31 (7 ):705 –710 .
- 12
Rafferty JP , Lemkau JP , Purdy RR , Rudisill JR. Validity of the Maslach Burnout Inventory for family practice physicians . J Clin Psychol . 1986 ;42 (3 ):488 –492 .
- 13↑
Maslach C , Jackson SE , Leiter MP. Maslach Burnout Inventory . 3rd ed. Consulting Psychologists Press ; 1996 .
- 14↑
West CP , Dyrbye LN , Satele DV , et al. Concurrent validity of single-item measures of emotional exhaustion and depersonalization in burnout assessment . J Gen Intern Med . 2012 ;27 (11 ):1445 –1452 .
- 15↑
Svensson M , Nilsson U , Svantesson M. Patients’ experience of mood while waiting for day surgery . J Clin Nurs . 2016 ;25 (17-18 ):2600 –2608 .
- 16↑
Park J , Son W , Park KS , et al. Educational and interactive informed consent process for treatment of unruptured intracranial aneurysms . J Neurosurg . 2017 ;126 (3 ):825 –830 .
- 17
Farrell EH , Whistance RN , Phillips K , et al. Systematic review and meta-analysis of audio-visual information aids for informed consent for invasive healthcare procedures in clinical practice . Patient Educ Couns . 2014 ;94 (1 ):20 –32 .
- 18
Delp C , Jones J. Communicating information to patients: the use of cartoon illustrations to improve comprehension of instructions . Acad Emerg Med . 1996 ;3 (3 ):264 –270 .
- 19
Aremu SK , Alabi BS , Segun-Busari S. The role of informed consent in risks recall in otorhinolaryngology surgeries: verbal (nonintervention) vs written (intervention) summaries of risks . Am J Otolaryngol . 2011 ;32 (6 ):485 –489 .
- 20↑
Villanueva C , Talwar A , Doyle M. Improving informed consent in cardiac surgery by enhancing preoperative education . Patient Educ Couns . 2018 ;101 (12 ):2047 –2053 .
- 21↑
Pandrangi VC , Gaston B , Appelbaum NP , et al. The application of virtual reality in patient education . Ann Vasc Surg . 2019 ;59 :184 –189 .
- 22
Bekelis K , Coy S , Simmons N. Operative duration and risk of surgical site infection in neurosurgery . World Neurosurg . 2016 ;94 :551 –555.e6 .
- 23
Jimenez YA , Cumming S , Wang W , et al. Patient education using virtual reality increases knowledge and positive experience for breast cancer patients undergoing radiation therapy . Support Care Cancer . 2018 ;26 (8 ):2879 –2888 .
- 24
Jimenez YA , Lewis SJ. Radiation therapy patient education review and a case study using the virtual environment for radiotherapy training system . J Med Imaging Radiat Sci . 2018 ;49 (1 ):106 –117 .
- 25↑
Bekelis K , Calnan D , Simmons N , et al. Effect of an immersive preoperative virtual reality experience on patient reported outcomes: a randomized controlled trial . Ann Surg . 2017 ;265 (6 ):1068 –1073 .
- 26↑
Raymond MR , Mee J , King A , et al. What new residents do during their initial months of training . Acad Med . 2011 ;86 (10 )(suppl ):S59 –S62 .
- 27
Eftekhari K , Binenbaum G , Jensen AK , et al. Confidence of ophthalmology residents in obtaining informed consent . J Cataract Refract Surg . 2015 ;41 (1 ):217 –221 .
- 28
Gaeta T , Torres R , Kotamraju R , et al. The need for emergency medicine resident training in informed consent for procedures . Acad Emerg Med . 2007 ;14 (9 ):785 –789 .
- 29
Nickels AS , Tilburt JC , Ross LF. Pediatric resident preparedness and educational experiences with informed consent . Acad Pediatr . 2016 ;16 (3 ):298 –304 .
- 30↑
Koller SE , Moore RF , Goldberg MB , et al. An informed consent program enhances surgery resident education . J Surg Educ . 2017 ;74 (5 ):906 –913 .
- 31↑
Thompson BM , Sparks RA , Seavey J , et al. Informed consent training improves surgery resident performance in simulated encounters with standardized patients . Am J Surg . 2015 ;210 (3 ):578 –584 .
