Over the last decade, safety and quality practices from the world of aviation have been increasingly adapted to the world of surgery. Numerous studies have shown promising results in improving patient outcomes and reducing costs with “aviation-like” workflows.1 In particular, any intervention that improves team communication and coordination often shows positive value that may extend beyond the scope of the direct discussion enabled by this communication. However, given the already limited time surgeons and surgical teams have to communicate, and the conservative hierarchical nature of the profession that may be in the way of free communication,2 these practices are often slow to be adopted and quickly abandoned. As a result, surgical team communication frequently remains optional, unstandardized, and lacking.
In this study, we demonstrate a unique, complete workflow of intelligent case-specific and team-specific briefing and debriefing, and ongoing collection and recollection of lessons learned. Because one of the authors of this study spent 20 years in the Air Force as an F-15 pilot, we used his experience and expertise to base our workflow design on proven combat aviation practices. We hypothesize that the adoption of this workflow through a technological platform will increase compliance and improve sustainability.
We report the widespread implementation of this workflow through a mobile app in a single institution and its effect on safety, efficiency, and education in neurosurgery cases.
Methods
Ethical Approval
This study qualifies as a quality improvement activity and does not involve human subjects.
Implementation of the Workflow
Prior to this study and during the preintervention period, pre- and postoperative team communication in our program was optional, and there was no standard and/or structured way to communicate. Information was transmitted, if at all, through oral communication, with no collection or ability of recollection of the data.
During the intervention period of this study, we implemented a standardized pre- and postoperative workflow that was executed through a mobile app prototype developed for the purposes of the study. The workflow included (Fig. 1): daily resident case assignments, surgeon-resident briefing and communication the night before surgery, surgery, surgeon-resident debriefing and feedback the days after surgery, and monthly case reports. The briefing and debriefing were built specifically for the case performed and included aspects of the case plan as well as personal and educational fields to promote continuous improvement (Table 1). The briefing and debriefing information from past cases was easily accessible for the users through the mobile app for personal reference. The workflow and usage of the communication platform was recommended but not mandated, and the degree of participation was left to the discretion of the surgeons and residents.
A schematic diagram of the continuous, pre-, and postoperative workflow implemented through the technological platform according to the role of the care-team member.
The fields that were briefed and debriefed for every case through the platform during the intervention period
| Briefed Fields | Debriefed Fields |
|---|---|
| Case description | Case results |
| Equipment | Main lessons learned |
| Preprocedure medications & interventions | Assessment of goals |
| Position | Personal feedback |
| Approach | Notes/comments |
| OR table | |
| Common pearls and pitfalls | |
| Personal improvement goals | |
| Notes/comments |
Data Collection
Data were collected prospectively from neurosurgery cases at NYU Langone Medical Center, Tisch campus. The preintervention (January–May 2020) and intervention (June–October 2020) periods were defined as the 5-month periods before and after implementation of this workflow. Given the change in case volume and possible change in the nature of the cases due to the rise in COVID-19 cases during March–May 2020, analysis of a pre-COVID period (October 2019–February 2020) was also performed. The data from this period were collected retrospectively and included internally reported morbidity and mortality rates only.
Subjective questionnaires were filled out by the participants (residents and surgeons) in the preintervention period and 3 months into the intervention period. The surgical risk of the three patient populations (preintervention, intervention, and pre-COVID periods) was assessed using patients’ American Society of Anesthesiologists (ASA) scale scores reported in the patients’ charts.
In addition to participant surveys, we used a double-blinded design to assess surgical team efficiency and communication. Throughout the preintervention and intervention periods, the circulating and scrub nurses/technicians prospectively reported the presence of any last-minute requests by the team. The surgical team was not aware of the operating room (OR) staff evaluating them, and the OR staff was not aware of the intervention.
Analysis of the cases in the intervention period was based on intention to treat regardless of whether the cases were actually assigned, briefed, and/or debriefed on the app.
Outcomes
The primary outcomes measured were morbidity and mortality rates and the incidence of last-minute requests. Morbidity and mortality were defined per the department’s criteria, which included: surgical site infection, return to the OR within 30 days, death from all causes within 30 days, or surgery-related complications as assessed by the department’s director of quality. Last-minute requests were defined as requests and/or changes to the surgical plans from expected on the morning of surgery (right before or after the patient was brought into the room) that affected the flow of the case, as assessed and reported by the OR staff in the double-blinded survey. Secondary outcomes measured were the users’ responses on subjective questionnaires.
Statistical Analysis
Statistical significance of proportions such as morbidity and mortality and last-minute request rates were assessed using the Fisher exact test. Ninety-five percent confidence intervals (CIs) are reported from a binomial distribution. Differences on survey responses were assessed using a 2-sample, 2-tailed Student t-test. A chi-square test of independence was used to test whether the distributions of ASA scores in the preintervention/pre-COVID and intervention periods were statistically significantly different. All hypothesis tests were 2-sided. Fisher exact tests and chi-square tests were performed using MATLAB (R2020b, MathWorks), whereas Microsoft Excel was used to conduct t-tests.
Results
All surgeries performed during the pre-COVID, preintervention, and intervention periods were included in the analysis (873, 637, and 893 cases, respectively). Case volume was only affected by COVID during parts of the preintervention period, and there was no difference in case volume between the pre-COVID and the intervention periods. There was no significant difference in preoperative ASA score between the preintervention and intervention (p = 0.24) and between the pre-COVID and intervention (p = 0.12) patient populations. Eighty-six percent of cases (768/893) in the intervention period were captured on the app. The average briefing rate for residents was 71%, and surgeons briefed on average in 81% of the cases briefed by the residents. The average debriefing rate for residents was 67%, and surgeons debriefed on average in 88% of the cases debriefed by the residents. Surgeons used the mobile app to provide personalized feedback to residents in 65% of the debriefed cases.
Changes in patient outcomes did not reach statistical significance (Table 2). Mortality rates alone were 0.94% in the preintervention period and 0.34% in the intervention period (p = 0.18). Morbidity and mortality rates were 5.3% in the preintervention period and 4.7% in the intervention period (p = 0.63). One patient died due to COVID-19 infection within 30 days of surgery during the preintervention period.
Morbidity and mortality results
| Variable | Preintervention | Intervention | p Value | ||
|---|---|---|---|---|---|
| No. (%) | 95% CI | No. (%) | 95% CI | ||
| Total no. of cases | 637 | 893 | |||
| M & M | 34 (5.34) | 3.72–7.38 | 42 (4.7) | 3.41–6.30 | 0.63 |
| Mortality | 6 (0.94) | 0.35–2.04 | 3 (0.34) | 0.069–0.97 | 0.18 |
| Pre-COVID | Intervention | ||||
|---|---|---|---|---|---|
| Total no. of cases | 873 | 893 | |||
| M & M | 51 (5.84) | 4.38–7.6 | 42 (4.7) | 3.41–6.30 | 0.29 |
| Mortality | 7 (0.8) | 0.32–1.64 | 3 (0.34) | 0.069–0.97 | 0.22 |
M & M = morbidity and mortality.
Mortality rates alone were 0.80% in the pre-COVID period and 0.34% in the intervention period (p = 0.22). Morbidity and mortality rates were 5.84% in the pre-COVID period and 4.7% in the intervention period (p = 0.29).
Last-minute request surveys were filled out by the OR staff in 33% (211/637) and 37% (334/893) of cases in the preintervention and intervention periods, respectively, and these rates were not significantly different (p > 0.05). The rate of last-minute requests significantly decreased from 16.6% to 10.5% (p = 0.048). A list of the common last-minute requests by categories and their distributions in the preintervention and intervention periods are presented in Table 3.
Comparison of surveys filled out, last-minute requests, list of the common last-minute requests by categories, and its distribution in the preintervention and intervention periods
| Variable | Preintervention | Intervention | p Value |
|---|---|---|---|
| Total surveys filled out | 0.093 | ||
| No. (%) | 211 (33) | 334 (37) | |
| 95% CI | 29.5%–36.9% | 34.2%–40.7% | |
| Total last-minute requests | 0.048 | ||
| No. (%) | 35 (16.6) | 35 (10.5) | |
| 95% CI | 11.8%–22.3% | 7.41%–14.3% | |
| Last-minute request category, % | |||
| Major equipment (e.g., missing trays, neuromonitoring, navigation) | 4.27 | 2.40 | |
| Room setup (e.g., wrong OR table, repositioning) | 6.16 | 3.59 | |
| Procedural changes (e.g., changes to the planned procedure) | 0.95 | 1.50 | |
| Anesthesia-related changes (e.g., unplanned transfusions or medications) | 1.42 | 0.00 | |
| Minor equipment (e.g., drill, fibrin glue, manometer) | 6.64 | 3.59 |
Results of the users’ subjective questionnaires in the preintervention and intervention periods are presented in Table 4. Across a broad range of metrics both residents and surgeons indicated that there was an improvement in the safety, efficiency, and educational aspects of the cases during the intervention period. Many of these changes were statistically significant. Specifically, surgeons indicated that there was an increase in preoperative briefing (p = 0.006), resident familiarity (p = 0.05), and preparedness with the case (p = 0.01). In addition, surgeons indicated that they were more likely to provide useful feedback (p = 0.03). Conversely, residents felt that there was an increase in preoperative briefing (p = 0.006), their own familiarity with the case (p = 0.003), and the surgeon providing useful feedback (p = 0.03). Residents further felt that they had superior knowledge of procedure steps (p = 0.0003), contributed more to the case (p = 0.01), were more comfortable raising safety concerns (p = 0.03), were more trusted by surgeons (p = 0.02), and that there was a positive learning environment (p = 0.003).
Participants’ subjective questionnaires
| Preintervention | Intervention | p Value | |
|---|---|---|---|
| Resident questionnaire | |||
| I have briefed with the attending prior to the case | 3.27 ± 0.24 | 4.38 ± 0.24 | 0.006 |
| I was familiar with position | 4.45 ± 0.14 | 4.83 ± 0.11 | 0.07 |
| I was familiar with procedure steps | 3.09 ± 0.28 | 4.75 ± 0.13 | 0.0003 |
| Familiar with case common pitfalls | 3.55 ± 0.25 | 4.67 ± 0.14 | 0.003 |
| Had a chance to prepare | 4.09 ± 0.3 | 4.85 ± 0.1 | 0.06 |
| I felt underprepared for the case | 2.45 ± 0.27 | 1.69 ± 0.24 | 0.06 |
| I identified improvement goals prior to the case | 2.91 ± 0.36 | 4.62 ± 0.14 | 0.002 |
| Attending provided useful feedback | 3.64 ± 0.24 | 4.38 ± 0.11 | 0.03 |
| Attending provided improvement points | 2.27 ± 0.24 | 4.46 ± 0.18 | 0.000003 |
| Positive learning environment | 3.91 ± 0.19 | 4.77 ± 0.12 | 0.003 |
| The attending trusted me | 3.73 ± 0.17 | 4.31 ± 0.13 | 0.02 |
| I contributed significantly to the case | 3.55 ± 0.25 | 4.46 ± 0.14 | 0.01 |
| I feel comfortable raising safety concerns | 4.18 ± 0.16 | 4.77 ± 0.17 | 0.03 |
| Attending questionnaire | |||
| I have briefed with the resident prior to the case | 3.08 ± 0.35 | 4.46 ± 0.27 | 0.006 |
| The resident was familiar with position | 4.23 ± 0.19 | 4.31 ± 0.21 | 0.8 |
| The resident was familiar with procedure steps | 3.54 ± 0.26 | 4.15 ± 0.19 | 0.08 |
| The resident was familiar with case common pitfalls | 3.85 ± 0.31 | 4.08 ± 0.18 | 0.5 |
| I was aware of resident’s familiarity with plan | 3.46 ± 0.36 | 4.31 ± 0.13 | 0.05 |
| I provided useful feedback during the procedure | 3.33 ± 0.35 | 4.31 ± 0.17 | 0.03 |
| The resident was underprepared | 2.85 ± 0.36 | 1.77 ± 0.23 | 0.02 |
| I was disappointed with resident’s preparedness | 2.85 ± 0.41 | 1.54 ± 0.18 | 0.01 |
| The resident contributed to safety | 3.62 ± 0.34 | 4.08 ± 0.18 | 0.3 |
| I have provided improvement points after the case | 3.38 ± 0.28 | 3.92 ± 0.29 | 0.2 |
| The teamwork contributed to safety | 3.77 ± 0.35 | 4.31 ± 0.24 | 0.2 |
| The teamwork contributed to efficiency | 4.00 ± 0.33 | 4.31 ± 0.26 | 0.5 |
Data given as means ± SDs (5-point evaluation scale) in the preintervention period and 3 months into the intervention.
Boldface type indicates statistical significance.
Residents estimated that filling out the briefing and debriefing took them on average 10.2 ± 1.8 minutes and 7.7 ± 1.0 minutes, respectively, and attendings estimated 6.8 ± 1.2 minutes and 5.3 ± 0.8 minutes, respectively. In a 5-point evaluation scale (1 = completely disagree, 3 = neutral, 5 = completely agree), when asked whether using the app increased their workload, the responses of residents and attendings averaged 3.0 ± 0.25 and 2.2 ± 0.26, respectively.
Discussion
In this study we used the personal experience of one of the authors as a former Air Force F-15 pilot to design a combat aviation pre- and postmission (i.e., pre- and postoperative) communication workflow. The workflow was implemented through a highly contextual technological platform, and its effect on safety, efficiency, and perioperative team communication was evaluated.
Surgery is often compared to aviation, and to combat aviation in particular.3,4 These two occupations share similar characteristics: both require years of training and highly technical skills, and both deal with highly consequential tasks, which may often determine life or death. New technologies are born and pioneered in these fields, from robotics and hardware, to improved precision and speed, to complicated software, to improved image and data processing. Both strive for perfection, and money is spent to improve results even by fractions of a percent.
And yet, despite all those similarities, surgery is lagging decades behind aviation in the development and adoption of practices of nontechnical skills. While aviation has pioneered the use of checklists, crew resource management, team coordination, and communication and simulations for decision-making, similar practices in surgery are not standardized and often do not last.
The importance of nontechnical skills in surgery and the impact of adopting their associated practices are well recognized. In fact, problems in perioperative communication and their role in preventable medical harms have been described in the surgical literature for decades.5–8 To address these problems, many studies in the last decade have investigated the impact of interventions to improve nontechnical surgical skills. In a systematic review, Buljac-Samardzic et al. (2010) found 48 articles that described interventions to improve team performance,9 while a similar review by the same group in 2020 found 297 studies that met the same inclusion criteria.1
A great success story of importing aviation-like practices into surgery is the incorporation of the surgical safety checklist into the OR’s routine operations around the world. In 2009, Haynes et al. reported the results of implementing the checklist at 8 different hospitals, which demonstrated significant reduction in morbidity and mortality.10 This checklist is not flawless, and many modifications of it exist and have been reported.11–14 However, performing a checklist became a universal standard: surgical teams should not and therefore do not operate without it.
Conversely, standard pre- and postoperative surgical team communication remains optional. Practices of briefing before a mission (i.e., surgery) and debriefing afterward are still not standardized, and often nonexistent. Numerous studies have described attempts to streamline this communication, some with promising results.15,16 In most cases, these attempts remained local and failed to last.
The first major obstacle in the way of sustained standard team communication is the lack of dedicated time for all members of the team to communicate before and after meeting in the OR. McGreevy and Otten described the workflow of combat aviators, which includes daily gatherings of all members of the team in the same room at the same time before and after missions.4 Theoretically, this workflow could improve communication among surgical teams, but it is doomed to fail in the reality of the work habits of surgical teams.
The second major obstacle to sustained standard team communication lies in the conservative hierarchical structure of medicine, and surgery in particular. The perception that the only person who directly affects the outcome in surgery is the attending surgeon promotes the lack of communication and affects patient safety.17–19 It can lead to team members not feeling comfortable in bothering the surgeons to discuss the plan or raise safety concerns, and to the surgeons not communicating the plan with their team.
We suggest that these inherent obstacles can be overcome through the use of a technological platform that we deployed in a single large academic medical center. Technological tools have been increasingly inserted into the daily workflows at hospitals to improve communication.19,20 The workflow we implemented allows for asynchronous communication, which eliminates the need for the physical presence of the surgeons and their residents in the same room at the same time. It also allows for free and standard communication regarding all aspects of the plan, which enables both sides to overcome the hierarchical hurdles. The briefing and debriefing rates of both residents and surgeons in this study suggest that once these obstacles are removed, surgical teams communicate in an efficient manner, similar to that demonstrated in combat aviation. Although use of the platform was not mandatory, residents and surgeons briefed and debriefed in more than 67% of the cases assigned to them on the app during the intervention period.
The improved communication between surgeons and residents led to a direct effect on OR efficiency in the form of a reduction in last-minute requests. Discrepancies between the case plan as understood by the team, and the actual case plan of the surgeon on the day of surgery are common, often manifest as unexpected changes and delays, and result in significant costs.21–23 In this study, last-minute requests were reported in 1 in every 6 cases during the preintervention period. Examples such as missing trays, unplanned neuromonitoring, and using the wrong OR table that were observed in this study could directly correlate to OR delays and costs. Dreyfus et al. reported an average of $1800 of unplanned costs per case due to unplanned instances, some of which were similar and at times identical to the last-minute requests we report here.24 Allowing the residents and surgeons to communicate in a standard and structured manner through a technological platform significantly reduced those instances by 35% during the intervention period. This number could likely be reduced even further with higher briefing rates and inclusion of more members of the team in the briefing.
The results of the participants’ questionnaires have demonstrated a subjective improvement after intervention in aspects of safety, communication, and education when compared to the preintervention period. The cycle of preoperative briefing and setting of improvement goals, and postoperative debriefing, feedback, and setting of future improvement goals enables a continuous improvement process. From an educational standpoint it allows for a structured, focused, and aligned teaching process between the surgeon and the trainee, rather than every case being a singular teaching event.
Limitations
There are several limitations of this study that are important to note. This is a single-institution study, in which the participants were aware of the intervention and the study. This poses a potential participation bias that needs to be considered when evaluating the briefing/debriefing rates and the subjective questionnaires. Reduction in morbidity and mortality with intervention in this study did not meet the threshold for statistical significance (p < 0.05), despite trending lower regardless of the period studied. This is likely due to the lack of statistical power in the setting of low morbidity and mortality rates. Given these rates, a larger multiinstitutional study with a minimum of 3100 cases should be performed to reach statistical significance in showing improvement in patient outcomes.
The effect of COVID-19 on the reported results in the preintervention period is hard to quantify. Case volume was not affected by COVID-19 during the intervention period, but it was reduced during the preintervention period. The effect was mitigated through the comparison of patients’ risk scores and the post hoc addition of the pre-COVID group, but it should be considered as a potential bias in this study. Lastly, there is no comparison between the information communicated during the preintervention period and that communicated during the intervention period. This is an inherent limitation due to the overall lack of communication and no collection of information communicated prior to the implementation of the new workflow.
Conclusions
Implementation of aviation-like structured team communication practices in the neurosurgery department through a technological platform subjectively improved education and communication between surgical teams and led to a reduction in last-minute surgical requests that could impact on costs. While showing multiinstitutional and multispecialty implementation as well as long-term sustainability of these practices is outside the scope of this study, these results indicate the possibility of a culture change that could affect patient care and should be further explored.
Disclosures
During the conduction of this study, the corresponding author (Dr. Ber) co-founded a company (Chiefy inc.), and the intellectual property used in and derived from this study was licensed for further development and commercialization of the technological platform. Dr. Golfinos and Dr. Harter are direct investors in Chiefy.
Author Contributions
Conception and design: Ber, Pacione. Acquisition of data: Ber, Senan, Youssefi. Analysis and interpretation of data: Ber. Drafting the article: Ber. Critically revising the article: London, Harter, Pacione. Reviewed submitted version of manuscript: Pacione. Statistical analysis: London. Administrative/technical/material support: Ber. Study supervision: Harter, Golfinos, Pacione.
References
- 1↑
Buljac-Samardzic M, Doekhie KD, van Wijngaarden JDH. Interventions to improve team effectiveness within health care: a systematic review of the past decade. Hum Resour Health. 2020;18(1):2.
- 2↑
Berger MS, Wachter RM, Greysen SR, Lau CY. Changing our culture to advance patient safety: the 2013 AANS Presidential Address. J Neurosurg. 2013;119(6):1359–1369.
- 3↑
McGreevy J, Otten T, Poggi M, et al. The challenge of changing roles and improving surgical care now: Crew Resource Management approach. Am Surg. 2006;72(11):1082–1087.1126–1148.
- 4↑
McGreevy JM, Otten TD. Briefing and debriefing in the operating room using fighter pilot crew resource management. J Am Coll Surg. 2007;205(1):169–176.
- 5
Greenberg CC, Regenbogen SE, Studdert DM, et al. Patterns of communication breakdowns resulting in injury to surgical patients. J Am Coll Surg. 2007;204(4):533–540.
- 6
Mazzocco K, Petitti DB, Fong KT, et al. Surgical team behaviors and patient outcomes. Am J Surg. 2009;197(5):678–685.
- 7
Rogers SO Jr, Gawande AA, Kwaan M, et al. Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery. 2006;140(1):25–33.
- 8
Rolston JD, Bernstein M. Errors in neurosurgery. Neurosurg Clin N Am. 2015;26(2):149–155, vii.
- 9↑
Buljac-Samardzic M, Dekker-van Doorn CM, van Wijngaarden JDH, van Wijk KP. Interventions to improve team effectiveness: a systematic review. Health Policy. 2010;94(3):183–195.
- 10↑
Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360(5):491–499.
- 11
Bliss LA, Ross-Richardson CB, Sanzari LJ, et al. Thirty-day outcomes support implementation of a surgical safety checklist. J Am Coll Surg. 2012;215(6):766–776.
- 12
Böhmer AB, Wappler F, Tinschmann T, et al. The implementation of a perioperative checklist increases patients’ perioperative safety and staff satisfaction. Acta Anaesthesiol Scand. 2012;56(3):332–338.
- 13
Dabholkar Y, Velankar H, Suryanarayan S, et al. Evaluation and customization of WHO safety checklist for patient safety in otorhinolaryngology. Indian J Otolaryngol Head Neck Surg. 2018;70(1):149–155.
- 14
Erestam S, Haglind E, Bock D, et al. Changes in safety climate and teamwork in the operating room after implementation of a revised WHO checklist: a prospective interventional study. Patient Saf Surg. 2017;11:4.
- 15↑
Einav Y, Gopher D, Kara I, et al. Preoperative briefing in the operating room: shared cognition, teamwork, and patient safety. Chest. 2010;137(2):443–449.
- 16↑
Papaspyros SC, Javangula KC, Adluri RK, O’Regan DJ. Briefing and debriefing in the cardiac operating room. Analysis of impact on theatre team attitude and patient safety. Interact Cardiovasc Thorac Surg. 2010;10(1):43–47.
- 17
Green B, Oeppen RS, Smith DW, Brennan PA. Challenging hierarchy in healthcare teams—ways to flatten gradients to improve teamwork and patient care. Br J Oral Maxillofac Surg. 2017;55(5):449–453.
- 18
Healey AN, Undre S, Vincent CA. Defining the technical skills of teamwork in surgery. Qual Saf Health Care. 2006;15(4):231–234.
- 19↑
Howe EE. Empowering certified nurse’s aides to improve quality of work life through a team communication program. Geriatr Nurs. 2014;35(2):132–136.
- 20↑
Foo E, McDonald R, Savage E, et al. Mobile task management tool that improves workflow of an acute general surgical service. ANZ J Surg. 2015;85(10):760–765.
- 21
Fong AJ, Smith M, Langerman A. Efficiency improvement in the operating room. J Surg Res. 2016;204(2):371–383.
- 22
Lee DJ, Ding J, Guzzo TJ. Improving operating room efficiency. Curr Urol Rep. 2019;20(6):28.
- 23
Weld LR, Stringer MT, Ebertowski JS, et al. TeamSTEPPS improves operating room efficiency and patient safety. Am J Med Qual. 2016;31(5):408–414.
- 24↑
Dreyfus D, Nair A, Rosales C. The impact of planning and communication on unplanned costs in surgical episodes of care: implications for reducing waste in hospital operating rooms. J Oper Manage. 2019;66(1-2):91–111.
