Mobile internet-based mixed-reality interactive telecollaboration system for neurosurgical procedures: technical feasibility and clinical implementation

Shiyu Zhang MD1,2, Fangye Li MD, PhD2, Yining Zhao MD, PhD3, Ruochu Xiong MD1,2, Jingyue Wang MD1,2, Zhichao Gan MD1,2, Xinghua Xu MD, PhD2, Qun Wang MD, PhD2, Huaping Zhang MD, PhD4, Jiashu Zhang MD, PhD2, and Xiaolei Chen MD, PhD2
View More View Less
  • 1 Medical School of Chinese PLA, Beijing, China;
  • | 2 Department of Neurosurgery, Chinese PLA General Hospital, Beijing, China;
  • | 3 Department of Neurosurgery, University Erlangen-Nürnberg, Erlangen, Germany; and
  • | 4 Department of Neurosurgery, Jingzhou Central Hospital, Jingzhou, Hubei, China
Free access

OBJECTIVE

To increase access to health interventions and healthcare services for patients in resource-constrained settings, strategies such as telemedicine must be implemented for the allocation of medical resources across geographic boundaries. Telecollaboration is the dominant form of surgical telemedicine. In this study, the authors report and evaluate a novel mobile internet-based mixed-reality interactive telecollaboration (MIMIT) system as a new paradigm for telemedicine and validate its clinical feasibility.

METHODS

The application of this system was demonstrated for long-distance, real-time collaboration of neuroendoscopic procedures. The system consists of a local video processing workstation, a head-mounted mixed-reality display device, and a mobile remote device, connected over mobile internet (4G or 5G), allowing global point-to-point communication. Using this system, 20 cases of neuroendoscopic surgery were performed and evaluated. The system setup, composite video latency, technical feasibility, clinical implementation, and future potential business model were analyzed and evaluated.

RESULTS

The MIMIT system allows two surgeons to perform complex visual and verbal communication during the operation. The average video delay time is 184.25 msec (range 160–230 msec) with 4G mobile internet, and 23.25 msec (range 20–26 msec) with 5G mobile internet. Excellent image resolution enabled remote neurosurgeons to visualize all critical anatomical structures intraoperatively. Remote instructors could easily make marks on the surgical view; then the composite image, as well as the audio conversation, was transferred to the local surgeon. In this way, a real-time, long-distance collaboration can occur. This system was used for 20 neuroendoscopic surgeries in various cities in China and even across countries (Boston, Massachusetts, to Jingzhou, China). Its simplicity and practicality have been recognized by both parties, and there were no technically related complications recorded.

CONCLUSIONS

The MIMIT system allows for real-time, long-distance telecollaborative neuroendoscopic procedures and surgical training through a commercially available and inexpensive system. It enables remote experts to implement real-time, long-distance intraoperative interaction to guide inexperienced local surgeons, thus integrating the best medical resources and possibly promoting both diagnosis and treatment. Moreover, it can popularize and improve neurosurgical endoscopy technology in more hospitals to benefit more patients, as well as more neurosurgeons.

ABBREVIATIONS

ETV = endoscopic third ventriculostomy; LVP = local video processing; MIMIT = mobile internet-based mixed-reality interactive telecollaboration; RMB = renminbi; VIPAR = virtual interactive presence and augmented reality.

OBJECTIVE

To increase access to health interventions and healthcare services for patients in resource-constrained settings, strategies such as telemedicine must be implemented for the allocation of medical resources across geographic boundaries. Telecollaboration is the dominant form of surgical telemedicine. In this study, the authors report and evaluate a novel mobile internet-based mixed-reality interactive telecollaboration (MIMIT) system as a new paradigm for telemedicine and validate its clinical feasibility.

METHODS

The application of this system was demonstrated for long-distance, real-time collaboration of neuroendoscopic procedures. The system consists of a local video processing workstation, a head-mounted mixed-reality display device, and a mobile remote device, connected over mobile internet (4G or 5G), allowing global point-to-point communication. Using this system, 20 cases of neuroendoscopic surgery were performed and evaluated. The system setup, composite video latency, technical feasibility, clinical implementation, and future potential business model were analyzed and evaluated.

RESULTS

The MIMIT system allows two surgeons to perform complex visual and verbal communication during the operation. The average video delay time is 184.25 msec (range 160–230 msec) with 4G mobile internet, and 23.25 msec (range 20–26 msec) with 5G mobile internet. Excellent image resolution enabled remote neurosurgeons to visualize all critical anatomical structures intraoperatively. Remote instructors could easily make marks on the surgical view; then the composite image, as well as the audio conversation, was transferred to the local surgeon. In this way, a real-time, long-distance collaboration can occur. This system was used for 20 neuroendoscopic surgeries in various cities in China and even across countries (Boston, Massachusetts, to Jingzhou, China). Its simplicity and practicality have been recognized by both parties, and there were no technically related complications recorded.

CONCLUSIONS

The MIMIT system allows for real-time, long-distance telecollaborative neuroendoscopic procedures and surgical training through a commercially available and inexpensive system. It enables remote experts to implement real-time, long-distance intraoperative interaction to guide inexperienced local surgeons, thus integrating the best medical resources and possibly promoting both diagnosis and treatment. Moreover, it can popularize and improve neurosurgical endoscopy technology in more hospitals to benefit more patients, as well as more neurosurgeons.

Unequal distribution of medical resources is a serious problem worldwide.1,2 In China, there are huge gaps in medical resource distribution among different parts of the country.3 Therefore, to increase access to health interventions and healthcare services for patients in resource-limited areas, strategies for the allocation of medical resources across geographic boundaries,3,4 such as telemedicine, must be developed and implemented.

Internet-based telecollaboration is the main form of surgical telemedicine. This system allows experienced surgical specialists to guide surgeons in remote areas who have little or no relevant experience in a real-time interactive manner. The first case of telecollaborative systems used in surgery was reported in the 1960s when physicians performed open heart surgery via satellite broadcast videoconferencing. Since then, more and more surgical telecollaboration systems have been used for various surgical procedures.58 However, the application of telecollaboration in neurosurgery is not as comprehensive as in other disciplines. Limited space and high-precision micromanipulation requirements in neurosurgery limit the applications of telecollaboration.5,8

Long-distance collaboration can be divided into three types: 1) real-time video conferencing, in which telesurgery specialists train local surgeons visually or verbally through live video and voice streaming or freehand sketching;9–11 2) robot-assisted remote surgery, where remote surgical experts operate remote robots directly through the network;12–14 and 3) virtual interactive presence and augmented reality (VIPAR) systems, in which these systems display information on the screen of a flat-panel monitor or smart glasses, allowing the local operator to simultaneously perceive the surgical field and virtual instructions.7,15,16 However, as described in the literature, the VIPAR system has some limitations. The first limitation is that erroneous interactions or serious surgical complications can result from network latency or outages in connectivity.17,18 Furthermore, due to the complexity of the system construction, highly skilled local surgeons are still required to address possible system failure or instability.19 Third, the software or hardware of the system is often customized, which limits its widespread adoption in remote areas.16,19–21

In this paper, we describe a mobile internet-based mixed-reality interactive telecollaboration (MIMIT) system for neurosurgical procedures. The technical feasibility, clinical implementation, and possible business model for this telecollaborative system are reported and analyzed.

Methods

System Overview

The MIMIT system consists of a head-mounted mixed-reality device (HoloLens, Microsoft Inc.),22 a local video processing (LVP) station installed at the site of the procedure, and a remote mobile device (smartphone or tablet PC) connected over a 4G or 5G wireless connection, providing worldwide connectivity.

The LVP station captures a video feed from an endoscope system or a microscope and sends it to the remote device via a cloud server. The remote mobile device can be an iPad, an Android phone, a tablet PC, or a conventional laptop. The mobile device captures the virtual marks of the remote specialist (using fingers or a mouse) for video composition. Next, the real-time hybrid video (operative video and the marks) is sent back to the local LVP station. The hybrid video is then transmitted to the head-mounted HoloLens via a local high-speed network so that a virtual holographic screen panel with hybrid video can be seen, providing effective interaction. A detailed schematic workflow of the MIMIT is given in Fig. 1.

FIG. 1.
FIG. 1.

Diagram of the MIMIT system. Local video and audio feeds are captured with the LVP station, then forwarded to the remote station. The remote instructor marks on the mobile device (a smartphone or a tablet PC), and the composite video is then sent back to the LVP station so that it can be displayed and viewed with a HoloLens connected to the LVP station.

We developed a specific app, “Telecollaboration for surgery” (Guangzhou Jincheng Airui Technology Co., Ltd.), which is downloadable on both iOS (iPad only) (search “缙铖远程医疗” [Chinese] or “Jincheng Airui Medical Technology” [English] in the App Store on an iPad; Fig. 2) and Android mobile systems. Android device users can download the Android app install file (in APK format) at the following link: https://drive.google.com/file/d/1-15w4-lQNs0u-ozH7NySoP4NxOUcNYAR/view?usp=sharing. Use of this app makes it a very straightforward process for the remote specialist to get detailed information of the case, perform the intraoperative telecollaboration, and collect payment after surgery.

FIG. 2.
FIG. 2.

Screenshot of a mobile device (iPad) with our MIMIT system app installed. A: Note the icon of the app (arrow) that we developed for telecollaboration surgery. B: For the English system, users can search “Guangzhou Jincheng Airui Technology Co., Ltd.” (arrow).

Local Station and Connectivity

The LVP station is placed in the operating room of the local hospital, and the video of a neuroendoscope or an operating microscope is captured by the LVP station through a DVI/SDI video port. The LVP station is connected to a 4G or 5G mobile network. A head-mounted mixed-reality device is used to provide a virtual holographic display screen panel in front of the operator’s eyes for intraoperative real-time guidance. The intraoperative mixed-reality view is shown in Fig. 3 and Video 1.

VIDEO 1. Clip showing an intraoperative telecollaboration from case 1. Note the holographic display panel in front of the local neurosurgeon. In the small video window in the lower right corner, the remote instructor can be seen performing the collaboration with a conventional laptop PC. © Xiaolei Chen, published with permission. Click here to view.

FIG. 3.
FIG. 3.

Case 1. A: Intraoperative picture of a local neurosurgeon with a head-mounted holographic display device (HoloLens). B: The mixed-reality holographic panel screen (white arrow). Note a mark (in green) made by the remote instructor overlaid on the endoscopic view. The small window (red arrow) shows the remote instructor providing real-time guidance.

Remote Station and Connectivity

A standard iPad, Android phone, tablet PC, or conventional Windows PC can be used as a remote workstation. In addition to the real-time audio conversation between the remote instructor and the local surgeon, the local operation video, transferred from the LVP station, can be displayed on the operation interface of our dedicated app so that a more experienced instructor can draw marks using his or her fingers or a mouse directly on the screen (Figs. 3 and 4). Before drawing the marks, the instructor can freeze the surgical view, so that different marks can be drawn on a steady surgical view. The color, size, and shape of the marks can be customized by the instructor. In this way, the remote instructor can provide real-time, long-distance interactive audio and video guidance with our telecollaboration app as long as there is mobile internet service, which makes telecollaboration available anywhere, without special meeting rooms. This novel setting even made a real “curbside consult” possible (case 3; Fig. 4A).

FIG. 4.
FIG. 4.

Cases 3 (A) and 14 (B and C). A: Case 3 was a transsphenoidal endoscopic removal of a recurrent pituitary adenoma. The local neurosurgeon was confused by the complicated anatomical structures intraoperatively and requested urgent telecollaboration. The remote instructor performed telecollaboration on the sidewalk of a street in Beijing using his mobile phone (inset). This case was a true “curbside consult.” Note the Beijing sky in the background in the inset window (arrow). B: In case 14, the remote instructor was performing telecollaboration in Boston, Massachusetts. He had just marked on the endoscopic view (arrow). C: The local LVP view in Jingzhou, Hubei, China. The marks on the endoscopic view (arrow) were updated instantly.

Audio and Video Composite Latency

For telecollaborative surgical procedures, audio and video latency is critical for both safety and clinical efficacy. In this study, both audio and composite video are transmitted via the 4G/5G mobile network. The latency depends on the transfer rate between the two workstations. Previous reports on remote interaction assessed the delay of internet transmission and video synthesis by intercepting offline video and performing frame-by-frame analysis.20,21 This requires too many human resources and is relatively subjective. To test the precise end-to-end latency, we programmed accurate time display software (millisecond clock; https://www.dropbox.com/s/umtllvl31fi70c2/Millisecond%20Clock.rar?dl=0), which can display the instant system time to a millisecond level. The program is installed on both the LVP station and a standard Windows PC at the remote site. Before each telecollaboration procedure, the LVP workstation, the remote station/device, and the standard Windows PC at the remote site are synchronized with internet time. Then, our millisecond clock program is started on both the LVP station and the PC at the remote site. The LVP workstation transmits the workstation millisecond clock video to the remote mobile device. The remote mobile device, with the LVP time display, was then placed beside the PC running the millisecond clock, so that the remote instant time as well as the LVP local instant time (displayed on the remote mobile device screen) can be displayed in the same picture. Photographs of the remote PC screen and remote mobile device screen were taken every minute until 10 photos were taken for analysis. By comparing the screen-displayed millisecond time of the mobile device and remote station, a precise end-to-end latency can be calculated and recorded. The time difference (subtracting one time from the other) is the precise latency of the composite video in the visual field of both participants (Fig. 5A). We calculate the time difference of 10 photos and take the average. The linear distance between the local and remote sites is recorded for each collaboration procedure (Table 1).

FIG. 5.
FIG. 5.

Composite video latency test. A: Case 17 was the first case in which we used 5G high-speed mobile internet. This is one of the 10 photos that we took for the latency test. On the instructor’s mobile phone, the collaborative window showed that the local LVP station time was 14:01:46:039, while the time on a synchronized laptop at the remote instructor’s site was 14:01:46:050. Thus, the composite video latency in this photo is 11 msec. With the millisecond clock app that we programmed, the average latency on the 5G network for this case was 23 msec. B: Scatterplot of the linear distance and composite video latency for all 20 cases confirmed that the latency on 5G mobile internet is significantly lower than that on 4G mobile internet.

TABLE 1.

Detailed information of telecollaborative neurosurgical cases performed in this study

Case No.StationProcedureMobile NetworkDistance Btwn Stations (km)Mean Latency (msec)
RemoteLocal
1BeijingBeijingEndoscopic port surgery for an intracerebral CM4G0.1160
2BeijingBeijingETV4G30162
3BeijingNanchangTranssphenoidal endoscopic removal of a recurrent pituitary adenoma4G1249180
4BeijingGuangzhouETV4G1901178
5BeijingJingzhouETV4G1128182
6BeijingWuhanTranssphenoidal endoscopic removal of a recurrent pituitary adenoma4G1045170
7BeijingXi’anETV w/ tumor biopsy4G900180
8BeijingSanyaETV w/ tumor biopsy4G2489198
9SanyaBeijingETV w/ tumor biopsy4G2489196
10BeijingJingzhouETV4G1128178
11BeijingGuangzhouETV4G1901180
12SanyaBeijingETV w/ tumor biopsy4G2489196
13SanyaBeijingETV w/ tumor biopsy4G2489190
14BostonJingzhouEndoscopic fenestration of a trapped temporal horn4G11,923230
15BeijingGuangzhouETV4G1901182
16BeijingGuangzhouETV4G1901186
17SanyaBeijingETV w/ tumor biopsy5G248923
18SanyaBeijingETV5G248920
19BeijingSanyaETV5G248926
20BeijingSanyaEndoscopic fenestration of an arachnoid cyst5G248924

CM = cavernous malformation.

All locations (remote and local) were in China, except for the remote location in case 14 (Boston, Massachusetts).

Payment Solution

To make a sustainable business model, we included a payment solution in our telecollaboration app. The consulting fee is approximately renminbi (RMB) 1000 yuan ($158 USD) per hour. The patient’s representatives can pay with Alipay or credit cards online, just like many other popular online medical consulting apps. Fifty percent of the payment is used for telecollaboration online platform maintenance, 30% of the payment is collected by the remote instructor, while the rest (20%) is collected by the local surgeon. In this way, a legal payment system could be established.

Liability Issues

To avoid malpractice and potential liability issues, only qualified neurosurgeons who finished basic neuroendoscopic training can operate at the local site. For the remote instructor, only specialists who have more than 10 years of experience in neuroendoscopy can be enrolled. All the instructors are registered to an internet hospital (Jincheng Internet Hospital, China). A consent form for telecollaboration surgery was obtained from the patient or patient’s representatives before every procedure. The local hospital takes full responsibility in case of any liability issues.

Results

From February 2017 to December 2019, 20 cases were included in our study. Twenty telecollaborative neuroendoscopic procedures were successfully performed. A consent form for telecollaboration surgery was obtained from each patient or patient’s representatives before every procedure. The local ethics committee approved our study. A successful implementation and trial of the MIMIT system took place between cities in China and the United States. The linear distance between the local site and remote site ranged from 0.1 km (case 1, same building, different rooms) to 11,923 km (case 14, from Boston, Massachusetts, to Jingzhou, China). General information on all 20 cases, as well as the distance between them, is shown in Table 1. In all cases, a stable network connection and telecollaboration could be achieved. There were no technically related complications or liability issues recorded. All surgical procedures were completed uneventfully.

Illustrative Cases

Case 1

Case 1 was the first case in our study. An intracerebral cavernous malformation was removed by endoscopic port surgery. The LVP station was located in a standard operating room, while the remote instructor was seated in a different room in the same building. The instructor used one laptop PC to conduct the telecollaboration and marked on the surgical view using a mouse (Fig. 3, Video 1). The MIMIT telecollaboration was satisfactory with a mean latency of 160 msec. The delay was mild but still notable.

Case 3

Case 3 involved transsphenoidal endoscopic removal for a recurrent pituitary adenoma. Intraoperatively, the local surgeon (in Nanchang, China) was confused by the abnormal anatomy. Hence, he requested telecollaboration with an experienced instructor in Beijing. At that time, the instructor was off duty and out of the hospital. Therefore, the instructor used his mobile phone and performed the telecollaboration in the street (Fig. 4A). This special situation made this case a true “curbside consult.” The surgery was finally successfully performed. The telecollaboration lasted 1.5 hours. The tumor was completely removed and no complications occurred.

Case 14

Case 14 suffered from a trapped temporal horn following intraventricular hemorrhage. A telecollaborative endoscopic fenestration of the temporal horn was planned. The local surgeon was performing surgery in Jingzhou, Hubei, China, while the remote instructor was in Boston, Massachusetts (Fig. 4B and C, Video 2).

VIDEO 2. Clip showing MIMIT telecollaboration in case 14. The first part of the video was taken in Boston, Massachusetts, showing that the instructor collaborated with the local neurosurgeon for an endoscopic fenestration of the trapped temporal horn. The second part of the video was captured by the LVP station at the same time in Jingzhou, Hubei, China, showing clear and almost instantly updated marks on the endoscopic view. © Xiaolei Chen, published with permission. Click here to view.

The instructor used an iPad connected to 4G internet service. The linear distance between the local and remote sites in this case was the longest in our study (11,923 km). The latency was 230 msec, which was notable but still acceptable.

Case 17

Case 17 was the first case for us to test our MIMIT system on 5G mobile internet. An endoscopic third ventriculostomy (ETV) with pineal region tumor biopsy was successfully performed using the MIMIT system. The local surgeon was in Beijing, while the instructor was in Sanya, Hainan, China. The distance between these two sites is 2489 km. With 5G high-speed mobile internet, the average latency was as low as 23 msec (Fig. 5A). The audio and video delay was simply not perceptible.

Video Composite Latency

Video composite latency analysis was performed with the data calculated by our millisecond clock program (Fig. 5A). The local station to remote station video latency averaged 184.25 msec (range 160–230 msec) with 4G mobile internet and significantly lower (23.25 msec, range 20–26 msec) with 5G internet. Of the 20 telecollaboration procedures that have been successfully completed, the shortest straight-line distance was 0.1 km (same building, different rooms), and the longest distance was 11,923 km (Boston, Massachusetts, to Jingzhou, China). In the statistical graph (Fig. 5B), we can see that the latency increases with the straight-line distance. The delay is mild but still perceptible. After we started our MIMIT system over 5G internet, there was a noticeable drop in latency. Despite the distances involved, video latency did not significantly interfere with the surgical procedures. The relationship between latency, distance, and network connection is shown in Table 1 and Fig. 5B.

Setup and Disassembly

The LVP workstation is encased in a single conventional computer case, which makes the setup and disassembly very easy. Setting up the LVP station and breakdown at the end of a case took less than 5 minutes. For the remote site, because we use personal mobile devices such as mobile smartphones and tablets, the setup time for the distant station consists only of starting our telecollaborative app and logging in to the system, which takes less than 1 minute. Surgical procedure times were not believed to be significantly affected by the use of the MIMIT system.

Clinical Implementation Analysis

MIMIT was used throughout the endoscopic procedures, without unacceptable interaction delay or obstacles affecting communication between the two sides. Although noticeable video and audio delays occasionally happened when we used a 4G connection, the internet connection was never lost during the procedures. There were no hardware failures or surgical complications. Each participant strongly agreed that the system was very helpful for the successful implementation of surgery and professional real-time guidance. Up to the last follow-up evaluation, no technically related complications had been recorded. Before the use of our system, many endoscopic operations could not be performed in local hospitals, even if the relevant endoscopy equipment in local hospitals was complete.

Discussion

In China, high-end medical resources are unequally distributed.3 Most experienced neurosurgery specialists usually work in metropolitan areas along the east coast, such as Beijing, Shanghai, or Guangzhou.3 There are huge gaps between these metropolitan areas and inland cities in west China, both in medical technology and in the number of neurosurgical specialists. In recent years, with economic development, more investments in medical equipment are possible for inland cities. Hence, the gap in new equipment between the two regions has been gradually closed. However, the complexity of neurosurgical execution cannot be easily conveyed by only purchasing new equipment. Well-trained, experienced neurosurgeons are essential. In our study, the full set of neuroendoscopic equipment is available in all local hospitals. Unfortunately, most neurosurgeons in local hospitals have limited experience in neuroendoscopic procedures. This situation necessitates the development of technologies to geographically extend the reach of expert neurosurgeons. Although traditional remote robotic surgery has expanded the scope of geographic intervention for surgeons, many shortcomings remain in the application of robotics in neurosurgery, such as expensive investment, delayed movement-related safety issues, and the need for skilled robotic surgeons, which limit its neurosurgical use.23 In recent years, remote interactive systems have developed rapidly, allowing surgeons to conduct long-distance, real-time surgical guidance, which plays a vital role in surgeon training and telecollaborative complex surgical procedures.2,16,20,21

For the MIMIT system we developed, the hardware is inexpensive and the system has proven to be technically feasible and helpful for improving local medical services as well as skill-building for local neurosurgeons. Theoretically, endoscopic, microscopic, and endovascular procedures, which can all export video signals, are ideally suited to the implementation of our MIMIT technology. In our system, the LVP software is commercially available, while the remote instructor app is free and downloadable in the iOS or Android app store. The local site composite video can be viewed using either a HoloLens (holographic display panel) or an inexpensive standard PC panel monitor. This feature makes our system more flexible for different medical centers and different procedures. For example, during future possible microsurgical or endovascular procedures, use of a head-mounted HoloLens may not be possible, but a simple panel monitor can easily take its place.

We used our MIMIT system to successfully perform 20 telecollaborative neuroendoscopic procedures. Compared with previously reported remote interactive systems,13,16,20,21 in our system we objectively and precisely evaluated the composite video delay of all 20 procedures. The distance between the two sites where we perform long-distance, real-time operative interaction ranges from 0.1 to 11,923 km, and the latency of the composite video is 184.25 msec when we use a 4G mobile network. This latency was significantly shortened to 23.25 msec when we used the 5G network in 2019. The 5G network greatly reduces latency and brings a better and safer interactive experience for both participants, which is consistent with previously reported laparoscopic surgery.13 With the rapid deployment of 5G networks in China, we expect that our MIMIT technology can be used between more centers via this faster network.

For the training of local neurosurgeons, expert surgeons may spend short periods of time providing hands-on demonstration or training in local hospitals. The number of short-term surgical trips has increased dramatically over the past 30 years,24 but the lack of emphasis on training and the frequent absence of follow-up have led to criticisms of the short-term trip model.25,26 Although less experienced surgeons may alternatively visit the more experienced expert for longer-term observerships, actual participation in surgery is largely prohibited. As a result, the ideal method for skill-building involves hands-on training of surgeons in their local centers, performing cases on their own patients. In trauma and critically ill patients, nonvirtual interactive tools for extending the expertise of subspecialists are associated with reduced morbidity and mortality.27,28 A versatile and easy-to-use telecollaboration technology to integrate the expertise of a remote surgeon into the surgical field could serve as a valuable adjunct to in-person training efforts. In our study, the MIMIT system allowed long-distance skill training and knowledge transfer between different hospitals.

To make our MIMIT technology sustainable, we designed and tested a feasible business model. A reasonable payment rate ($158 USD/hr) for telecollaboration is collected and distributed legally between the online platform, the local neurosurgeon, and the remote instructor. In this way, the patient actually paid much less than with a typical expert short-term travel model; the patient no longer needs to pay for travel and accommodation expenses, as well as the honorarium of the expert. For the expert specialist, he or she no longer needs to travel more than 12 hours just for a 2-hour endoscopic procedure. Comfortably seated in his/her own office or home, he/she can easily finish 3 or 4 telecollaborative cases in 1 day and collect enough payments. For the local neurosurgeons, interactive telecollaboration systems such as the MIMIT serve as a bridge, providing new skills to local surgeons who are already generally trained for basic neurosurgical skills. They can gain not only firsthand operative experiences but also hands-on demonstrations from experts. In addition, their payments for telecollaboration can serve as a motivation. Lastly, the telecollaboration service company in our MIMIT system designates 50% of the payment for hardware and online platform maintenance. The hardware of the LVP station and a HoloLens cost approximately RMB 30,000 yuan ($4743 USD), which is not expensive. The local hospital can easily cover this part of the cost. However, the development of the MIMIT system, including hardware and software development, took 6 months and cost about RMB 800,000 yuan (about $126,000 USD). For maintenance, it costs about RMB 25,000 yuan ($4000 USD) per month for the rental fee of a web server and relevant cost of labor. The development cost and the later maintenance fee are first covered by a company (Guangzhou Jincheng Airui Technology Co., Ltd.). Like most companies providing an internet-based service, such as Uber, it is reasonable for the company to take 50% of the payment to reimburse the previous development and later maintenance costs. It is inexpensive for the local hospital to obtain the LVP hardware and start the business. The company also has ways to balance the development and maintenance investment in the long run, making the MIMIT system sustainable. Hence, our business model makes a win-win situation possible for the patients, local neurosurgeons, experts, and the online telecollaboration platform company.

Our MIMIT technology is not meant to replace standard neurosurgical training, but instead acts as a complementary method that facilitates mentoring without the physical presence of the experienced expert. We expect that this technology will act as a mentorship bridge, i.e., taking a neurosurgeon with fundamental neurosurgical skills and providing real-time feedback to coach them toward true expertise.

Our ongoing efforts are underway to build a smart online case/mission recruitment and distribution system, as well as an online rating/comment system for both local neurosurgeons and remote instructors. We believe that some basic concepts of successful online businesses, such as Uber or Facebook, can be carefully adopted for the redistribution of medical expertise and relevant resources in China. Future issues facing the widespread adoption of digital telecollaboration systems include reimbursement and liability, as well as rigorous assessment of the impact on patient outcomes.

Conclusions

The MIMIT system allows for real-time, long-distance telecollaborative neuroendoscopic procedures and surgical training through a commercially available and inexpensive system. It enables remote experts to implement real-time, long-distance intraoperative interaction to guide inexperienced local surgeons, thus integrating excellent medical resources and possibly promoting both diagnosis and treatment. Moreover, it can popularize and improve neurosurgical endoscopic technology in more hospitals to benefit more patients, as well as more neurosurgeons.

Acknowledgments

This study was funded by the National Key Research and Development Program of China (grant no. 2018YFC1312602) and the National Natural Science Foundation of China (grant no. 81771481). We would like to thank Mrs. Winnie Chen for programming the millisecond clock app. We would also like to thank Guangzhou Jincheng Airui Technology Co., Ltd., for their support in developing the infrastructure of the MIMIT system.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Chen. Acquisition of data: Chen, S Zhang, Li, J Wang, Q Wang, H Zhang. Analysis and interpretation of data: Zhao, Xiong, Gan, J Zhang. Drafting the article: Chen, S Zhang, Zhao, Gan. Critically revising the article: Chen, S Zhang, Li, Gan, Xu. Reviewed submitted version of manuscript: Chen, Xiong, Gan, J Zhang. Approved the final version of the manuscript on behalf of all authors: Chen. Study supervision: Chen.

Supplemental Information

References

  • 1

    Anwar SL, Harahap WA, Aryandono T. Perspectives on how to navigate cancer surgery in the breast, head and neck, skin, and soft tissue tumor in limited-resource countries during COVID-19 pandemic. Int J Surg. 2020;79:206212.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2

    Aldawoodi NN, Muncey AR, Serdiuk AA, et al. A retrospective analysis of patients undergoing telemedicine evaluation in the preanesthesia testing clinic at H. Lee Moffitt Cancer Center. Cancer Control. 2021;28:10732748211044347.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Chai KC, Zhang YB, Chang KC. Regional disparity of medical resources and its effect on mortality rates in China. Front Public Health. 2020;8:8.

  • 4

    Snyder JJ, Salkowski N, Wey A, Pyke J, Israni AK, Kasiske BL. Organ distribution without geographic boundaries: a possible framework for organ allocation. Am J Transplant. 2018;18(11):26352640.

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Rocque BG, Davis MC, McClugage SG, et al. Surgical treatment of epilepsy in Vietnam: program development and international collaboration. Neurosurg Focus. 2018;45(4):E3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6

    Lepard JR, Akbari SHA, Haji F, Davis MC, Harkness W, Johnston JM. The initial experience of InterSurgeon: an online platform to facilitate global neurosurgical partnerships. Neurosurg Focus. 2020;48(3):E15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7

    Haji FA, Lepard JR, Davis MC, et al. A model for global surgical training and capacity development: the Children’s of Alabama-Viet Nam pediatric neurosurgery partnership. Childs Nerv Syst. 2021;37(2):627636.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Henderson F Jr, Lepard J, Seibly J, Rambo W Jr, Boswell S, Copeland WR III. An online tumor board with international neurosurgical collaboration guides surgical decision-making in Western Kenya. Childs Nerv Syst. 2021;37(2):715719.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9

    Schoen DC, Prater K. Role of telehealth in pre-anesthetic evaluations. AANA J. 2019;87(1):4349.

  • 10

    McGillion M, Ouellette C, Good A, et al. Postoperative remote automated monitoring and virtual hospital-to-home care system following cardiac and major vascular surgery: user testing study. J Med Internet Res. 2020;22(3):e15548.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Xu X, Zeng Z, Qi Y, et al. Remote video-based outcome measures of patients with Parkinson’s disease after deep brain stimulation using smartphones: a pilot study. Neurosurg Focus. 2021;51(5):E2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12

    Zhou X, Zhang H, Feng M, Zhao J, Fu Y. New remote centre of motion mechanism for robot-assisted minimally invasive surgery. Biomed Eng Online. 2018;17(1):170.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13

    Zheng J, Wang Y, Zhang J, et al. 5G ultra-remote robot-assisted laparoscopic surgery in China. Surg Endosc. 2020;34(11):51725180.

  • 14

    Xiong R, Zhang S, Gan Z, et al. A novel 3D-vision-based collaborative robot as a scope holding system for port surgery: a technical feasibility study. Neurosurg Focus. 2022;52(1):E13.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15

    Swanson M, MacKay M, Yu S, Kagiliery A, Bloom K, Schwebel DC. Supporting caregiver use of child restraints in rural communities via interactive virtual presence. Health Educ Behav. 2020;47(2):264271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16

    Dream S, Kuo JH, Wang TS. Virtual interactive presence, a novel approach to remote proctoring for the adoption of innovative technologies and interventions. Am J Surg. Published online September 14,2021.doi: 10.1016/j.amjsurg.2021.09.007

    • Search Google Scholar
    • Export Citation
  • 17

    Kothgassner OD, Goreis A, Kafka JX, et al. Agency and gender influence older adults’ presence-related experiences in an interactive virtual environment. Cyberpsychol Behav Soc Netw. 2018;21(5):318324.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18

    Gromer D, Reinke M, Christner I, Pauli P. Causal interactive links between presence and fear in virtual reality height exposure. Front Psychol. 2019;10:141.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19

    Schwebel DC, MacKay JM, Redden D. Study protocol: a randomised non-inferiority trial using interactive virtual presence to remotely assist parents with child restraint installations. Inj Prev. 2020;26(3):289294.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 20

    Shenai MB, Tubbs RS, Guthrie BL, Cohen-Gadol AA. Virtual interactive presence for real-time, long-distance surgical collaboration during complex microsurgical procedures. J Neurosurg. 2014;121(2):277284.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21

    Davis MC, Can DD, Pindrik J, Rocque BG, Johnston JM. Virtual interactive presence in global surgical education: international collaboration through augmented reality. World Neurosurg. 2016;86:103111.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22

    Qi Z, Li Y, Xu X, et al. Holographic mixed-reality neuronavigation with a head-mounted device: technical feasibility and clinical application. Neurosurg Focus. 2021;51(2):E22.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23

    Mendez I, Hill R, Clarke D, Kolyvas G, Walling S. Robotic long-distance telementoring in neurosurgery. Neurosurgery. 2005;56(3):434440.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Warf BC. Neurosurgical humanitarian aid. J Neurosurg Pediatr. 2009;4(1):13.

  • 25

    Dupuis CC. Humanitarian missions in the third world: a polite dissent. Plast Reconstr Surg. 2004;113(1):433435.

  • 26

    Maki J, Qualls M, White B, Kleefield S, Crone R. Health impact assessment and short-term medical missions: a methods study to evaluate quality of care. BMC Health Serv Res. 2008;8:121.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 27

    Wilcox ME, Adhikari NK. The effect of telemedicine in critically ill patients: systematic review and meta-analysis. Crit Care. 2012;16(4):R127.

  • 28

    Marttos A, Kelly E, Graygo J, et al. Usability of telepresence in a level 1 trauma center. Telemed J E Health. 2013;19(4):248251.

  • View in gallery

    Diagram of the MIMIT system. Local video and audio feeds are captured with the LVP station, then forwarded to the remote station. The remote instructor marks on the mobile device (a smartphone or a tablet PC), and the composite video is then sent back to the LVP station so that it can be displayed and viewed with a HoloLens connected to the LVP station.

  • View in gallery

    Screenshot of a mobile device (iPad) with our MIMIT system app installed. A: Note the icon of the app (arrow) that we developed for telecollaboration surgery. B: For the English system, users can search “Guangzhou Jincheng Airui Technology Co., Ltd.” (arrow).

  • View in gallery

    Case 1. A: Intraoperative picture of a local neurosurgeon with a head-mounted holographic display device (HoloLens). B: The mixed-reality holographic panel screen (white arrow). Note a mark (in green) made by the remote instructor overlaid on the endoscopic view. The small window (red arrow) shows the remote instructor providing real-time guidance.

  • View in gallery

    Cases 3 (A) and 14 (B and C). A: Case 3 was a transsphenoidal endoscopic removal of a recurrent pituitary adenoma. The local neurosurgeon was confused by the complicated anatomical structures intraoperatively and requested urgent telecollaboration. The remote instructor performed telecollaboration on the sidewalk of a street in Beijing using his mobile phone (inset). This case was a true “curbside consult.” Note the Beijing sky in the background in the inset window (arrow). B: In case 14, the remote instructor was performing telecollaboration in Boston, Massachusetts. He had just marked on the endoscopic view (arrow). C: The local LVP view in Jingzhou, Hubei, China. The marks on the endoscopic view (arrow) were updated instantly.

  • View in gallery

    Composite video latency test. A: Case 17 was the first case in which we used 5G high-speed mobile internet. This is one of the 10 photos that we took for the latency test. On the instructor’s mobile phone, the collaborative window showed that the local LVP station time was 14:01:46:039, while the time on a synchronized laptop at the remote instructor’s site was 14:01:46:050. Thus, the composite video latency in this photo is 11 msec. With the millisecond clock app that we programmed, the average latency on the 5G network for this case was 23 msec. B: Scatterplot of the linear distance and composite video latency for all 20 cases confirmed that the latency on 5G mobile internet is significantly lower than that on 4G mobile internet.

  • 1

    Anwar SL, Harahap WA, Aryandono T. Perspectives on how to navigate cancer surgery in the breast, head and neck, skin, and soft tissue tumor in limited-resource countries during COVID-19 pandemic. Int J Surg. 2020;79:206212.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2

    Aldawoodi NN, Muncey AR, Serdiuk AA, et al. A retrospective analysis of patients undergoing telemedicine evaluation in the preanesthesia testing clinic at H. Lee Moffitt Cancer Center. Cancer Control. 2021;28:10732748211044347.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3

    Chai KC, Zhang YB, Chang KC. Regional disparity of medical resources and its effect on mortality rates in China. Front Public Health. 2020;8:8.

  • 4

    Snyder JJ, Salkowski N, Wey A, Pyke J, Israni AK, Kasiske BL. Organ distribution without geographic boundaries: a possible framework for organ allocation. Am J Transplant. 2018;18(11):26352640.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 5

    Rocque BG, Davis MC, McClugage SG, et al. Surgical treatment of epilepsy in Vietnam: program development and international collaboration. Neurosurg Focus. 2018;45(4):E3.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6

    Lepard JR, Akbari SHA, Haji F, Davis MC, Harkness W, Johnston JM. The initial experience of InterSurgeon: an online platform to facilitate global neurosurgical partnerships. Neurosurg Focus. 2020;48(3):E15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7

    Haji FA, Lepard JR, Davis MC, et al. A model for global surgical training and capacity development: the Children’s of Alabama-Viet Nam pediatric neurosurgery partnership. Childs Nerv Syst. 2021;37(2):627636.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Henderson F Jr, Lepard J, Seibly J, Rambo W Jr, Boswell S, Copeland WR III. An online tumor board with international neurosurgical collaboration guides surgical decision-making in Western Kenya. Childs Nerv Syst. 2021;37(2):715719.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9

    Schoen DC, Prater K. Role of telehealth in pre-anesthetic evaluations. AANA J. 2019;87(1):4349.

  • 10

    McGillion M, Ouellette C, Good A, et al. Postoperative remote automated monitoring and virtual hospital-to-home care system following cardiac and major vascular surgery: user testing study. J Med Internet Res. 2020;22(3):e15548.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Xu X, Zeng Z, Qi Y, et al. Remote video-based outcome measures of patients with Parkinson’s disease after deep brain stimulation using smartphones: a pilot study. Neurosurg Focus. 2021;51(5):E2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 12

    Zhou X, Zhang H, Feng M, Zhao J, Fu Y. New remote centre of motion mechanism for robot-assisted minimally invasive surgery. Biomed Eng Online. 2018;17(1):170.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13

    Zheng J, Wang Y, Zhang J, et al. 5G ultra-remote robot-assisted laparoscopic surgery in China. Surg Endosc. 2020;34(11):51725180.

  • 14

    Xiong R, Zhang S, Gan Z, et al. A novel 3D-vision-based collaborative robot as a scope holding system for port surgery: a technical feasibility study. Neurosurg Focus. 2022;52(1):E13.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15

    Swanson M, MacKay M, Yu S, Kagiliery A, Bloom K, Schwebel DC. Supporting caregiver use of child restraints in rural communities via interactive virtual presence. Health Educ Behav. 2020;47(2):264271.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16

    Dream S, Kuo JH, Wang TS. Virtual interactive presence, a novel approach to remote proctoring for the adoption of innovative technologies and interventions. Am J Surg. Published online September 14,2021.doi: 10.1016/j.amjsurg.2021.09.007

    • Search Google Scholar
    • Export Citation
  • 17

    Kothgassner OD, Goreis A, Kafka JX, et al. Agency and gender influence older adults’ presence-related experiences in an interactive virtual environment. Cyberpsychol Behav Soc Netw. 2018;21(5):318324.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18

    Gromer D, Reinke M, Christner I, Pauli P. Causal interactive links between presence and fear in virtual reality height exposure. Front Psychol. 2019;10:141.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19

    Schwebel DC, MacKay JM, Redden D. Study protocol: a randomised non-inferiority trial using interactive virtual presence to remotely assist parents with child restraint installations. Inj Prev. 2020;26(3):289294.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 20

    Shenai MB, Tubbs RS, Guthrie BL, Cohen-Gadol AA. Virtual interactive presence for real-time, long-distance surgical collaboration during complex microsurgical procedures. J Neurosurg. 2014;121(2):277284.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 21

    Davis MC, Can DD, Pindrik J, Rocque BG, Johnston JM. Virtual interactive presence in global surgical education: international collaboration through augmented reality. World Neurosurg. 2016;86:103111.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22

    Qi Z, Li Y, Xu X, et al. Holographic mixed-reality neuronavigation with a head-mounted device: technical feasibility and clinical application. Neurosurg Focus. 2021;51(2):E22.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23

    Mendez I, Hill R, Clarke D, Kolyvas G, Walling S. Robotic long-distance telementoring in neurosurgery. Neurosurgery. 2005;56(3):434440.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Warf BC. Neurosurgical humanitarian aid. J Neurosurg Pediatr. 2009;4(1):13.

  • 25

    Dupuis CC. Humanitarian missions in the third world: a polite dissent. Plast Reconstr Surg. 2004;113(1):433435.

  • 26

    Maki J, Qualls M, White B, Kleefield S, Crone R. Health impact assessment and short-term medical missions: a methods study to evaluate quality of care. BMC Health Serv Res. 2008;8:121.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 27

    Wilcox ME, Adhikari NK. The effect of telemedicine in critically ill patients: systematic review and meta-analysis. Crit Care. 2012;16(4):R127.

  • 28

    Marttos A, Kelly E, Graygo J, et al. Usability of telepresence in a level 1 trauma center. Telemed J E Health. 2013;19(4):248251.

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 130 130 130
PDF Downloads 210 210 210
EPUB Downloads 0 0 0