Intelligent control of neurosurgical robot MM-3 using dynamic motion scaling

Free access

OBJECTIVE

Advanced and intelligent robotic control is necessary for neurosurgical robots, which require great accuracy and precision. In this article, the authors propose methods for dynamically and automatically controlling the motion-scaling ratio of a master-slave neurosurgical robotic system to reduce the task completion time.

METHODS

Three dynamic motion-scaling modes were proposed and compared with the conventional fixed motion-scaling mode. These 3 modes were defined as follows: 1) the distance between a target point and the tip of the slave manipulator, 2) the distance between the tips of the slave manipulators, and 3) the velocity of the master manipulator. Five test subjects, 2 of whom were neurosurgeons, sutured 0.3-mm artificial blood vessels using the MM-3 neurosurgical robot in each mode.

RESULTS

The task time, total path length, and helpfulness score were evaluated. Although no statistically significant differences were observed, the mode using the distance between the tips of the slave manipulators improves the suturing performance.

CONCLUSIONS

Dynamic motion scaling has great potential for the intelligent and accurate control of neurosurgical robots.

ABBREVIATIONS AI = artificial intelligence; Thh = higher threshold; Thl = lower threshold.

OBJECTIVE

Advanced and intelligent robotic control is necessary for neurosurgical robots, which require great accuracy and precision. In this article, the authors propose methods for dynamically and automatically controlling the motion-scaling ratio of a master-slave neurosurgical robotic system to reduce the task completion time.

METHODS

Three dynamic motion-scaling modes were proposed and compared with the conventional fixed motion-scaling mode. These 3 modes were defined as follows: 1) the distance between a target point and the tip of the slave manipulator, 2) the distance between the tips of the slave manipulators, and 3) the velocity of the master manipulator. Five test subjects, 2 of whom were neurosurgeons, sutured 0.3-mm artificial blood vessels using the MM-3 neurosurgical robot in each mode.

RESULTS

The task time, total path length, and helpfulness score were evaluated. Although no statistically significant differences were observed, the mode using the distance between the tips of the slave manipulators improves the suturing performance.

CONCLUSIONS

Dynamic motion scaling has great potential for the intelligent and accurate control of neurosurgical robots.

ABBREVIATIONS AI = artificial intelligence; Thh = higher threshold; Thl = lower threshold.

Technologies for surgical robots have been developed mainly in the field of laparoscopic surgery, and more advanced robotic technologies are necessary for neurosurgical robots, which require greater accuracy and safety. To this end, we have been developing the neurosurgical robot MM-3, which has been designed to assist surgeons in performing complex microsurgical tasks.7 The MM-3 robotic system uses the so-called master-slave control, in which the hand motion of the operator (i.e., surgeon) is measured with master manipulator(s), and the motion is scaled down and then replicated by the slave manipulator(s). The motion-scaling ratio—the ratio of the slave robot’s motion to the corresponding master manipulator’s motion—needs to be small for accuracy in precise tasks. However, a small motion-scaling ratio can prolong the task completion time and increase the physical and mental burdens of the operator, as a larger hand motion is required. In this paper, we propose a method to dynamically and automatically control the motion-scaling ratio by recognizing the intention of the operator. In other words, the robot’s velocity would automatically decrease when the robotic system recognizes that the operator is initiating a precise task, such as needle driving or looping, while the velocity would automatically increase for a coarse otion.

Methods

Microsurgical Robotic System MM-3 and Its Control

The MM-3 is a prototype robot being developed by the University of Tokyo, and access is limited to those who are involved in the research. In the MM-3 system, the operator moves a pair of master manipulators while seeing a stereo-microscopic view and bimanually controls the slave manipulators, as shown in Fig. 1.

Fig. 1.
Fig. 1.

Photographs of the master-slave microsurgical robot MM-3.

A fixed motion-scaling ratio of 0.33 had been used previously for anastomoses of 0.3-mm artificial blood vessels. However, the task completion time for this procedure was much longer than that for a manual operation. Therefore, we proposed a method to dynamically and automatically modulate the motion-scaling ratio during a task.5 Preliminary experiments were carried out by engineering students, who concluded that dynamic motion scaling had great potential.4 This article proposes 4 motion-scaling modes: fixed, target, slave, and velocity as demonstrated in Video 1. The performance of these 4 modes was evaluated by neurosurgeons and engineering students.

VIDEO 1. Video clip showing the master-slave system of the MM-3 robot in use. Copyright University of Tokyo. Published with permission. Click here to view.

Fixed Mode

This is the conventional mode used in most existing surgical robots. The motion-scaling ratio is preset and remains unchanged during a task.

Target Mode

The target mode allows for dynamic motion scaling using the distance between a target and the tip of the slave manipulator. A smaller motion-scaling ratio is used when the distance between a predefined target (e.g., the edge of a blood vessel) and the tip of the slave manipulator is smaller. This mode assumes that the operator wants to carry out a more precise task when the robot is closer to the target.

Slave Mode

The slave mode provides dynamic motion scaling using the distance between the tips of the slave manipulators. A smaller motion-scaling ratio is used when the distance between the tips of the slave manipulators is smaller. This mode assumes that the operator wants to perform a more precise task when the slave manipulators approach each other. This mode applies only to a bimanual task, and the same motion-scaling ratio is used for both slave manipulators.

Velocity Mode

The velocity mode allows for dynamic motion scaling using the velocity of the master manipulator. A smaller motion-scaling ratio is used when the velocity of the master manipulator is slower. This mode assumes that the operator wants to perform a task more precisely when slowing down his/her hand motion.

Experimental Methods

The experimental task was to suture 0.3-mm artificial blood vessels made of silicone (Microvascular Practice Card, Muranaka Medical Instruments Co.) with a 12-0 surgical suture (10V43–10R, Muranaka Medical Instruments) incorporating 1 stitch and 1 knot. The target was set at the middle of the 2 artificial blood vessels to be sutured. The motion-scaling ratio for the fixed mode was set as 0.33. The lower and higher thresholds (Thl and Thh, respectively) for the control in the target, slave, and velocity modes were defined as Thl_Target (mm), Thh_Target (mm), Thl_Slave (mm), Thh_Slave (mm), Thl_Velocity (mm/sec), and Thh_Velocity (mm/sec). The motion-scaling ratio was modulated linearly from 0.2 to 0.5, as shown in Fig. 2. Each threshold was determined by analyzing the distances between the target and the tips of the slave manipulators, the distance between the tips of the slave manipulators, and the velocities of the master manipulators during a pair of suturing tasks performed by 2 engineering students who were experienced in using the MM-3 robot.

Fig. 2.
Fig. 2.

Modulation of the motion-scaling ratio.

Then, 2 neurosurgeons and 3 engineering students were asked to perform the same suturing task in each mode in random order. The time from needle insertion to the completion of knot tying and the total path length of the right and left master manipulators were measured. The total path length is equal to the total path length of the right and left hands of the operator; a larger path length indicates inefficiency in manipulation and resultant physical fatigue. Additionally, a survey was conducted to evaluate the mental burden on the operator. The helpfulness score was designated on a scale of 1 to 5 points as follows: 1, very frustrated; 2, frustrated; 3, normal; 4, helpful; and 5, very helpful. All participants were asked to score each mode immediately after performing each task. The results were analyzed using the Wilcoxon rank-sum test with JMP statistical software (SAS Institute, Inc.); p < 0.05 was deemed statistically significant.

Results

The distances between the target and the tips of the slave manipulators varied in a range from 0 mm to approximately 4 mm in the needle-driving and looping tasks, which require very great precision, whereas the distance increased to more than 25 mm in the thread-pulling task (i.e., pulling the thread after needle extraction and before the looping task), which does not need precision. Therefore, the thresholds were determined as Thl_Target = 4 mm and Thh_Target = 6 mm, to provide a stable fixed motion during the precise tasks and prevent a sudden change in motion scaling during a coarse motion. The distance between the tips of the slave manipulators varied from 1 mm to approximately 4 mm in the needle-driving task and from 1 mm to approximately 3 mm in the looping task. The distance increased to more than 25 mm in the thread-pulling task. Accordingly, the thresholds were determined as Thl_Slave = 4 mm and Thh_Slave = 6 mm. The velocities of the master manipulators were slower than 50 mm/sec for most of the time in the needle-driving and looping tasks. Therefore, the thresholds were determined to prevent sudden changes in coarse motion as Thl_Velocity = 50 mm/sec and Thh_Velocity = 100 mm/sec. The suturing experiment was conducted with these values.

Figure 3 shows the time and total path length for the 5 subjects, and the details of the results including the helpfulness score are shown in Tables 13. No statistically significant difference was observed when comparing each pair of the modes for each metric.

Fig. 3.
Fig. 3.

Experimental results: time (left) and total path length (right).

TABLE 1.

Time in seconds to complete tasks with various motion control modes

Subject No.Fixed   
111711411493
296969686
31057676136
4147136136173
59910410483
Mean112.8105.2105.2114.2
SD18.619.819.835.1
A longer task completion time indicates inefficiency in manipulation.
TABLE 2.

Total path length in millimeters to complete tasks with various motion control modes

Subject No.FixedTargetSlaveVelocity
11913.32293.61294.0977.9
21130.31356.81373.9988.0
3905.0681.9734.61022.3
41325.31171.91896.61726.7
5801.51046.1754.8584.0
Mean1215.11310.11210.81059.8
SD393.5539.1433.3370.0
The total path length of the right and left master manipulators is equal to the total path length of the right and left hands of the operator. A larger path length indicates inefficiency in manipulation and resultant physical fatigue.
TABLE 3.

Helpfulness score in completing tasks with various motion control modes

Subject No.FixedTargetSlaveVelocity
14332
22233
33343
43343
53334
The score indicates the mental burden on the operator. The scores are as follows: 1, very frustrating; 2, frustrating; 3, normal; 4, helpful; and 5, very helpful.

Discussion

Navigation-based neurosurgical robots have been used clinically. The history of neurosurgical robots was summarized by Smith and associates.8 Master-slave neurosurgical robots, such as NeuRobot,2 Neuromate,9 and our MM-3, are designed to assist surgeons in performing complicated tasks, but further technological advances are needed before they can be widely used in a clinical setting. Neurosurgical master-slave robots must be designed with stricter parameters than laparoscopic surgical robots. Specifically, the small workspace, small microscopic view field, greater precision, and instruments with a small diameter are all challenges for researchers in engineering, and the state-of-the-art laparoscopic robotic technologies cannot simply be transferred to neurosurgical robots. The implementation of intelligent functions in master-slave neurosurgical robots would propel advances in robotic neurosurgery.

Theoretically, there should be great advantages to the proposed dynamic motion-scaling methods, but we did not observe any statistical significance in the experimental results. This outcome was partially because the number of experiment participants was limited. The MM-3 robot is still under development, and only researchers who have been involved in the development are capable of performing robotic microsuturing. The participants in the experiment were qualified, as all of them were capable of suturing 0.3-mm artificial blood vessels using the MM-3 robot. The task was also designed to be simple so that the total experiment time for each participant would be less than 30 minutes. The time needed for the thread-pulling task, which should benefit most from this dynamic motion scaling, accounted for less than 10% of the total time needed for the suturing incorporating 1 stitch and 1 knot. The dynamic motion-scaling modes would have a greater impact if they were applied to a task requiring more frequent coarse motions. Surgical usefulness will be evaluated in the future with a more complex and difficult task performed by neurosurgeons who were not involved in the development.

Although there was no statistically significant difference, the average times for target mode and slave mode both decreased 6.7% from the conventional fixed mode, while the time increased 1.2% for the velocity mode. The average total path length for target mode increased 7.8%, but the average total path lengths for slave and velocity modes decreased by 0.3% and 12.8%, respectively. According to these results, the slave mode would be the most useful. Moreover, none of the test subjects experienced frustration with this mode, as the result of the helpfulness score shows. Further investigations will better consider the experimental tasks.

The partial automation of surgical robots is currently being intensely discussed in several engineering communities. The automation of dynamic motion scaling is a new and original proposal that is trying to incorporate intelligence into the neurosurgical robot. Compared with other automated controls of surgical robots, such as in fully automated suturing,6 the responsibility of the operator is greater in our scenario, which is an advantage considering regulatory issues.

Our approach is a step toward artificial intelligence (AI)–based robotic surgery, and this paper describes possible methods to associate a surgeon’s hand motion with the intended motion accuracy. Once the learning process is implemented, the methods will be sophisticated, leading to an autonomous and real-time customization of robotic control parameters. The next step is to collect data from board-certified neurosurgeons and assimilate the data through machine-learning algorithms to extract and implement skills in the robot. Even a young and inexperienced surgeon would then be able to perform skilled microsurgical tasks in AI-based robotic surgery. To advance AI-based robotic microsurgery, we have been studying assessments of neurosurgical skill,3 autonomous recognition of surgical tools in a microscopic view,1 and autonomous recognition of microsurgical processes. Regulatory issues must also be discussed with authorities because AI-based surgical robots can be customized to each operator and is capable of evolving after their approval.

Conclusions

Dynamic motion scaling has great potential for the intelligent and accurate control of neurosurgical robots. Dynamic motion scaling based on the distance between the tips of the slave manipulators could be the most useful, but further investigations are needed.

Acknowledgments

This work was partially supported by the ImPACT Program of the Council for Science, Technology and Innovation (Cabinet Office, Government of Japan) and the Grants-in-Aid for Scientific Research S 23226006 of the Japan Society for the Promotion of Science (JSPS). We thank Mr. Miyamoto and Mr. Ikemoto of the School of Engineering, the University of Tokyo, for their technical help, and Ms. Julie Yamamoto for editorial and language assistance.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Morita, Ko, Nakazawa, Kurose, Harada, Sora, Shono, Nakatomi. Acquisition of data: Ko, Nakazawa, Kurose, Shono, Nakatomi. Analysis and interpretation of data: Morita, Ko, Nakazawa, Kurose, Harada, Shono, Nakatomi. Drafting the article: Morita, Ko, Harada. Critically revising the article: Morita, Harada. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Morita. Statistical analysis: Harada. Administrative/technical/material support: Ko, Nakazawa, Kurose, Harada, Sora. Study supervision: Morita, Harada, Mitsuishi, Sora, Shono, Nakatomi, Saito.

Supplemental Information

References

  • 1

    Baek YMTanaka SHarada KSugita NMorita ASora S: Robust visual tracking of robotic forceps under a microscope using kinematic data fusion. IEEE/ASME Trans Mechatron 19:2782882014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2

    Goto THongo KKakizawa YMuraoka HMiyairi YTanaka Y: Clinical application of robotic telemanipulation system in neurosurgery. Case report. J Neurosurg 99:108210842003

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Harada KMorita AMinakawa YBaek YMSora SSugita N: Assessing microneurosurgical skill with medico-engineering technology. World Neurosurg 84:9649712015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Ko SNakazawa AKurose YHarada KShono NSaito N: Comparison of dynamic motion scaling methods for master-slave microsurgical robotic system in Twelfth Asian Conference on Computer-Aided Surgery (ACCAS2016). Seoul: ACCAS2016 pp 140141

    • Search Google Scholar
    • Export Citation
  • 5

    Ko SNakazawa ATanaka SNukariya HHarada KMitsuishi M: Dynamic motion scaling for master–slave microsurgical robotic system and its preliminary evaluation in Proceedings of the 6th International Conference on Advanced Mechatronics (ICAM2016). Tokyo: Japan Society of Mechanical Engineers2015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6

    Liu TÇavuşoğlu MC: Needle grasp and entry port selection for automatic execution of suturing tasks in robotic minimally invasive surgery. IEEE Trans Autom Sci Eng 13:5525632016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7

    Mitsuishi MMorita ASugita NSora SMochizuki RTanimoto K: Master-slave robotic platform and its feasibility study for micro-neurosurgery. Int J Med Robot 9:1801892013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8

    Smith JAJivraj JWong RYang V: 30 years of neurosurgical robots: review and trends for manipulators and associated navigational systems. Ann Biomed Eng 44:8368462016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    von Langsdorff DPaquis PFontaine D: In vivo measurement of the frame-based application accuracy of the Neuromate neurosurgical robot. J Neurosurg 122:1911942015

    • Crossref
    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Contributor Notes

Correspondence Akio Morita, Department of Neurological Surgery, Nippon Medical School, 1-1-5, Sendagi, Bunkyo-ku, Tokyo 113-8603, Japan. email: amor-tky@umin.ac.jp.INCLUDE WHEN CITING DOI: 10.3171/2017.2.FOCUS16568.Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.
Headings
Figures
References
  • 1

    Baek YMTanaka SHarada KSugita NMorita ASora S: Robust visual tracking of robotic forceps under a microscope using kinematic data fusion. IEEE/ASME Trans Mechatron 19:2782882014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2

    Goto THongo KKakizawa YMuraoka HMiyairi YTanaka Y: Clinical application of robotic telemanipulation system in neurosurgery. Case report. J Neurosurg 99:108210842003

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Harada KMorita AMinakawa YBaek YMSora SSugita N: Assessing microneurosurgical skill with medico-engineering technology. World Neurosurg 84:9649712015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Ko SNakazawa AKurose YHarada KShono NSaito N: Comparison of dynamic motion scaling methods for master-slave microsurgical robotic system in Twelfth Asian Conference on Computer-Aided Surgery (ACCAS2016). Seoul: ACCAS2016 pp 140141

    • Search Google Scholar
    • Export Citation
  • 5

    Ko SNakazawa ATanaka SNukariya HHarada KMitsuishi M: Dynamic motion scaling for master–slave microsurgical robotic system and its preliminary evaluation in Proceedings of the 6th International Conference on Advanced Mechatronics (ICAM2016). Tokyo: Japan Society of Mechanical Engineers2015

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 6

    Liu TÇavuşoğlu MC: Needle grasp and entry port selection for automatic execution of suturing tasks in robotic minimally invasive surgery. IEEE Trans Autom Sci Eng 13:5525632016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 7

    Mitsuishi MMorita ASugita NSora SMochizuki RTanimoto K: Master-slave robotic platform and its feasibility study for micro-neurosurgery. Int J Med Robot 9:1801892013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8

    Smith JAJivraj JWong RYang V: 30 years of neurosurgical robots: review and trends for manipulators and associated navigational systems. Ann Biomed Eng 44:8368462016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    von Langsdorff DPaquis PFontaine D: In vivo measurement of the frame-based application accuracy of the Neuromate neurosurgical robot. J Neurosurg 122:1911942015

    • Crossref
    • Search Google Scholar
    • Export Citation
TrendMD
Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 328 200 25
PDF Downloads 340 160 14
EPUB Downloads 0 0 0
PubMed
Google Scholar