Cooperative Control of Bimanual Continuum Robots for Automated Knot-Tying in Robot-Assisted Surgical Suturing

Enoch Quaicoe, Ayman Nada, Hiroyuki Ishii, Haitham El-Hussieny

Abstract


Knot-tying, a crucial yet intricate surgical task, remains a challenge in Robot-assisted Minimally Invasive Surgery (RAMIS) performed under teleoperation. While existing studies on automated knot-tying mostly focus on rigid-link robots, whose dexterity, adaptability, and inherent safety in RAMIS are outperformed by continuum robots, this research takes a novel approach by developing a unique cooperative control scheme for bimanual continuum robots, specifically designed for automated knot-tying tasks in RAMIS. We meticulously plan two effective knot-tying trajectory scenarios and develop the cooperative control scheme for the bimanual continuum robots, leveraging the well-known Jacobian transpose kinematic algorithms to ensure their precise and collaborative knot-tying trajectory tracking performance. The control scheme incorporates a switching mechanism to guarantee the robots’ collaboration and synchronous operation during the knot-tying trajectory tracking process. The effectiveness of our cooperative control scheme is illustrated through simulation studies using MATLAB/Simulink in terms of trajectory tracking performance. Meanwhile, ten Monte Carlo simulations are conducted to analyze the system’s robustness against pulse disturbances that could occur in surgical settings. All ten simulations returned similar error values despite the increasing disturbance levels applied. The results not only demonstrate the seamless collaboration and synchronous operation of the bimanual continuum robots in precisely tracking the pre-planned knot-tying trajectories with errors less than 0.0017 m but also highlight the stability, effective tuning and robustness of our cooperative control system against pulse disturbances. This study demonstrates precision, robustness, and autonomy in bimanual continuum robotic knottying in RAMIS, promising safe robot-patient interaction and reduced surgeon workload and surgery time.

Keywords


Continuum Robots; Automated Suture Knot-Tying; Surgical Suturing; Robot-Assisted Minimally Invasive Surgery; Trajectory Planning; Kinematic Modeling; Cooperative Control.

Full Text:

PDF

References


P. E. Dupont et al., “A decade retrospective of medical robotics research from 2010 to 2020,” Science robotics, vol. 6, no. 60, 2021, doi: 10.1126/scirobotics.abi8017.

K. Kawashima, T. Kanno, and K. Tadano, “Robots in laparoscopic surgery: current and future status,” BMC Biomedical Engineering, vol. 1, no. 12, pp. 1–6, 2019, doi: 10.1186/s42490-019-0012-1.

K. R. Sheth and C. J. Koh, “The future of robotic surgery in pediatric urology: Upcoming technology and evolution within the field,” Frontiers in Pediatrics, vol. 7, p. 259, 2019, doi: 10.3389/fped.2019.00259.

T. Haidegger and J. Sandor, “Robot-assisted minimally invasive surgery ´ in the age of surgical data science,” Magyar Sebeszet, vol. 74, no. 4, pp. 127–135, 2021, doi: 10.1556/1046.74.2021.4.5.

J. Klodmann et al., “An introduction to robotically assisted surgical systems: Current developments and focus areas of research,” Current Robotics Reports, vol. 2, no. 3, pp. 321–332, 2021, doi: 10.1007/s43154- 021-00064-3.

G. Dagnino and D. Kundrat, “Robot-assistive minimally invasive surgery: trends and future directions,” International Journal of Intelligent Robotics and Applications, pp. 1–15, 2024, doi: 10.1007/s41315-024-00341-2.

P. Probst, “A review of the role of robotics in surgery: To davinci and beyond!,” Missouri Medicine, vol. 120, no. 5, pp. 389–396, 2023.

L. Cao et al., “Sewing up the wounds: A robotic suturing system for flexible endoscopy,” IEEE Robotics & Automation Magazine, vol. 27, no. 3, pp. 45–54, 2020, doi: 10.1109/MRA.2019.2963161.

A. Attanasio, B. Scaglioni, E. De Momi, P. Fiorini, and P. Valdastri, “Autonomy in surgical robotics,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, pp. 651–679, 2021, doi: 10.1146/annurevcontrol-062420-090543.

M. M. Marinho, K. Harada, A. Morita, and M. Mitsuishi, “Smartarm: Integration and validation of a versatile surgical robotic system for constrained workspaces,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 16, no. 2, 2020, doi: 10.1002/rcs.2053.

M. M. Marinho, K. Harada, K. Deie, T. Ishimaru, and M. Mitsuishi, “Smartarm: Suturing feasibility of a surgical robotic system on a neonatal chest model,” IEEE Transactions on Medical Robotics and Bionics, vol. 3, no. 1, pp. 253–256, 2021, doi: 10.1109/TMRB.2021.3049878.

Y. Hu, W. Li, L. Zhang, and G.-Z. Yang, “Designing, prototyping, and testing a flexible suturing robot for transanal endoscopic microsurgery,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1669–1675, 2019, doi: 10.1109/LRA.2019.2896883.

A. Ehrampoosh, B. Shirinzadeh, J. Pinskier, J. Smith, R. Moshinsky, and Y. Zhong, “A force-feedback methodology for teleoperated suturing task in robotic-assisted minimally invasive surgery,” Sensors, vol. 22, no. 20, p. 7829, 2022, doi: 10.3390/s22207829.

I. El Rassi and J.-M. El Rassi, “A review of haptic feedback in teleoperated robotic surgery,” Journal of medical engineering & technology, vol. 44, no. 5, pp. 247–254, 2020, doi: 10.1080/03091902.2020.1772391.

R. V. Patel, S. F. Atashzar, and M. Tavakoli, “Haptic feedback and forcebased teleoperation in surgical robotics,” Proceedings of the IEEE, vol. 110, no. 7, pp. 1012–1027, 2022, doi: 10.1109/JPROC.2022.3180052.

E. Abdi, D. Kulic, and E. Croft, “Haptics in teleoperated medical ´ interventions: Force measurement, haptic interfaces and their influence on user’s performance,” IEEE Transactions on Biomedical Engineering, vol. 67, no. 12, pp. 3438–3451, 2020, doi: 10.1109/TBME.2020.2987603.

D. Zhang, W. Si, W. Fan, Y. Guan, and C. Yang, “From teleoperation to autonomous robot-assisted microsurgery: A survey,” Machine Intelligence Research, vol. 19, no. 4, pp. 288–306, 2022, doi: 10.1007/s11633-022- 1332-5.

T. Haidegger, “Autonomy for surgical robots: Concepts and paradigms,” IEEE Transactions on Medical Robotics and Bionics, vol. 1, no. 2, pp. 65–76, 2019, doi: 10.1109/TMRB.2019.2913282.

A. A. Gumbs et al., “Artificial intelligence surgery: How do we get to autonomous actions in surgery?” Sensors, vol. 21, no. 16, p. 5526, 2021, doi: 10.3390/s21165526.

F. Richter et al., “Autonomous robotic suction to clear the surgical field for hemostasis using image-based blood flow detection,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1383–1390, 2021, doi: 10.1109/LRA.2021.3056057.

A. Attanasio et al., “Autonomous tissue retraction in robotic assisted minimally invasive surgery–a feasibility study,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6528–6535, 2020, doi: 10.1109/LRA.2020.3013914.

G. Fagogenis et al., “Autonomous robotic intracardiac catheter navigation using haptic vision,” Science robotics, vol. 4, no. 29, 2019, doi: 10.1126/scirobotics.aaw1977.

H. Saeidi et al., “Autonomous robotic laparoscopic surgery for intestinal anastomosis,” Science robotics, vol. 7, no. 62, 2022, doi: 10.1126/scirobotics.abj2908.

S. A. Pedram et al., “Autonomous suturing framework and quantification using a cable-driven surgical robot,” IEEE Transactions on Robotics, vol. 37, no. 2, pp. 404–417, 2020, doi: 10.1109/TRO.2020.3031236.

T. D. Nagy and T. Haidegger, “A dvrk-based framework for surgical subtask automation,” Acta Polytechnica Hungarica, vol. 16, no. 8, pp. 61–78, 2019, doi: 10.12700/aph.16.8.2019.8.5.

H. Su, A. Mariani, S. E. Ovur, A. Menciassi, G. Ferrigno, and E. De Momi, “Toward teaching by demonstration for robot-assisted minimally invasive surgery,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 484–494, 2021, doi: 10.1109/TASE.2020.3045655.

M. Wagner et al., “A learning robot for cognitive camera control in minimally invasive surgery,” Surgical Endoscopy, vol. 35, no. 9, pp. 5365– 5374, 2021, doi: 10.1007/s00464-021-08509-8.

B. Lu, X. Yu, J. Lai, K. Huang, K. C. Chan, and H. K. Chu, “A learning approach for suture thread detection with feature enhancement and segmentation for 3-d shape reconstruction,” IEEE Transactions on Automation Science and Engineering, vol. 17, no. 2, pp. 858–870, 2019, doi: 10.1109/TASE.2019.2950005.

B. Lu, H. K. Chu, K. Huang, and J. Lai, “Surgical suture thread detection and 3-d reconstruction using a model-free approach in a calibrated stereo visual system,” IEEE/ASME Transactions on Mechatronics, vol. 25, no. 2, pp. 792–803, 2019, doi: 10.1109/TMECH.2019.2942715.

B. Lu et al., “Toward image-guided automated suture grasping under complex environments: A learning-enabled and optimization-based holistic framework,” IEEE Transactions on Automation Science and Engineering, vol. 19, no. 4, pp. 3794–3808, 2021, doi: 10.1109/TASE.2021.3136185.

B. Lu, H. K. Chu, K. C. Huang, and L. Cheng, “Vision-based surgical suture looping through trajectory planning for wound suturing,” IEEE Transactions on Automation Science and Engineering, vol. 16, no. 2, pp. 542–556, 2019, doi: 10.1109/TASE.2018.2840532.

J. R. U. Roldan and D. Milutinovic, “Suture looping task pose planner ´ in a constrained surgical environment,” Journal of Intelligent & Robotic Systems, vol. 106, no. 78, 2022, doi: 10.1007/s10846-022-01772-4.

P. R. Koninckx, A. Ussia, A. Wattiez, W. Kondo, and A. Romeo, “Laparoscopic surgery: A systematic review of loop and knot security, varying with the suture and sequences, throws, rotation and destabilization of half-knots or half-hitches,” Journal of Clinical Medicine, vol. 12, no. 19, p. 6166, 2023, doi: 10.3390/jcm12196166.

A. Romeo, L. F. Fernandes, G. V. Cervantes, R. Botchorishvili, C. Benedetto, L. Adamyan, A. Ussia, A. Wattiez, W. Kondo, and P. R. Koninckx, “Which knots are recommended in laparoscopic surgery and how to avoid insecure knots,” Journal of Minimally Invasive Gynecology, vol. 27, no. 6, pp. 1395–1404, 2020, doi: 10.1016/j.jmig.2019.09.782.

X. Qin et al., “Beyond the square knot: A validation study for a novel knot-tying method named “inverse 9”,” Heliyon, vol. 9, no. 10, 2023, doi: 10.1016/j.heliyon.2023.e20673.

O. Yasa et al., “An overview of soft robotics,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 6, pp. 1–29, 2023, doi: 10.1146/annurev-control-062322-100607.

M. Runciman, A. Darzi, and G. P. Mylonas, “Soft robotics in minimally invasive surgery,” Soft robotics, vol. 6, no. 4, pp. 423–443, 2019, doi: 10.1089/soro.2018.0136.

O. M. Omisore, S. Han, J. Xiong, H. Li, Z. Li, and L. Wang, “A review on flexible robotic systems for minimally invasive surgery,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 52, no. 1, pp. 631–644, 2020, doi: 10.1109/TSMC.2020.3026174.

S. Kolachalama et al., “Continuum robots for manipulation applications: A survey,” Journal of Robotics, vol. 2020, no. 4187048, p. 19, 2020, doi: 10.1155/2020/4187048.

S. Aracri et al., “Soft robots for ocean exploration and offshore operations: A perspective,” Soft Robotics, vol. 8, no. 6, pp. 625–639, 2021, doi: 10.1089/soro.2020.0011.

G. Li et al., “Bioinspired soft robots for deep-sea exploration,” Nature Communications, vol. 14, no. 1, p. 7097, 2023, doi: 10.1038/s41467-023- 42882-3.

G. Li et al., “Self-powered soft robot in the mariana trench,” Nature, vol. 591, no. 7848, pp. 66–71, 2021, doi: 10.1038/s41586-020-03153-z.

M. M. Coad et al., “Vine robots,” IEEE Robotics & Automation Magazine, vol. 27, no. 3, pp. 120–132, 2019, doi: 10.1109/MRA.2019.2947538.

Y. Zhang and M. Lu, “A review of recent advancements in soft and flexible robots for medical applications,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 16, no. 3, 2020, doi: 10.1002/rcs.2096.

H. Alfalahi, F. Renda, and C. Stefanini, “Concentric tube robots for minimally invasive surgery: Current applications and future opportunities,” IEEE Transactions on Medical Robotics and Bionics, vol. 2, no. 3, pp. 410–424, 2020, doi: 10.1109/TMRB.2020.3000899.

W. Shen, G. Yang, T. Zheng, Y. Wang, K. Yang, and Z. Fang, “An accuracy enhancement method for a cable-driven continuum robot with a flexible backbone,” IEEE Access, vol. 8, pp. 37474–37481, 2020, doi: 10.1109/ACCESS.2020.2975087.

E. Amanov, T.-D. Nguyen, and J. Burgner-Kahrs, “Tendon-driven continuum robots with extensible sections—a model-based evaluation of path-following motions,” The International Journal of Robotics Research, vol. 40, no. 1, pp. 7–23, 2021, doi: 10.1177/0278364919886047.

M. T. Chikhaoui, S. Lilge, S. Kleinschmidt, and J. Burgner-Kahrs, “Comparison of modeling approaches for a tendon actuated continuum robot with three extensible segments,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 989–996, 2019, doi: 10.1109/LRA.2019.2893610.

S. Grazioso, G. Di Gironimo, and B. Siciliano, “A geometrically exact model for soft continuum robots: The finite element deformation space formulation,” Soft robotics, vol. 6, no. 6, pp. 790–811, 2019, doi: 10.1089/soro.2018.0047.

F. Janabi-Sharifi, A. Jalali, and I. D. Walker, “Cosserat rod-based dynamic modeling of tendon-driven continuum robots: A tutorial,” IEEE Access, vol. 9, pp. 68 703–68 719, 2021, doi: 10.1109/ACCESS.2021.3077186.

C. Della Santina and D. Rus, “Control oriented modeling of soft robots: the polynomial curvature case,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 290–298, 2019, doi: 10.1109/LRA.2019.2955936.

J. Lai, B. Lu, Q. Zhao, and H. K. Chu, “Constrained motion planning of a cable-driven soft robot with compressible curvature modeling,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4813–4820, 2022, doi: 10.1109/LRA.2022.3152318.

H. El-Hussieny, I. A. Hameed, and A. A. Nada, “Deep cnn-based static modeling of soft robots utilizing absolute nodal coordinate formulation,” Biomimetics, vol. 8, no. 8, p. 611, 2023, doi: 10.3390/biomimetics8080611.

A. Nada and H. El-Hussieny, “Development of inverse static model of continuum robots based on absolute nodal coordinates formulation for large deformation applications,” Acta Mechanica, pp. 1–23, 2023, doi: 10.1007/s00707-023-03814-w.

A. Amouri, “Investigation of the constant curvature kinematic assumption of a 2-dofs cable-driven continuum robot,” UPB Scientific Bulletin, Series D: Mechanical Engineering, no. 3, pp. 27–38, 2019.

T. Xu, J. Zhang, M. Salehizadeh, O. Onaizah, and E. Diller, “Millimeterscale flexible robots with programmable three-dimensional magnetization and motions,” Science Robotics, vol. 4, no. 29, 2019, doi: 10.1126/scirobotics.aav4494.

I. A. Seleem, H. El-Hussieny, and H. Ishii, “Recent developments of actuation mechanisms for continuum robots: A review,” International Journal of Control, Automation and Systems, vol. 21, pp. 1592–1609, 2023, doi: 10.1007/s12555-022-0159-8.

U. Gupta, L. Qin, Y. Wang, H. Godaba, and J. Zhu, “Soft robots based on dielectric elastomer actuators: A review,” Smart Materials and Structures, vol. 28, no. 10, p. 103002, 2019, doi: 10.1088/1361-665X/ab3a77.

J.-H. Youn et al., “Dielectric elastomer actuator for soft robotics applications and challenges,” Applied Sciences, vol. 10, no. 2, p. 640, 2020, doi: 10.3390/app10020640.

Y. Kim, G. A. Parada, S. Liu, and X. Zhao, “Ferromagnetic soft continuum robots,” Science Robotics, vol. 4, no. 33, 2019, doi: 10.1126/scirobotics.aax7329.

C. Della Santina, R. K. Katzschmann, A. Bicchi, and D. Rus, “Modelbased dynamic feedback control of a planar soft robot: trajectory tracking and interaction with the environment,” The International Journal of Robotics Research, vol. 39, no. 4, pp. 490–513, 2020, doi: 10.1177/0278364919897292.

X. Wang, Y. Li, and K.-W. Kwok, “A survey for machine learning-based control of continuum robots,” Frontiers in Robotics and AI, vol. 8, p. 730330, 2021, doi: 10.3389/frobt.2021.730330.

I. A. Seleem, H. El-Hussieny, S. F. Assal, and H. Ishii, “Development and stability analysis of an imitation learning-based pose planning approach for multi-section continuum robot,” IEEE Access, vol. 8, pp. 99366– 99379, 2020, doi: 10.1109/ACCESS.2020.2997636.

C. Della Santina, C. Duriez, and D. Rus, “Model-based control of soft robots: A survey of the state of the art and open challenges,” IEEE Control Systems Magazine, vol. 43, no. 3, pp. 30–65, 2023, doi: 10.1109/MCS.2023.3253419.

H. El-Hussieny, I. A. Hameed, and J.-H. Ryu, “Nonlinear model predictive growth control of a class of plant-inspired soft growing robots,” IEEE Access, vol. 8, pp. 214495–214503, 2020, doi: 10.1109/ACCESS.2020.3041616.

H. El-Hussieny, I. A. Hameed, and A. B. Zaky, “Plant-inspired soft growing robots: A control approach using nonlinear model predictive techniques,” Applied Sciences, vol. 13, no. 4, p. 2601, 2023, doi: 10.3390/app13042601.

I. A. Seleem, H. El-Hussieny, and H. Ishii, “Imitation-based path planning and nonlinear model predictive control of a multi-section continuum robots,” Journal of Intelligent & Robotic Systems, vol. 108, no. 9, pp. 1–13, 2023, doi: 10.1007/s10846-023-01811-8.

I. A. Seleem, S. F. Assal, H. Ishii, and H. El-Hussieny, “Guided pose planning and tracking for multi-section continuum robots considering robot dynamics,” IEEE Access, vol. 7, pp. 166690–166703, 2019, doi: 10.1109/ACCESS.2019.2953122.

I. A. Seleem, H. El-Hussieny, and H. Ishii, “Imitation-based motion planning and control of a multi-section continuum robot interacting with the environment,” IEEE Robotics and Automation Letters, vol. 8, no. 3, pp. 1351–1358, 2023, doi: 10.1109/LRA.2023.3239306.

J. Chen, J. Yan, Y. Qiu, H. Fang, J. Chen, and S. S. Cheng, “A crossentropy motion planning framework for hybrid continuum robots,” IEEE Robotics and Automation Letters, vol. 8, no. 12, pp. 8200–8207, 2023, doi: 10.1109/LRA.2023.3325777.

P. J. Sincak et al., “Sensing of continuum robots: A review,” Sensors, vol. 24, no. 4, p. 1311, 2024, doi: 10.3390/s24041311.

S. E. Navarro et al., “A model-based sensor fusion approach for force and shape estimation in soft robotics,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 5621–5628, 2020, doi: 10.1109/LRA.2020.3008120.

F. Xu, H. Wang, W. Chen, and Y. Miao, “Visual servoing of a cable-driven soft robot manipulator with shape feature,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4281–4288, 2021, doi: 10.1109/LRA.2021.3067285.

F. Campisano et al., “Teleoperation and contact detection of a waterjetactuated soft continuum manipulator for low-cost gastroscopy,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6427–6434, 2020, doi: 10.1109/LRA.2020.3013900.

A. Pore et al., “Autonomous navigation for robot-assisted intraluminal and endovascular procedures: A systematic review,” IEEE Transactions on Robotics, vol. 39, no. 4, pp. 2529–2548, 2023, doi: 10.1109/TRO.2023.3269384.

S. R. Ravigopal, T. A. Brumfiel, A. Sarma, and J. P. Desai, “Fluoroscopic image-based 3-d environment reconstruction and automated path planning for a robotically steerable guidewire,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11918–11925, 2022, doi: 10.1109/LRA.2022.3207568.

I. Tamadon et al., “Semiautonomous robotic manipulator for minimally invasive aortic valve replacement,” IEEE Transactions on Robotics, vol. 39, no. 6, pp. 4500–4519, 2023, doi: 10.1109/TRO.2023.3315966.

X. Huang, J. Zou, and G. Gu, “Kinematic modeling and control of variable curvature soft continuum robots,” IEEE/ASME Transactions on Mechatronics, vol. 26, no. 6, pp. 3175–3185, 2021, doi: 10.1109/TMECH.2021.3055339.

M. Zhao, F. Shi, T. Anzai, K. Okada, and M. Inaba, “Online motion planning for deforming maneuvering and manipulation by multilinked aerial robot based on differential kinematics,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 1602–1609, 2020, doi: 10.1109/LRA.2020.2967285.

H. Nigatu and D. Kim, “Optimization of 3-dof manipulators’ parasitic motion with the instantaneous restriction space-based analytic coupling relation,” Applied Sciences, vol. 11, no. 10, p. 4690, 2021, doi: 10.3390/app11104690.




DOI: https://doi.org/10.18196/jrc.v5i4.21617

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Enoch Quaicoe, Ayman Nada, Hiroyuki Ishii, Haitham El-Hussieny

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Journal of Robotics and Control (JRC)

P-ISSN: 2715-5056 || E-ISSN: 2715-5072
Organized by Peneliti Teknologi Teknik Indonesia
Published by Universitas Muhammadiyah Yogyakarta in collaboration with Peneliti Teknologi Teknik Indonesia, Indonesia and the Department of Electrical Engineering
Website: http://journal.umy.ac.id/index.php/jrc
Email: jrcofumy@gmail.com

Kuliah Teknik Elektro Terbaik