Integration of new features for telerobotic surgery into the MiroSurge system

Minimally invasive robotic surgery has gained wide acceptance recently. Computer-aided features to assist the surgeon during these interventions may help to develop safer, faster, and more accurate procedures. Especially physiological motion compensation of the beating heart and online soft tissue modelling are promising features that were developed recently. This paper presents the integration of these new features into the minimally invasive robotic surgery platform MiroSurge. A central aim of this research platform is to enable evaluation and comparison of new functionalities for minimally invasive robotic surgery. The system structure of MiroSurge is presented as well as the interfaces for the new functionalities. Some details about the modules for motion tracking and for soft tissue simulation are given. Results are shown with an experimental setup that includes a heart motion simulator and dedicated silicone organ models. Both features are integrated seamlessly and work reliably in the chosen setup. The MiroSurge platform thus shows the potential to provide valuable results in evaluating new functionalities for minimally invasive robotic surgery.

[1]  Gerd Hirzinger,et al.  Planning and control of a teleoperation system for research in minimally invasive robotic surgery , 2009, 2009 IEEE International Conference on Robotics and Automation.

[2]  Guang-Zhong Yang,et al.  i-BRUSH: A Gaze-Contingent Virtual Paintbrush for Dense 3D Reconstruction in Robotic Assisted Surgery , 2009, MICCAI.

[3]  Ming-Hsuan Yang,et al.  A direct method for modeling non-rigid motion with thin plate spline , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  S.S. Sastry,et al.  Control algorithms for active relative motion cancelling for robotic assisted off-pump coronary artery bypass graft surgery , 2005, ICAR '05. Proceedings., 12th International Conference on Advanced Robotics, 2005..

[5]  Philippe Poignet,et al.  Motion Tracking for Beating Heart Surgery , 2011 .

[6]  Guang-Zhong Yang,et al.  Soft-Tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures , 2005, MICCAI.

[7]  Gregory D. Hager,et al.  Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation , 2004, MICCAI.

[8]  Markus H. Gross,et al.  Eurographics Symposium on Point-based Graphics (2005) a Unified Lagrangian Approach to Solid-fluid Animation , 2022 .

[9]  Francesca Pizzorni Ferrarese,et al.  An entropy based multi-thresholding method for semi-automatic segmentation of liver tumors , 2008, The MIDAS Journal.

[10]  Rainer Christian Konietschke,et al.  Planning of Workplaces with Multiple Kinematically Redundant Robots , 2007 .

[11]  Nitish V. Thakor,et al.  Surgical Motion Adaptive Robotic Technology (S.M.A.R.T): Taking the Motion out of Physiological Motion , 2001, MICCAI.

[12]  Tobias Ortmaier,et al.  Motion estimation in beating heart surgery , 2005, IEEE Transactions on Biomedical Engineering.

[13]  Ana Luisa Trejos,et al.  On the Feasibility of a Moving Support for Surgery on the Beating Heart , 1999, MICCAI.

[14]  Guang-Zhong Yang,et al.  Real-time intra-operative 3D tissue deformation recovery , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[15]  Alin Albu-Schäffer,et al.  MIMO State Feedback Controller for a Flexible Joint Robot with Strong Joint Coupling , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[16]  A. Costello,et al.  Outcomes after robot-assisted laparoscopic radical prostatectomy. , 2009, Asian journal of andrology.

[17]  Selim Benhimane,et al.  Real-time image-based tracking of planes using efficient second-order minimization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[18]  K. Bathe.  Finite Element Procedures , 1995 .

[19]  Philippe Poignet,et al.  Towards robust 3D visual tracking for motion compensation in beating heart surgery , 2011, Medical Image Anal..

[20]  Stefan Wesarg,et al.  VR-Based Simulators for Training in Minimally Invasive Surgery , 2007, IEEE Computer Graphics and Applications.

[21]  Alin Albu-Schäffer,et al.  DLR MiroSurge: a versatile system for research in endoscopic telesurgery , 2010, International Journal of Computer Assisted Radiology and Surgery.

[22]  Alessandro De Luca,et al.  Friction observer and compensation for control of robots with joint torque measurement , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Dinesh K. Pai,et al.  DyRT: dynamic response textures for real time deformation simulation with graphics hardware , 2002, SIGGRAPH.

[24]  Danail Stoyanov,et al.  A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery , 2005, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[25]  Rüdiger Westermann,et al.  Interactive Simulation of Deformable Bodies on GPUs , 2005, SimVis.

[26]  Russell H. Taylor,et al.  Medical robotics in computer-integrated surgery , 2003, IEEE Trans. Robotics Autom..

[27]  Kok-Meng Lee,et al.  An Adaptive Meshless Method for Modeling Large Mechanical Deformation and Contacts , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[28]  John Hu,et al.  Soft Tissue Deformation and Cutting Simulation for the Multimodal Surgery Training , 2006, 19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).

[29]  Philippe Poignet,et al.  Beating heart motion prediction for robust visual tracking , 2010, 2010 IEEE International Conference on Robotics and Automation.

[30]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[31]  Paolo Fiorini,et al.  Calibration of mass spring models for organ simulations , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Andrew Zisserman,et al.  Direct Estimation of Non-Rigid Registrations , 2004 .

[33]  Paolo Fiorini,et al.  Simulation of deformable environment with haptic feedback on GPU , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Markus H. Gross,et al.  Simulating facial surgery using finite element models , 1996, SIGGRAPH.

[35]  Adrien Bartoli,et al.  Generalized Thin-Plate SplineWarps , 2007, CVPR.

[36]  Uwe D. Hanebeck,et al.  Model-based Motion Estimation of Elastic Surfaces for Minimally Invasive Cardiac Surgery , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[37]  Thomas Sangild Sørensen,et al.  A GPU accelerated spring mass system for surgical simulation. , 2005, Studies in health technology and informatics.

[38]  Tobias Ortmaier,et al.  Tracking local motion on the beating heart , 2002, SPIE Medical Imaging.

[39]  Gerd Hirzinger,et al.  Planning and Real Time Control of a Minimally Invasive Robotic Surgery System , 2009 .

[40]  W. Eric L. Grimson,et al.  Volumetric object modeling for surgical simulation , 1998, Medical Image Anal..

[41]  Grigore C. Burdea,et al.  Human performance using virtual reality tumor palpation simulation , 1997, Comput. Graph..

[42]  Tobias Ortmaier,et al.  Accuracy Identification of Markerless Registration with the DLR Handheld 3D-Modeller in Medical Applications , 2007 .

[43]  Guang-Zhong Yang,et al.  Stabilization of Image Motion for Robotic Assisted Beating Heart Surgery , 2007, MICCAI.

[44]  Guang-Zhong Yang,et al.  Belief Propagation for Depth Cue Fusion in Minimally Invasive Surgery , 2008, MICCAI.

[45]  Markus H. Gross,et al.  Particle-based fluid-fluid interaction , 2005, SCA '05.

[46]  Tobias Ortmaier,et al.  Remote Minimally Invasive Surgery , 2008 .

[47]  Ezio Malis.  An efficient unified approach to direct visual tracking of rigid and deformable surfaces , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[48]  Pheng-Ann Heng,et al.  A hybrid condensed finite element model with GPU acceleration for interactive 3D soft tissue cutting: Research Articles , 2004 .

[49]  Adrien Bartoli,et al.  Direct Estimation of Non-Rigid Registration , 2004, BMVC.

[50]  Geraldo F. Silveira,et al.  Real-time Visual Tracking under Arbitrary Illumination Changes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Tobias Ortmaier,et al.  Potential, Limitations and Challenges of Markerless Registration with the DLR 3D-Modeller in Medical Applications. , 2007 .

[52]  Tobias Ortmaier,et al.  Telemanipulator for remote minimally invasive surgery , 2008, IEEE Robotics & Automation Magazine.

[53]  Tobias Ortmaier,et al.  Reconstruction of Image Structure in Presence of Specular Reflections , 2001, DAGM-Symposium.

[54]  Chao Liu,et al.  Three-dimensional Motion Tracking for Beating Heart Surgery Using a Thin-plate Spline Deformable Model , 2010, Int. J. Robotics Res..