MiroSurgeAdvanced User Interaction Modalities in Minimally Invasive Robotic Surgery

This paper presents MiroSurge, a telepresence system for minimally invasive surgery developed at the German Aerospace Center (DLR), and introduces MiroSurge's new user interaction modalities: (1) haptic feedback with software-based preservation of the fulcrum point, (2) an ultrasound-based approach to the quasi-tactile detection of pulsating vessels, and (3) a contact-free interface between surgeon and telesurgery system, where stereo vision is augmented with force vectors at the tool tip. All interaction modalities aim to increase the user's perception beyond stereo imaging by either augmenting the images or by using haptic interfaces. MiroSurge currently provides surgeons with two different interfaces. The first option, bimanual haptic interaction with force and partial tactile feedback, allows for direct perception of the remote environment. Alternatively, users can choose to control the surgical instruments by optically tracked forceps held in their hands. Force feedback is then provided in augmented stereo images by constantly updated force vectors displayed at the centers of the teleoperated instruments, regardless of the instruments' position within the video image. To determine the centerpoints of the instruments, artificial markers are attached and optically tracked. A new approach to detecting pulsating vessels beneath covering tissue with an omnidirectional ultrasound Doppler sensor is presented. The measurement results are computed and can be provided acoustically (by displaying the typical Doppler sound), optically (by augmenting the endoscopic video stream), or kinesthetically (by a gentle twitching of the haptic input devices). The control structure preserves the fulcrum point in minimally invasive surgery and user commands are followed by the surgical instrument. Haptic feedback allows the user to distinguish between interaction with soft and hard environments. The paper includes technical evaluations of the features presented, as well as an overview of the system integration of MiroSurge.

[1]  Christopher R. Wagner,et al.  Mechanisms of performance enhancement with force feedback , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[2]  Colin Deane.  Ultrasonography in Vascular Diagnosis. A Therapy-Oriented Textbook and Atlas , 2012 .

[3]  D G Jayne,et al.  Robotic platforms for general and colorectal surgery , 2011, Colorectal disease : the official journal of the Association of Coloproctology of Great Britain and Ireland.

[4]  Robert D. Howe,et al.  Tactile tracking of arteries in robotic surgery , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[5]  D. Yuh,et al.  Effect of sensory substitution on suture-manipulation forces for robotic surgical systems. , 2005, The Journal of thoracic and cardiovascular surgery.

[6]  A. Castellanos,et al.  Force Feedback Plays a Significant Role in Minimally Invasive Surgery: Results and Analysis , 2005, Annals of surgery.

[7]  Susan J. Lederman,et al.  Design and Performance of a Tactile Shape Display Using RC Servomotors (Short Paper) , 2004 .

[8]  B. Hannaford,et al.  Force controlled and teleoperated endoscopic grasper for minimally invasive surgery-experimental performance evaluation , 1999, IEEE Transactions on Biomedical Engineering.

[9]  Alin Albu-Schäffer,et al.  The DLR MIRO: a versatile lightweight robot for surgical applications , 2008, Ind. Robot.

[10]  Gerd Hirzinger,et al.  Planning and control of a teleoperation system for research in minimally invasive robotic surgery , 2009, 2009 IEEE International Conference on Robotics and Automation.

[11]  M. Huang,et al.  Needlescopic, Laparoscopic, and Open Appendectomy: A Comparative Study , 2001, Surgical laparoscopy, endoscopy & percutaneous techniques.

[12]  Tobias Ortmaier,et al.  A hands-on-robot for accurate placement of pedicle screws , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[13]  Gerd Hirzinger,et al.  Flexible Signal-Oriented Hardware Abstraction for Rapid Prototyping of Robotic Systems , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  R. Howe,et al.  A Palpation System for Artery Localization in Laparoscopic Surgery , 1994 .

[15]  B. Hoffmann,et al.  Actuator Array for Use in Minimally Invasive Surgery , 1997 .

[16]  Rajnikant V. Patel,et al.  Tool/tissue interaction feedback modalities in robot-assisted lump localization , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  Bernhard Kübler,et al.  Development of actuated and sensor integrated forceps for minimally invasive robotic surgery , 2006 .

[18]  Tobias Ortmaier,et al.  Telemanipulator for remote minimally invasive surgery , 2008, IEEE Robotics & Automation Magazine.

[19]  G Hirzinger,et al.  Development of actuated and sensor integrated forceps for minimally invasive robotic surger , 2005, The international journal of medical robotics + computer assisted surgery : MRCAS.

[20]  Alessandro De Luca,et al.  Friction observer and compensation for control of robots with joint torque measurement , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Christopher R. Wagner,et al.  A compliant tactile display for teletaction , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[22]  S. Sauerland,et al.  Laparoscopic versus open surgery for suspected appendicitis. , 2002, The Cochrane database of systematic reviews.

[23]  Alois Knoll,et al.  Haptic Feedback in a Telepresence System for Endoscopic Heart Surgery , 2007, PRESENCE: Teleoperators and Virtual Environments.

[24]  M. Helmy.  A comparative study between laparoscopic versus open appendicectomy in men. , 2001, Journal of the Egyptian Society of Parasitology.

[25]  Gerd Hirzinger,et al.  Prototypic Force Feedback Instrument for Minimally Invasive Robotic Surgery , 2008 .

[26]  Miguel L Franco,et al.  An integrated pneumatic tactile feedback actuator array for robotic surgery , 2009, The international journal of medical robotics + computer assisted surgery : MRCAS.

[27]  Berthold Bäuml,et al.  Agile Robot Development (aRD): A Pragmatic Approach to Robotic Software , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Gerd Hirzinger,et al.  DLR MiroSurge -- towards versatility in surgical robotics , 2008 .

[29]  Alin Albu-Schäffer,et al.  DLR MiroSurge: a versatile system for research in endoscopic telesurgery , 2010, International Journal of Computer Assisted Radiology and Surgery.

[30]  Alin Albu-Schäffer,et al.  MIMO State Feedback Controller for a Flexible Joint Robot with Strong Joint Coupling , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.