Motion Tracking for Minimally Invasive Robotic Surgery

Minimally invasive surgery is a modern surgical technique in which the instruments are inserted into the patient through small incisions. An endoscopic camera provides the view to the site of surgery inside the patient. While the patient benefits from strongly reduced tissue traumatisation, the surgeon has to cope with a number of disadvantages. These drawbacks arise from the fact that, in contrast to open surgery, direct contact and view to the field of surgery are lost in minimally invasive scenarios. A sophisticated robotic system can compensate for the increased demands posed to the surgeon and provide assistance for the complicated tasks. To enable the robotic system to provide particular assistance by partly autonomous tasks e.g. by guiding the surgeon to a preoperatively planned situs or by moving the camera along the changing focus of surgery, the knowledge of intraoperative changes inside the patient becomes important. Two main types of targets can be identified in endoscopic video images, which are instruments and organs. Depending on these types different strategies for motion tracking become advantageous. Tracking of image motion from endoscopic video images can be based solely on structure information provided by the object itself or can involve artifical landmarks to aid the tracking process. In the first case, the use of natural landmarks refers to the fact that the genuine structure of the target is used to find reference positions which can be tracked. This can involve intensity or feature based tracking strategies. In the second case of artifical landmarks, markers with a special geometry or colour can be used. This enables particular tracking strategies, making use of the distinctive property of these markers. This chapter describes different motion tracking strategies used to accomplish the task of motion detection in minimally invasive surgical environments. Two example scenario are provided for which two different motion tracking strategies have been successfully implemented. Both are partly autonomous task scenarios, providing automated camera guidance for laparoscopic surgery and motion compensation of the beating heart.

[1]  S. Hayati,et al.  A robot with improved absolute positioning accuracy for CT guided stereotactic brain surgery , 1988, IEEE Transactions on Biomedical Engineering.

[2]  Steven K. Feiner,et al.  Computer graphics: principles and practice (2nd ed.) , 1990 .

[3]  M. Moran.  Stationary and automated laparoscopically assisted technologies. , 1993, Journal of laparoendoscopic surgery.

[4]  Peter Corke,et al.  VISUAL CONTROL OF ROBOT MANIPULATORS – A REVIEW , 1993 .

[5]  Hurteau Hurteau,et al.  Laparoscopic surgery assisted by a robotic cameraman: concept and experimental results , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[6]  Peter Kazanzides,et al.  An image-directed robotic system for precise orthopaedic surgery , 1994, IEEE Trans. Robotics Autom..

[7]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Yuan-Fang Wang,et al.  Image analysis for automated tracking in robot-assisted endoscopic surgery , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[9]  Russell H. Taylor,et al.  A telerobotic assistant for laparoscopic surgery , 1995 .

[10]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Peter Kazanzides,et al.  An integrated system for cementless hip replacement , 1995 .

[12]  Demetri Terzopoulos,et al.  A dynamic finite element surface model for segmentation and tracking in multidimensional medical images with application to cardiac 4D image analysis. , 1995, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[13]  Laurent D. Cohen,et al.  Tracking Medical 3D Data with a Deformable Parametric Model , 1996, ECCV.

[14]  D J Turner,et al.  Solo surgery--with the aid of a robotic assistant. , 1996, Journal of telemedicine and telecare.

[15]  Demetri Terzopoulos,et al.  Deformable models in medical image analysis: a survey , 1996, Medical Image Anal..

[16]  G. Hirzinger,et al.  Real time visual servoing for laparoscopic surgery. , 1997 .

[17]  Lerner Ej.  Computer-integrated surgery. , 1997, New Jersey medicine : the journal of the Medical Society of New Jersey.

[18]  Gerd Hirzinger,et al.  Selbststeuernde farbcodierte Kamerafuehrung bei laparoskopischen Eingriffen. , 1997 .

[19]  W. Bargar,et al.  Primary and Revision Total Hip Replacement Using the Robodoc® System , 1998, Clinical orthopaedics and related research.

[20]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  F. Mohr,et al.  Computer-enhanced coronary artery bypass surgery. , 1999, The Journal of thoracic and cardiovascular surgery.

[22]  C. Detter,et al.  Early experience with robotic technology for coronary artery surgery. , 1999, The Annals of thoracic surgery.

[23]  F. Mohr,et al.  Endoscopic coronary artery bypass grafting on the beating heart using a computer enhanced telemanipulation system. , 1999, The heart surgery forum.

[24]  G. Hirzinger,et al.  Self-guided robotic camera control for laparoscopic surgery compared with human camera control. , 1999, American journal of surgery.

[25]  Tobias Ortmaier,et al.  Cartesian control issues for minimally invasive robot surgery , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[26]  W. Boyd,et al.  Closed-chest coronary artery bypass grafting on the beating heart with the use of a computer-enhanced surgical robotic system. , 2000, The Journal of thoracic and cardiovascular surgery.

[27]  John Kenneth Salisbury,et al.  The Intuitive/sup TM/ telesurgery system: overview and application , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[28]  Alin Albu-Schäffer,et al.  On a new generation of torque controlled light-weight robots , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[29]  Tobias Ortmaier,et al.  Reconstruction of Image Structure in Presence of Specular Reflections , 2001, DAGM-Symposium.

[30]  Tobias Ortmaier,et al.  Teleoperation Concepts in Minimal Invasive Surgery , 2001 .

[31]  Tobias Ortmaier,et al.  The DLR Minimally Invasive Robotics Surgery Scenario , 2001 .

[32]  Yoshihiko Nakamura,et al.  Heartbeat synchronization for robotic cardiac surgery , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[33]  Tobias Ortmaier,et al.  Tracking local motion on the beating heart , 2002, SPIE Medical Imaging.

[34]  Martin Gröger,et al.  Robust Motion Estimation in Robotic Surgery on the Beating Heart , 2002 .

[35]  Volkmar Falk,et al.  Limitations for manual and telemanipulator-assisted motion tracking--implications for endoscopic beating-heart surgery. , 2003, The Annals of thoracic surgery.

[36]  Klaus Arbter,et al.  Ein Entwurfswerkzeug für Farbklassifikatoren in Echtzeitanwendungen , 2004, Workshop Farbbildverarbeitung.

[37]  Florent Nageotte,et al.  Detection of grey regions in color images : application to the segmentation of a surgical instrument in robotized laparoscopy , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[38]  Luc Soler,et al.  Beating heart tracking in robotic surgery using 500 Hz visual servoing, model predictive control and an adaptive observer , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[39]  Gerd Hirzinger,et al.  Structure driven substitution of specular reflections for realtime heart surface tracking , 2005, IEEE International Conference on Image Processing 2005.

[40]  Tobias Ortmaier,et al.  Motion estimation in beating heart surgery , 2005, IEEE Transactions on Biomedical Engineering.

[41]  Adriano Cavalcanti,et al.  Robots in Surgery , 2005 .

[42]  G Hirzinger,et al.  Development of actuated and sensor integrated forceps for minimally invasive robotic surger , 2005, The international journal of medical robotics + computer assisted surgery : MRCAS.

[43]  Gerd Hirzinger,et al.  OPTICAL FLOW TO ANALYSE STABILISED IMAGES OF THE BEATING HEART , 2006 .

[44]  Bernhard Kübler,et al.  Development of actuated and sensor integrated forceps for minimally invasive robotic surgery , 2006 .

[45]  Florent Nageotte,et al.  Segmentation and Guidance of Multiple Rigid Objects for Intra-operative Endoscopic Vision , 2006, WDV.

[46]  Philippe Cinquin,et al.  Automatic Detection of Instruments in Laparoscopic Images: A First Step Towards High-level Command of Robotic Endoscopic Holders , 2007, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[47]  Jean-Alexandre Long,et al.  Automatic detection of instruments in laparoscopic images: a first step towards high level command of robotized endoscopic holders , 2006 .

[48]  Gerd Hirzinger,et al.  Image stabilisation of the beating heart by local linear interpolation , 2006, SPIE Medical Imaging.

[49]  Tobias Ortmaier,et al.  A hands-on-robot for accurate placement of pedicle screws , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[50]  Paolo Dario,et al.  Tracking endoscopic instruments without a localizer: A shape-analysis-based approach , 2007, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.