Assisting Movement Training and Execution With Visual and Haptic Feedback

In the practice of motor skills in general, errors in the execution of movements may go unnoticed when a human instructor is not available. In this case, a computer system or robotic device able to detect movement errors and propose corrections would be of great help. This paper addresses the problem of how to detect such execution errors and how to provide feedback to the human to correct his/her motor skill using a general, principled methodology based on imitation learning. The core idea is to compare the observed skill with a probabilistic model learned from expert demonstrations. The intensity of the feedback is regulated by the likelihood of the model given the observed skill. Based on demonstrations, our system can, for example, detect errors in the writing of characters with multiple strokes. Moreover, by using a haptic device, the Haption Virtuose 6D, we demonstrate a method to generate haptic feedback based on a distribution over trajectories, which could be used as an auxiliary means of communication between an instructor and an apprentice. Additionally, given a performance measurement, the haptic device can help the human discover and perform better movements to solve a given task. In this case, the human first tries a few times to solve the task without assistance. Our framework, in turn, uses a reinforcement learning algorithm to compute haptic feedback, which guides the human toward better solutions.

[1]  Andreas Kramer,et al.  Robotic guidance induces long-lasting changes in the movement pattern of a novel sport-specific motor task. , 2014, Human movement science.

[2]  Pieter Abbeel,et al.  Learning for control from multiple demonstrations , 2008, ICML '08.

[3]  C. Goodall.  Procrustes methods in the statistical analysis of shape , 1991 .

[4]  Freek Stulp,et al.  Co-manipulation with multiple probabilistic virtual guides , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Massimo Bergamasco,et al.  Teaching to write Japanese characters using a haptic interface , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[6]  Stefan Schaal,et al.  Reinforcement learning by reward-weighted regression for operational space control , 2007, ICML '07.

[7]  Oliver Kroemer,et al.  Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks , 2017, Auton. Robots.

[8]  Siddhartha S. Srinivasa,et al.  Object recognition and full pose registration from a single image for robotic manipulation , 2009, 2009 IEEE International Conference on Robotics and Automation.

[9]  Masud Moshtaghi,et al.  Detecting unseen anomalies in weight training exercises , 2016, OZCHI.

[10]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[11]  P. Sanguansat.  Multiple Multidimensional Sequence Alignment Using Generalized Dynamic Time Warping , 2012 .

[12]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2013, Psychonomic bulletin & review.

[13]  A. Timmermans,et al.  Technology-assisted training of arm-hand skills in stroke: concepts on reacquisition of motor control and therapist guidelines for rehabilitation technology design , 2009, Journal of NeuroEngineering and Rehabilitation.

[14]  Louis B. Rosenberg,et al.  The Use of Virtual Fixtures as Perceptual Overlays to Enhance Operator Performance in Remote Environments. , 1992 .

[15]  Yiannis Demiris,et al.  Learning assistance by demonstration , 2015, HRI 2015.

[16]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[17]  Jan Peters,et al.  Movement primitives with multiple phase parameters , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.

[19]  Benjamin Rosman,et al.  Knowledge transfer for learning robot models via Local Procrustes Analysis , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[20]  Toru Tsumugiwa,et al.  Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[21]  Jan Peters,et al.  Alignment-based transfer learning for robot models , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[22]  Radford M. Neal,et al.  Multiple Alignment of Continuous Time Series , 2004, NIPS.

[23]  Siddhartha S. Srinivasa,et al.  Distance metrics and algorithms for task space path optimization , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Stefan Wermter,et al.  Human motion assessment in real time using recurrent self-organization , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[25]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .