Communicative Capital for Prosthetic Agents

This work presents an overarching perspective on the role that machine intelligence can play in enhancing human abilities, especially those that have been diminished due to injury or illness. As a primary contribution, we develop the hypothesis that assistive devices, and specifically artificial arms and hands, can and should be viewed as agents in order for us to most effectively improve their collaboration with their human users. We believe that increased agency will enable more powerful interactions between human users and next generation prosthetic devices, especially when the sensorimotor space of the prosthetic technology greatly exceeds the conventional control and communication channels available to a prosthetic user. To more concretely examine an agency-based view on prosthetic devices, we propose a new schema for interpreting the capacity of a human-machine collaboration as a function of both the human's and machine's degrees of agency. We then introduce the idea of communicative capital as a way of thinking about the communication resources developed by a human and a machine during their ongoing interaction. Using this schema of agency and capacity, we examine the benefits and disadvantages of increasing the agency of a prosthetic limb. To do so, we present an analysis of examples from the literature where building communicative capital has enabled a progression of fruitful, task-directed interactions between prostheses and their human users. We then describe further work that is needed to concretely evaluate the hypothesis that prostheses are best thought of as agents. The agent-based viewpoint developed in this article significantly extends current thinking on how best to support the natural, functional use of increasingly complex prosthetic enhancements, and opens the door for more powerful interactions between humans and their assistive technologies.

[1]  Harry E. Stephanou,et al.  A Hierarchical Approach to the Control of a Prosthetic Arm , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  V. Mathiowetz,et al.  Adult norms for the Box and Block Test of manual dexterity. , 1985, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[3]  Long Ji Lin,et al.  Programming Robots Using Reinforcement Learning and Teaching , 1991, AAAI.

[4]  L.-J. Lin,et al.  Hierarchical learning of robot skills by reinforcement , 1993, IEEE International Conference on Neural Networks.

[5]  Pat Langley,et al.  Machine Learning for Adaptive User Interfaces , 1997, KI.

[6]  C. Dewdney Last Flesh: Life in the Transhuman Era , 1998 .

[7]  M. Gazzaniga Cerebral specialization and interhemispheric communication: does the corpus callosum enable the human condition? , 2000, Brain : a journal of neurology.

[8]  Rodney A. Brooks,et al.  Flesh and Machines: How Robots Will Change Us , 2002 .

[9]  C. Light,et al.  Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: normative data, reliability, and validity. , 2002, Archives of physical medicine and rehabilitation.

[10]  J. Geary Body Electric: An Anatomy of the New Bionic Senses , 2002 .

[11]  Pierre-Yves Oudeyer,et al.  Robotic clicker training , 2002, Robotics Auton. Syst..

[12]  K. Doya,et al.  A unifying computational framework for motor control and social interaction. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[13]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[14]  Gul A. Agha,et al.  Towards a hierarchical taxonomy of autonomous agents , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[15]  Longxin Lin Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching , 2004, Machine Learning.

[16]  D. Feil-Seifer,et al.  Defining socially assistive robotics , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  B Hudgins,et al.  Myoelectric signal processing for control of powered limb prostheses. , 2006, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[19]  H. Bekkering,et al.  Joint action: bodies and minds moving together , 2006, Trends in Cognitive Sciences.

[20]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[21]  J. Pincus The Brain That Changes Itself: Stories of Personal Triumph From the Frontiers of Brain Science , 2008 .

[22]  Kathryn Ziegler-Graham,et al.  Estimating the prevalence of limb loss in the United States: 2005 to 2050. , 2008, Archives of physical medicine and rehabilitation.

[23]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[24]  Peter Stone,et al.  Interactively shaping agents via human reinforcement: the TAMER framework , 2009, K-CAP '09.

[25]  Jacqueline S. Hebert,et al.  Outcome measures in amputation rehabilitation: ICF body functions , 2009, Disability and rehabilitation.

[26]  Natalie Sebanz,et al.  Prediction in Joint Action: What, When, and Where , 2009, Top. Cogn. Sci..

[27]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[28]  S Micera,et al.  Control of Hand Prostheses Using Peripheral Information , 2010, IEEE Reviews in Biomedical Engineering.

[29]  Stefanie Tellex,et al.  Toward understanding natural language directions , 2010, HRI 2010.

[30]  Farbod Fahimi,et al.  Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[31]  Linda Resnik,et al.  Development and testing of new upper-limb prosthetic devices: research designs for usability testing. , 2011, Journal of rehabilitation research and development.

[32]  Patrick M. Pilarski,et al.  Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction , 2011, AAMAS.

[33]  M. Lundberg,et al.  My prosthesis as a part of me: a qualitative analysis of living with an osseointegrated prosthetic limb , 2011, Prosthetics and orthotics international.

[34]  T Walley Williams,et al.  Progress on stabilizing and controlling powered upper-limb prostheses. , 2011, Journal of rehabilitation research and development.

[35]  Giovanni Pezzulo,et al.  What should I do next? Using shared representations to solve interaction problems , 2011, Experimental Brain Research.

[36]  Frank Moss The Sorcerers and Their Apprentices: How the Digital Magicians of the MIT Media Lab Are Creating the Innovative Technologies That Will Transform Our Lives , 2011 .

[37]  Stefano Stramigioli,et al.  Myoelectric forearm prostheses: state of the art from a user-centered perspective. , 2011, Journal of rehabilitation research and development.

[38]  Estela Bicho,et al.  Neuro-cognitive mechanisms of decision making in joint action: a human-robot interaction study. , 2011, Human movement science.

[39]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[40]  Peter Stone,et al.  Reinforcement learning from simultaneous human and MDP reward , 2012, AAMAS.

[41]  Patrick M. Pilarski,et al.  Between Instruction and Reward: Human-Prompted Switching , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[42]  L. Resnik,et al.  Advanced upper limb prosthetic devices: implications for upper limb prosthetic rehabilitation. , 2012, Archives of physical medicine and rehabilitation.

[43]  J. M. Carmena Becoming Bionic , 2012, IEEE Spectrum.

[44]  TaeChoong Chung,et al.  Learning via human feedback in continuous state and action spaces , 2013, Applied Intelligence.

[45]  Cynthia Breazeal,et al.  Training a Robot via Human Feedback: A Case Study , 2013, ICSR.

[46]  Patrick M. Pilarski,et al.  Adaptive artificial limbs: a real-time approach to prediction and anticipation , 2013, IEEE Robotics & Automation Magazine.

[47]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[48]  G. Pezzulo,et al.  Human Sensorimotor Communication: A Theory of Signaling in Online Social Interactions , 2013, PloS one.

[49]  Patrick M. Pilarski,et al.  Real-time prediction learning for the simultaneous actuation of multiple prosthetic joints , 2013, 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR).

[50]  Elizabeth A. Croft,et al.  Design and impact of hesitation gestures during human-robot resource conflicts , 2013, HRI 2013.

[51]  Andrea Lockerd Thomaz,et al.  Policy Shaping: Integrating Human Feedback with Reinforcement Learning , 2013, NIPS.

[52]  Jian Huang,et al.  Study of reinforcement learning based shared control of walking-aid robot , 2013, Proceedings of the 2013 IEEE/SICE International Symposium on System Integration.

[53]  David L. Roberts,et al.  A Strategy-Aware Technique for Learning Behaviors from Discrete Human Feedback , 2014, AAAI.

[54]  Alan K. Mackworth,et al.  A Wizard-of-Oz Intelligent Wheelchair Study with Cognitively-Impaired Older Adults : Attitudes toward User Control , 2014 .

[55]  Patrick M. Pilarski,et al.  Using Learned Predictions as Feedback to Improve Control and Communication with an Artificial Limb: Preliminary Findings , 2014, ArXiv.

[56]  Panagiotis K. Artemiadis,et al.  Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography , 2014, Front. Neurorobot..

[57]  Dario Farina,et al.  Stereovision and augmented reality for closed-loop control of grasping in hand prostheses , 2014, Journal of neural engineering.

[58]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[59]  J. Olson,et al.  The evolution of functional hand replacement: From iron prostheses to hand transplantation. , 2014, Plastic surgery.

[60]  J. Hebert,et al.  Updates in Targeted Sensory Reinnervation for Upper Limb Amputation , 2014, Current Surgery Reports.

[61]  Jacqueline S. Hebert,et al.  Applications of sensory feedback in motorized upper extremity prosthesis: a review , 2014, Expert review of medical devices.

[62]  Max Ortiz-Catalan,et al.  An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs , 2014, Science Translational Medicine.

[63]  Andrea Lockerd Thomaz,et al.  Robot Learning from Human Teachers , 2014, Robot Learning from Human Teachers.

[64]  Patrick M. Pilarski,et al.  A Collaborative Approach to the Simultaneous Multi-joint Control of a Prosthetic Arm , 2015, 2015 IEEE International Conference on Rehabilitation Robotics (ICORR).

[65]  A. Clark Embodied Prediction , 2015 .

[66]  P. Pilarski Prosthetic Devices as Goal-Seeking Agents , 2015 .

[67]  John Markoff,et al.  Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots , 2015 .

[68]  G. Pezzulo,et al.  Interactional leader–follower sensorimotor communication strategies during repetitive joint actions , 2015, Journal of The Royal Society Interface.

[69]  Peter Stone,et al.  Framing reinforcement learning from human reward: Reward positivity, temporal discounting, episodicity, and performance , 2015, Artif. Intell..

[70]  Patrick M. Pilarski,et al.  Machine learning and unlearning to autonomously switch between the functions of a myoelectric arm , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[71]  Patrick M. Pilarski,et al.  Face valuing: Training user interfaces with facial expressions and reinforcement learning , 2016, ArXiv.

[72]  Chang Liu,et al.  Cooperative Search Using Human-UAV Teams , 2016 .

[73]  Patrick M. Pilarski,et al.  Steps toward knowledgeable neuroprostheses , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[74]  Brian Scassellati,et al.  Socially Assistive Robotics: Methods and Implications for the Future of Work and Care , 2022, Robophilosophy.

[75]  Ann L. Edwards,et al.  Adaptive and Autonomous Switching: Shared Control of Powered Prosthetic Arms Using Reinforcement Learning , 2016 .

[76]  Patrick M. Pilarski,et al.  Introspective Agents: Confidence Measures for General Value Functions , 2016, AGI.

[77]  Patrick M. Pilarski,et al.  Simultaneous Control and Human Feedback in the Training of a Robotic Agent with Actor-Critic Reinforcement Learning , 2016, ArXiv.