Memory and mental time travel in humans and social robots

From neuroscience, brain imaging and the psychology of memory, we are beginning to assemble an integrated theory of the brain subsystems and pathways that allow the compression, storage and reconstruction of memories for past events and their use in contextualizing the present and reasoning about the future—mental time travel (MTT). Using computational models, embedded in humanoid robots, we are seeking to test the sufficiency of this theoretical account and to evaluate the usefulness of brain-inspired memory systems for social robots. In this contribution, we describe the use of machine learning techniques—Gaussian process latent variable models—to build a multimodal memory system for the iCub humanoid robot and summarize results of the deployment of this system for human–robot interaction. We also outline the further steps required to create a more complete robotic implementation of human-like autobiographical memory and MTT. We propose that generative memory models, such as those that form the core of our robot memory system, can provide a solution to the symbol grounding problem in embodied artificial intelligence. This article is part of the theme issue ‘From social brains to social robots: applying neurocognitive insights to human–robot interaction’.

[1]  Uriel Martinez-Hernandez,et al.  Expressive touch: Control of robot emotional expression by touch , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[2]  Neil D. Lawrence,et al.  A Top-Down Approach for a Synthetic Autobiographical Memory System , 2015, Living Machines.

[3]  Astrid Paeschke,et al.  A database of German emotional speech , 2005, INTERSPEECH.

[4]  Peter Ford Dominey,et al.  The Role of Autobiographical Memory in the Development of a Robot Self , 2017, Front. Neurorobot..

[5]  D. Berntsen Voluntary and involuntary access to autobiographical memory. , 1998, Memory.

[6]  M. Donald Understanding Autobiographical Memory: Evolutionary origins of autobiographical memory: a retrieval hypothesis , 2012 .

[7]  Kerstin Dautenhahn,et al.  Socially intelligent robots: dimensions of human–robot interaction , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[8]  Tony J. Prescott,et al.  Neuromorphic and Brain-Based Robots: Biomimetic robots as scientific models: a view from the whisker tip , 2011 .

[9]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[10]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2014, The Journal of Neuroscience.

[11]  E. Marg A VISION OF THE BRAIN , 1994 .

[12]  E. Balint Memory and consciousness. , 1987, The International journal of psycho-analysis.

[13]  Edmund T Rolls,et al.  Convergence of sensory systems in the orbitofrontal cortex in primates and brain design for emotion. , 2004, The anatomical record. Part A, Discoveries in molecular, cellular, and evolutionary biology.

[14]  Neil D. Lawrence,et al.  iCub Visual Memory Inspector: Visualising the iCub's Thoughts , 2016, Living Machines.

[15]  I. Fried,et al.  Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall , 2008, Science.

[16]  Neil D. Lawrence,et al.  Manifold Relevance Determination , 2012, ICML.

[17]  Neil D. Lawrence,et al.  Hierarchical Gaussian process latent variable models , 2007, ICML '07.

[18]  Praminda Caleb-Solly,et al.  Robotics in Social Care: A Connected Care EcoSystem for Independent Living , 2017 .

[19]  David C. Rubin,et al.  Belief and recollection of autobiographical memories , 2003, Memory & cognition.

[20]  Zheng Fang,et al.  Comparison of different implementations of MFCC , 2001 .

[21]  张国亮,et al.  Comparison of Different Implementations of MFCC , 2001 .

[22]  David C. Rubin,et al.  The Basic-Systems Model of Episodic Memory , 2006, Perspectives on psychological science : a journal of the Association for Psychological Science.

[23]  Paul F. M. J. Verschure,et al.  Distributed Adaptive Control: A theory of the Mind, Brain, Body Nexus , 2012, BICA 2012.

[24]  Peter Ford Dominey,et al.  DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[25]  Tony J. Prescott,et al.  The Synthetic Psychology of the Self , 2018, Intelligent Systems, Control and Automation: Science and Engineering.

[26]  D. Rubin,et al.  The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving. , 2008, Cerebral cortex.

[27]  Brian Scassellati,et al.  Socially assistive robotics [Grand Challenges of Robotics] , 2007, IEEE Robotics & Automation Magazine.

[28]  Robyn Fivush,et al.  Autobiographical Memory and the Construction of A Narrative Self : Developmental and Cultural Perspectives , 2003 .

[29]  J. Tenenbaum,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Theory-based Bayesian models of inductive learning and reasoning , 2022 .

[30]  Christopher K. I. Williams,et al.  From Neuron to Cognition via Computational > Neuroscience , 2016 .

[31]  Bruce A. MacDonald,et al.  The Role of Healthcare Robots for Older People at Home: A Review , 2014, Int. J. Soc. Robotics.

[32]  Cynthia Breazeal,et al.  An Embodied Cognition Approach to Mindreading Skills for Socially Intelligent Robots , 2009, Int. J. Robotics Res..

[33]  Tony J. Prescott,et al.  Action recognition with unsynchronised multi-sensory data , 2017, 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[34]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[35]  Philip A. Kragel,et al.  Dynamic neural networks supporting memory retrieval , 2011, NeuroImage.

[36]  H. Eichenbaum,et al.  Medial Entorhinal Cortex Selectively Supports Temporal Coding by Hippocampal Neurons , 2017, Neuron.

[37]  M. Hasselmo How We Remember: Brain Mechanisms of Episodic Memory , 2011 .

[38]  Tobias Sommer,et al.  The Role of the Human Entorhinal Cortex in a Representational Account of Memory , 2015, Front. Hum. Neurosci..

[39]  Tony J. Prescott,et al.  Embodied Models and Neurorobotics , 2016 .

[40]  Christoph Kayser,et al.  The Multisensory Nature of Unisensory Cortices: A Puzzle Continued , 2010, Neuron.

[41]  Neil D. Lawrence,et al.  An integrated probabilistic framework for robot perception, learning and memory , 2016, 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[42]  Jessica B. Hamrick,et al.  Simulation as an engine of physical scene understanding , 2013, Proceedings of the National Academy of Sciences.

[43]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[44]  Dharshan Kumaran,et al.  Strong Evidence for Pattern Separation in Human Dentate Gyrus , 2016, The Journal of Neuroscience.

[45]  T. Bonhoeffer,et al.  Grid cells and cortical representation , 2014, Nature Reviews Neuroscience.

[46]  D. Rubin,et al.  Experimental manipulations of the phenomenology of memory , 2003, Memory & cognition.

[47]  D. Dennett Producing future by telling stories , 1996 .

[48]  Abigail Sellen,et al.  Do life-logging technologies support memory for the past?: an experimental study using sensecam , 2007, CHI.

[49]  Tony Belpaeme,et al.  A review of long-term memory in natural and synthetic systems , 2012, Adapt. Behav..

[50]  Giorgio Metta,et al.  iCub-HRI: A Software Framework for Complex Human–Robot Interaction Scenarios on the iCub Humanoid Robot , 2018, Front. Robot. AI.

[51]  David C. Rubin,et al.  The Role of Narrative in Recollection: A View from Cognitive Psychology and Neuropsychology , 2012 .

[52]  Neil D. Lawrence,et al.  Variational Inference for Latent Variables and Uncertain Inputs in Gaussian Processes , 2016, J. Mach. Learn. Res..

[53]  Tony J. Prescott,et al.  A Living Machines approach to the sciences of mind and brain , 2018, Oxford Scholarship Online.

[54]  Yoko Yamaguchi,et al.  A theory of hippocampal memory based on theta phase precession , 2003, Biological Cybernetics.

[55]  Zachariah M. Reagh,et al.  Object and spatial mnemonic interference differentially engage lateral and medial entorhinal cortex in humans , 2014, Proceedings of the National Academy of Sciences.

[56]  Denis Perrin,et al.  Memory as mental time travel , 2017 .

[57]  Asohan Amarasingham,et al.  Internally Generated Cell Assembly Sequences in the Rat Hippocampus , 2008, Science.

[58]  D. Rubin,et al.  Visual memory loss and autobiographical amnesia: a case study , 2005, Neuropsychologia.

[59]  Brian Scassellati,et al.  The Grand Challenges in Socially Assistive Robotics , 2007 .

[60]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[61]  Neil D. Lawrence,et al.  Bayesian Gaussian Process Latent Variable Model , 2010, AISTATS.

[62]  Alessandro Roncone,et al.  Peripersonal Space and Margin of Safety around the Body: Learning Visuo-Tactile Associations in a Humanoid Robot with Artificial Skin , 2016, PloS one.

[63]  Giorgio Metta,et al.  Robotic Homunculus: Learning of Artificial Skin Representation in a Humanoid Robot Motivated by Primary Somatosensory Cortex , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[64]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[65]  H. Eichenbaum Time cells in the hippocampus: a new dimension for mapping memories , 2014, Nature Reviews Neuroscience.

[66]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[67]  David C. Rubin,et al.  The role of narrative in recollection: A view from cognitive and neuropsychology. , 2003 .

[68]  R. N. Spreng,et al.  The Future of Memory: Remembering, Imagining, and the Brain , 2012, Neuron.

[69]  Silvio Savarese,et al.  Watch-n-Patch: Unsupervised Learning of Actions and Relations , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  David C. Rubin,et al.  The basic systems model of autobiographical memory , 2012 .

[71]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[72]  Tony J. Prescott,et al.  Hippocampus as unitary coherent particle filter , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[73]  Adrian Rubio Solis,et al.  Bayesian perception of touch for control of robot emotion , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[74]  Paul F. M. J. Verschure,et al.  The why, what, where, when and how of goal-directed choice: neuronal and computational principles , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.