Why internal feedback is necessary in the perception-action loop

Animals can move reliably in unpredictable environments. The study of sensorimotor control has assumed that sensory information from the environment leads to actions, which then act back on the environment, creating a single, unidirectional perception-action loop. This loop contains internal delays in sensory and motor pathways, which can lead to unstable control. We show here that these delays can be compensated by internal feedback signals that flow backwards, which are ubiquitous in neural sensorimotor systems. Recent ad-vances in control theory show that internal feedback plays a key role in compensating internal delays. Based on these theoretical ad-vances, we use a basic, mathematically tractable control model to show that internal feedback has an indispensable role in state estimation, can explain localization of function – why different parts of cortex control different parts of the body – and how attention can improve motor performance, all of which are crucial for effective sensorimotor control. Control theory can explain anatomical, physiological and behavioral observations, including motor signals in visual cortex, heterogeneous kinetics of sensory receptors and the presence of giant Betz cells in motor cortex.

[1]  John C. Doyle,et al.  Internal Feedback in Biological Control: Architectures and Examples , 2021, 2022 American Control Conference (ACC).

[2]  Anish A. Sarma,et al.  Internal Feedback in Biological Control: Diversity, Delays, and Standard Theory , 2021, 2022 American Control Conference (ACC).

[3]  Jing Shuang Lisa Li Internal Feedback in Biological Control: Locality and System Level Synthesis , 2021, ACC.

[4]  Krishna V. Shenoy,et al.  Computation Through Neural Population Dynamics. , 2020, Annual review of neuroscience.

[5]  Francis R. Willett,et al.  Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way , 2020, Cell.

[6]  Mark M Churchland,et al.  Motor cortex signals for each arm are mixed across hemispheres and neurons yet partitioned within the population response , 2019, eLife.

[7]  T. Sejnowski,et al.  Diversity-enabled sweet spots in layered architectures and speed–accuracy trade-offs in sensorimotor control , 2019, Proceedings of the National Academy of Sciences.

[8]  Matthew T. Kaufman,et al.  Single-trial neural dynamics are dominated by richly varied movements , 2019, Nature Neuroscience.

[9]  Francis R. Willett,et al.  Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis , 2018, bioRxiv.

[10]  James J. DiCarlo,et al.  Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior , 2018, Nature Neuroscience.

[11]  Nicholas A. Steinmetz,et al.  Spontaneous behaviors drive multidimensional, brainwide activity , 2019, Science.

[12]  Georg B. Keller,et al.  A Sensorimotor Circuit in Mouse Cortex for Visual Flow Predictions , 2017, Neuron.

[13]  Nikolai Matni,et al.  Hard limits on robust control over delayed and quantized communication channels with applications to sensorimotor control , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[14]  J Anthony Movshon,et al.  Visual Response Properties of V1 Neurons Projecting to V2 in Macaque , 2013, The Journal of Neuroscience.

[15]  Lars Muckli,et al.  Network interactions: non-geniculate input to V1 , 2013, Current Opinion in Neurobiology.

[16]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[17]  Matthew W Self,et al.  Different glutamate receptors convey feedforward and recurrent processing in macaque V1 , 2012, Proceedings of the National Academy of Sciences.

[18]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[19]  David W. Franklin,et al.  Computational Mechanisms of Sensorimotor Control , 2011, Neuron.

[20]  Tim Gollisch,et al.  Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina , 2010, Neuron.

[21]  P. Swain,et al.  Noisy information processing through transcriptional regulation , 2007, Proceedings of the National Academy of Sciences.

[22]  D. Attwell,et al.  Neuroenergetics and the kinetic design of excitatory synapses , 2005, Nature Reviews Neuroscience.

[23]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[24]  Edward M. Callaway,et al.  Feedforward, feedback and inhibitory connections in primate visual cortex , 2004, Neural Networks.

[25]  J. Bullier,et al.  Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons? , 2003, Journal of Physiology-Paris.

[26]  M. Livingstone,et al.  Mechanisms of Direction Selectivity in Macaque V1 , 1998, Neuron.

[27]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[28]  S. Palay,et al.  Meynert cells in the primate visual cortex , 1974, Journal of neurocytology.

[29]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[30]  Tamal Mukherjee,et al.  System-Level Synthesis , 2003 .

[31]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[32]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.