Active Inference or Control as Inference? A Unifying View

Active inference (AI) is a persuasive theoretical framework from computational neuroscience that seeks to describe action and perception as inference-based computation. However, this framework has yet to provide practical sensorimotor control algorithms that are competitive with alternative approaches. In this work, we frame active inference through the lens of control as inference (CaI), a body of work that presents trajectory optimization as inference. From the wider view of `probabilistic numerics', CaI offers principled, numerically robust optimal control solvers that provide uncertainty quantification, and can scale to nonlinear problems with approximate inference. We show that AI may be framed as partially-observed CaI when the cost function is defined specifically in the observation states.

[1]  Anil K. Seth,et al.  Reinforcement Learning through Active Inference , 2020, ArXiv.

[2]  Marc Toussaint,et al.  Hierarchical POMDP Controller Optimization by Likelihood Maximization , 2008, UAI.

[3]  Tim Verbelen,et al.  Bayesian policy selection using active inference , 2019, ICLR 2019.

[4]  Sergey Levine,et al.  Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review , 2018, ArXiv.

[5]  Kai Ueltzhöffer,et al.  Deep active inference , 2017, Biological Cybernetics.

[6]  Hagai Attias,et al.  Planning by Probabilistic Inference , 2003, AISTATS.

[7]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[8]  Henk Wymeersch,et al.  Application of the Free Energy Principle to Estimation and Control , 2019, IEEE Transactions on Signal Processing.

[9]  Karl J. Friston,et al.  A Factor Graph Description of Deep Temporal Active Inference , 2017, Front. Comput. Neurosci..

[10]  Beren Millidge,et al.  Deep Active Inference as Variational Policy Gradients , 2019, Journal of Mathematical Psychology.

[11]  Karl J. Friston,et al.  Active Inference: A Process Theory , 2017, Neural Computation.

[12]  H. Kappen.  Path integrals and symmetry breaking for optimal control theory , 2005, physics/0505066.

[13]  Emanuel Todorov,et al.  Linearly-solvable Markov decision problems , 2006, NIPS.

[14]  Miroslav Kárný,et al.  Towards fully probabilistic control design , 1996, Autom..

[15]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[16]  Christoph Salge,et al.  Expanding the Active Inference Landscape: More Intrinsic Motivations in the Perception-Action Loop , 2018, Front. Neurorobot..

[17]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[18]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[19]  Karl J. Friston,et al.  Reinforcement Learning or Active Inference? , 2009, PloS one.

[20]  Samuel J. Gershman,et al.  Representation learning with reward prediction errors , 2019, Neurons, Behavior, Data analysis, and Theory.

[21]  Li Ping,et al.  The Factor Graph Approach to Model-Based Signal Processing , 2007, Proceedings of the IEEE.

[22]  Gordon Cheng,et al.  Active inference body perception and action for humanoid robots , 2019, ArXiv.

[23]  Jan Peters,et al.  Stochastic Optimal Control as Approximate Input Inference , 2019, CoRL.

[24]  Yuval Tassa,et al.  Synthesis and stabilization of complex behaviors through online trajectory optimization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Christian Hoffmann,et al.  Linear Optimal Control on Factor Graphs — A Message Passing Perspective — , 2017 .

[26]  Marc Toussaint,et al.  Robot trajectory optimization using approximate inference , 2009, ICML '09.

[27]  P. Verschure,et al.  About the goal of a goals’ goal theory , 2015, Cognitive neuroscience.