Inference and Learning for Active Sensing, Experimental Design and Control

In this paper we argue that maximum expected utility is a suitable framework for modeling a broad range of decision problems arising in pattern recognition and related fields. Examples include, among others, gaze planning and other active vision problems, active learning, sensor and actuator placement and coordination, intelligent human-computer interfaces, and optimal control. Following this remark, we present a common inference and learning framework for attacking these problems. We demonstrate this approach on three examples: (i) active sensing with nonlinear, non-Gaussian, continuous models, (ii) optimal experimental design to discriminate among competing scientific models, and (iii) nonlinear optimal control.

[1]  Jay I. Myung,et al.  Optimal experimental design for model discrimination. , 2009, Psychological review.

[2]  J. Bernardo.  Expected Information as Expected Utility , 1979 .

[3]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[4]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[5]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[6]  D. Rubin,et al.  The Precise Time Course of Retention , 1999 .

[7]  Arnaud Doucet,et al.  SMC Samplers for Bayesian Optimal Nonlinear Design , 2006, 2006 IEEE Nonlinear Statistical Signal Processing Workshop.

[8]  P. Müller,et al.  Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation , 2004 .

[9]  D. Rubin,et al.  One Hundred Years of Forgetting : A Quantitative Description of Retention , 1996 .

[10]  Yoav Shoham,et al.  Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations , 2009 .

[11]  T. Loredo.  Bayesian Adaptive Exploration , 2004, astro-ph/0409386.

[12]  Nando de Freitas,et al.  Bayesian Policy Learning with Trans-Dimensional MCMC , 2007, NIPS.

[13]  J. Neumann,et al.  The Theory of Games and Economic Behaviour , 1944 .

[14]  Geoffrey E. Hinton,et al.  Using EM for Reinforcement Learning , 2000 .

[15]  Geoffrey E. Hinton,et al.  Using Expectation-Maximization for Reinforcement Learning , 1997, Neural Computation.

[16]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[17]  Matt Hoffman,et al.  On Solving General State-Space Sequential Decision Problems using Inference Algorithms , 2007 .

[18]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[19]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .