Modeling dynamical systems, both for control purposes and to make predictions about their behavior, is ubiquitous in science and engineering. Predictive state representations (PSRs) are a recently introduced class of models for discrete-time dynamical systems. The key idea behind PSRs and the closely related OOMs (Jaeger's observable operator models) is to represent the state of the system as a set of predictions of observable outcomes of experiments one can do in the system. This makes PSRs rather different from history-based models such as nth-order Markov models and hidden-state-based models such as HMMs and POMDPs. We introduce an interesting construct, the system-dynamics matrix, and show how PSRs can be derived simply from it. We also use this construct to show formally that PSRs are more general than both nth-order Markov models and HMMs/POMDPs. Finally, we discuss the main difference between PSRs and OOMs and conclude with directions for future work.
[1]
Shun-ichi Amari,et al.
Identifiability of hidden Markov information sources and their minimum degrees of freedom
,
1992,
IEEE Trans. Inf. Theory.
[2]
Leslie Pack Kaelbling,et al.
Learning Topological Maps with Weak Local Odometric Information
,
1997,
IJCAI.
[3]
Observable Operator Processes and Conditioned Continuation Representations 1
,
1997
.
[4]
Richard S. Sutton,et al.
Predictive Representations of State
,
2001,
NIPS.
[5]
H. Jaeger.
Discrete-time, discrete-valued observable operator models: a tutorial
,
2003
.
[6]
Satinder P. Singh,et al.
A Nonlinear Predictive State Representation
,
2003,
NIPS.
[7]
Peter Stone,et al.
Learning Predictive State Representations
,
2003,
ICML.
[8]
Michael R. James,et al.
Learning and discovery of predictive state representations in dynamical systems with reset
,
2004,
ICML.