Solving POMDPs Using Selected Past Events

We present new algorithms for solving Partially Observed Markov Decision Processes. These algorithms are build on theoretical results showing that if one can find an observable with required properties, it is possible to build an extension of the state space using past events which defines a Markov Decision Process equivalent to the original problem. Thus, solving POMDPs, which is a very hard task, is seen as solving a MPD, where numerous existing algorithms can be successfully used. Our first algorithm uses reinforcement learning to solve POMDPs when the evolution model is unknown, whereas our second algorithm, more efficient, can only be applied when the model is known.