Hierarchical Approaches to Concurrency, Multiagency, and Partial Observability

In this chapter the authors summarize their research in hierarchical probabilistic models for decision making involving concurrent action, multiagent coordination, and hidden state estimation in stochastic environments. A hierarchical model for learning concurrent plans is first described for observable single agent domains, which combines compact state representations with temporal process abstractions to determine how to parallelize multiple threads of activity. A hierarchical model for multiagent coordination is then presented, where primitive joint actions and joint states are hidden. Here, high-level coordination is learned by exploiting overall task structure, which greatly speeds up convergence by abstracting from low-level steps that do not need to be synchronized. Finally, a hierarchical frame-work for hidden state estimation and action is presented, based on multi-resolution statistical modeling of the past history of observations and actions.