A Taxonomy for Abstract Environments.

AbstractA wide range of problems can be abstractly expressed in terms of two systems inter-acting over time with the second system providing feedback to the flrst on its successin achieving some goal. The agent{environment model in reinforment learning and thecontroller{plant model from control theory are both examples of this framework. Problemswhich can be expressed in terms of this framework include statistical sampling processes,prediction problems, classiflcation problems, decision problems, games and function opti-misation problems. While these problem classes are very common and strong relationshipsbetween them are apparent, they are not usually considered in toto . Here we formallydeflne many of these common problem classes as environments using the chronological sys-tem formalism. From this we establish the elementary relationships between the problemclasses giving a taxonomy of abstract environments. 1 Agent-Environment Model 1.1 Introduction The agent-environment model is very simple and yet surprisingly general. It consists of twoentities called the