Policy Generation for Continuous-time Stochastic Domains with Concurrency

We adopt the framework of Younes, Musliner, & Simmons for planning with concurrency in continuous-time stochastic domains. Our contribution is a set of concrete techniques for policy generation, failure analysis, and repair. These techniques have been implemented in TEMPASTIC, a novel temporal probabilistic planner, and we demonstrate the performance of the planner on two variations of a transportation domain with concurrent actions and exogenous events. TEMPASTIC makes use of a deterministic temporal planner to generate initial policies. Policies are represented using decision trees, and we use incremental decision tree induction to efficiently incorporate changes suggested by the failure analysis.

[1]  Carlos Guestrin,et al.  Multiagent Planning with Factored MDPs , 2001, NIPS.

[2]  Håkan L. S. Younes,et al.  On the Role of Ground Actions in Refinement Planning , 2002, AIPS.

[3]  David J. Musliner,et al.  Toward Decision-Theoretic CIRCA with Application to Real-Time Computer Security Control , 2002 .

[4]  Håkan L. S. Younes.  Extending PDDL to Model Stochastic Decision Processes , 2003 .

[5]  Reid G. Simmons,et al.  A Theory of Debugging Plans and Interpretations , 1988, AAAI.

[6]  David E. Smith,et al.  Planning Under Continuous Time and Resource Uncertainty: A Challenge for AI , 2002, AIPS Workshop on Planning for Temporal Domains.

[7]  John L. Bresina,et al.  Anytime Synthetic Projection: Maximizing the Probability of Goal Satisfaction , 1990, AAAI.

[8]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[9]  Michael L. Littman,et al.  Exact Solutions to Time-Dependent MDPs , 2000, NIPS.

[10]  Blai Bonet,et al.  A Robust and Fast Action Selection Mechanism for Planning , 1997, AAAI/IAAI.

[11]  P. Glynn.  A GSMP formalism for discrete event systems , 1989, Proc. IEEE.

[12]  Jim Blythe,et al.  Planning with External Events , 1994, UAI.

[13]  Daniel S. Weld.  An Introduction to Least Commitment Planning , 1994, AI Mag..

[14]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..

[15]  Håkan L. S. Younes,et al.  VHPOP: Versatile Heuristic Partial Order Planner , 2003, J. Artif. Intell. Res..

[16]  Craig Boutilier,et al.  Exploiting Structure in Policy Construction , 1995, IJCAI.

[17]  Håkan L. S. Younes,et al.  Numerical vs. Statistical Probabilistic Model Checking: An Empirical Study , 2004, TACAS.

[18]  Paul E. Utgoff,et al.  Decision Tree Induction Based on Efficient Tree Restructuring , 1997, Machine Learning.

[19]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[20]  Håkan L. S. Younes,et al.  Probabilistic Verification of Discrete Event Systems Using Acceptance Sampling , 2002, CAV.

[21]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[22]  Robert K. Brayton,et al.  Model-checking continuous-time Markov chains , 2000, TOCL.

[23]  David J. Musliner,et al.  AF ramework for Planning in Continuous-time Stochastic Domains H˚ akan L. S. Younes , 2003 .

[24]  Rina Dechter,et al.  Temporal Constraint Networks , 1989, Artif. Intell..

[25]  W. Nelson.  Weibull Analysis of Reliability Data with Few or No Failures , 1985 .