FF-Replan: A Baseline for Probabilistic Planning

FF-Replan was the winner of the 2004 International Probabilistic Planning Competition (IPPC-04) (Younes & Littman 2004a) and was also the top performer on IPPC-06 domains, though it was not an official entry. This success was quite surprising, due to the simplicity of the approach. In particular, FF-Replan calls FF on a carefully constructed deterministic variant of the planning problem and selects actions according to the plan until observing an unexpected effect, upon which it replans. Despite the obvious shortcomings of the approach and its strawman nature, it is the state-of-the-art in probabilistic planning as measured on recent competition benchmarks. This paper gives the first technical description of FF-Replan and provides an analysis of its results on all of the recent IPPC-04 and IPPC-06 domains. We hope that this will inspire extensions and insight into the approach and planning domains themselves that will soon lead to the dethroning of FF-Replan.

[1]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[2]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[3]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[4]  Ronen I. Brafman,et al.  Structured Reachability Analysis for Markov Decision Processes , 1998, UAI.

[5]  John Langford,et al.  Probabilistic Planning in the Graphplan Framework , 1999, ECP.

[6]  Craig Boutilier,et al.  Stochastic dynamic programming with factored representations , 2000, Artif. Intell..

[7]  Bernhard Nebel,et al.  The FF Planning System: Fast Plan Generation Through Heuristic Search , 2011, J. Artif. Intell. Res..

[8]  Gang Wu,et al.  Burst-level congestion control using hindsight optimization , 2002, IEEE Trans. Autom. Control..

[9]  Robert Givan,et al.  Approximate Policy Iteration with a Policy Language Bias , 2003, NIPS.

[10]  David E. Smith,et al.  Optimal Limited Contingency Planning , 2002, UAI.

[11]  Håkan L. S. Younes,et al.  PPDDL 1 . 0 : An Extension to PDDL for Expressing Planning Domains with Probabilistic Effects , 2004 .

[12]  A. S. Yoon,et al.  Learning re-active policies for probabilistic planning domains , 2004 .

[13]  Scott Sanner,et al.  Approximate Linear Programming for First-order MDPs , 2005, UAI.

[14]  Eldar Karabaev,et al.  A Heuristic Search Algorithm for Solving First-Order MDPs , 2005, UAI.

[15]  Pascal Van Hentenryck,et al.  Performance Analysis of Online Anticipatory Algorithms for Large Multistage Stochastic Integer Programs , 2007, IJCAI.

[16]  Piergiorgio Bertoli,et al.  A Hybridized Planner for Stochastic Domains , 2007, IJCAI.

[17]  Olivier Buffet,et al.  The factored policy-gradient planner , 2009, Artif. Intell..

[18]  H. Ector Geener.  Classical, Probabilistic, and Contingent Planning: Three Models, One Algorithm , 2022 .