A case study on learning a steering controller from scratch with reinforcement learning

In this case study we show how reinforcement learning can be applied successfully for low level control tasks in autonomous driving like steering control as an alternative to controllers from classical control theory. We describe the learning procedure and compare the resulting control policies with a classical controller. The experiments are made both in simulation and on a real car and we discuss the case of driving forwards as well as of driving backwards.

[1]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[2]  Steffen Gutjahr,et al.  Extended Bayesian learning , 1997, ESANN.

[3]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Martin A. Riedmiller,et al.  The Neuro Slot Car Racer: Reinforcement Learning in a Real World Setting , 2009, 2009 International Conference on Machine Learning and Applications.

[6]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[7]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[8]  C.M. Clark,et al.  Reinforcement learning of adaptive longitudinal vehicle control for dynamic collaborative driving , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[11]  Se-Young Oh,et al.  A new reinforcement learning vehicle control architecture for vision-based road following , 2000, IEEE Trans. Veh. Technol..

[12]  TomMart,et al.  Navigation of Autonomous Vehicles in Unknown Environments using Reinforcement Learning , 2007 .

[13]  Martin A. Riedmiller,et al.  Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms , 1994 .

[14]  Martin A. Riedmiller,et al.  Learning to Drive a Real Car in 20 Minutes , 2007, 2007 Frontiers in the Convergence of Bioscience and Information Technologies.

[15]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[16]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[17]  N. H. C. Yung,et al.  Automated Vehicle Overtaking based on a Multiple-Goal Reinforcement Learning Framework , 2007, 2007 IEEE Intelligent Transportation Systems Conference.