A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots

We study the problem of perceiving forest or mountain trails from a single monocular image acquired from the viewpoint of a robot traveling on the trail itself. Previous literature focused on trail segmentation, and used low-level features such as image saliency or appearance contrast; we propose a different approach based on a deep neural network used as a supervised image classifier. By operating on the whole image at once, our system outputs the main direction of the trail compared to the viewing direction. Qualitative and quantitative results computed on a large real-world dataset (which we provide for download) show that our approach outperforms alternatives, and yields an accuracy comparable to the accuracy of humans that are tested on the same image classification task. Preliminary results on using this information for quadrotor control in unseen trails are reported. To the best of our knowledge, this is the first letter that describes an approach to perceive forest trials, which is demonstrated on a quadrotor micro aerial vehicle.

[1]  E. D. Dickmanns,et al.  A Curvature-based Scheme for Improving Road Vehicle Guidance by Computer Vision , 1987, Other Conferences.

[2]  Jürgen Schmidhuber,et al.  Learning to Generate Artificial Fovea Trajectories for Target Detection , 1991, Int. J. Neural Syst..

[3]  Dean A. Pomerleau,et al.  Neural Network Perception for Mobile Robot Guidance , 1993 .

[4]  Reinhold Behringer,et al.  The seeing passenger car 'VaMoRs-P' , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.

[5]  Anat Levin,et al.  Learning to Combine Bottom-Up and Top-Down Segmentation , 2006, ECCV.

[6]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[7]  Ernst D. Dickmanns,et al.  Dynamic Vision for Perception and Control of Motion , 2007 .

[8]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[9]  Anat Levin,et al.  Learning to Combine Bottom-Up and Top-Down Segmentation , 2006, International Journal of Computer Vision.

[10]  Jan Peters,et al.  Learning motor primitives for robotics , 2009, 2009 IEEE International Conference on Robotics and Automation.

[11]  Yann LeCun,et al.  Learning long‐range vision for autonomous off‐road driving , 2009, J. Field Robotics.

[12]  James Sean Humbert,et al.  Implementation of wide-field integration of optic flow for autonomous quadrotor navigation , 2009, Auton. Robots.

[13]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[14]  Yan Lu,et al.  Appearance contrast for fast, robust trail-following , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Dario Floreano,et al.  Vision-based control of near-obstacle flight , 2009, Auton. Robots.

[16]  Yann LeCun,et al.  A multirange architecture for collision‐free off‐road robot navigation , 2009, J. Field Robotics.

[17]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[18]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[19]  Mandyam V Srinivasan,et al.  Visual control of navigation in insects and its relevance for robotics , 2011, Current Opinion in Neurobiology.

[20]  Luca Maria Gambardella,et al.  Flexible, High Performance Convolutional Neural Networks for Image Classification , 2011, IJCAI.

[21]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[22]  Alexander Toet,et al.  Computational versus Psychophysical Bottom-Up Image Saliency: A Comparative Evaluation Study , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Roland Siegwart,et al.  Starleth: A compliant quadrupedal robot for fast, efficient, and versatile locomotion , 2012 .

[24]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[25]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Russ Tedrake,et al.  Safety verification of reactive controllers for UAV flight in cluttered environments using barrier certificates , 2012, 2012 IEEE International Conference on Robotics and Automation.

[27]  Yan Lu,et al.  A Trail-Following Robot Which Uses Appearance and Structural Cues , 2012, FSR.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[30]  Martial Hebert,et al.  Learning monocular reactive UAV control in cluttered natural environments , 2012, 2013 IEEE International Conference on Robotics and Automation.

[31]  Nelson Alves,et al.  Tracking natural trails with swarm‐based visual saliency , 2013, J. Field Robotics.

[32]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Dario Floreano,et al.  A Collision‐resilient Flying Robot , 2014, J. Field Robotics.

[34]  Flavio Fontana,et al.  Autonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle , 2016, J. Field Robotics.