LIMO: Lidar-Monocular Visual Odometry

Higher level functionality in autonomous driving depends strongly on a precise motion estimate of the vehicle. Powerful algorithms have been developed. However, their great majority focuses on either binocular imagery or pure LIDAR measurements. The promising combination of camera and LIDAR for visual localization has mostly been unattended. In this work we fill this gap, by proposing a depth extraction algorithm from LIDAR measurements for camera feature tracks and estimating motion by robustified keyframe based Bundle Adjustment. Semantic labeling is used for outlier rejection and weighting of vegetation landmarks. The capability of this sensor combination is demonstrated on the competitive KITTI dataset, achieving a placement among the top 15. The code is released to the community.

[1]  Ji Zhang,et al.  Real-time depth enhanced monocular odometry , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Ji Zhang,et al.  Visual-lidar odometry and mapping: low-drift, robust, and fast , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[4]  Ivan Petrovic,et al.  Stereo odometry based on careful feature selection and tracking , 2015, 2015 European Conference on Mobile Robots (ECMR).

[5]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[6]  Andreas Geiger,et al.  Automatic camera and range sensor calibration using a single shot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[7]  Volker Willert,et al.  Flow-decoupled normalized reprojection error for visual odometry , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[8]  Andrew W. Fitzgibbon,et al.  Invariant Fitting of Two View Geometry , 2003, BMVC.

[9]  Martin Lauer,et al.  Momo: Monocular motion estimation on manifolds , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[10]  Igor Cvisic.  SOFT-SLAM : Computationally Efficient Stereo Visual SLAM for Autonomous UAVs , 2017 .

[11]  Ivan Markovic,et al.  SOFT‐SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles , 2018, J. Field Robotics.

[12]  Daniel Cremers.  Direct methods for 3D reconstruction and visual SLAM , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[13]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[14]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[15]  Christoph Gustav Keller,et al.  Multi trajectory pose adjustment for life-long mapping , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[16]  Sinisa Segvic,et al.  Improving the Egomotion Estimation by Correcting the Calibration Bias , 2015, VISAPP.

[17]  Martin Lauer,et al.  Robust scale estimation for monocular visual odometry using structure from motion and vanishing points , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[18]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[19]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[20]  Sei Ikeda,et al.  Visual SLAM algorithms: a survey from 2010 to 2016 , 2017, IPSJ Transactions on Computer Vision and Applications.

[21]  Roland Siegwart,et al.  Fusion of IMU and Vision for Absolute Scale Estimation in Monocular SLAM , 2011, J. Intell. Robotic Syst..

[22]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jörg Stückler,et al.  Large-scale direct SLAM with stereo cameras , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Wolfram Burgard,et al.  Monocular camera localization in 3D LiDAR maps , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Martin Lauer,et al.  Photometric laser scanner to camera calibration for low resolution sensors , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).