Robust Change Detection Based on Neural Descriptor Fields

The ability to reason about changes in the environment is crucial for robots operating over extended periods of time. Agents are expected to capture changes during operation so that actions can be followed to ensure a smooth progression of the working session. However, varying viewing angles and accumulated localization errors make it easy for robots to falsely detect changes in the surrounding world due to low observation overlap and drifted object associations. In this paper, based on the recently proposed category-level Neural Descriptor Fields (NDFs), we develop an object-level online change detection approach that is robust to partially overlapping observations and noisy localization results. Utilizing the shape completion capability and SE(3)-equivariance of NDFs, we represent objects with compact shape codes encoding full object shapes from partial observations. The objects are then organized in a spatial tree structure based on object centers recovered from NDFs for fast queries of object neighborhoods. By associating objects via shape code similarity and comparing local object-neighbor spatial layout, our proposed approach demonstrates robustness to low observation overlap and localization noises. We conduct experiments on both synthetic and real-world sequences and achieve improved change detection results compared to multiple baseline methods. Project web-page: ?http://yilundu.github.io/ndf_change

[1]  Vincent Sitzmann,et al.  Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[2]  Juan I. Nieto,et al.  Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[3]  Roland Siegwart,et al.  NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping , 2021, 2021 International Conference on 3D Vision (3DV).

[4]  Li Fei-Fei,et al.  ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations , 2021, CoRL.

[5]  Andrea Tagliasacchi,et al.  Vector Neurons: A General Framework for SO(3)-Equivariant Networks , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Shimin Hu,et al.  DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jiaolong Yang,et al.  Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Carlos Campos,et al.  ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM , 2020, IEEE Transactions on Robotics.

[9]  Robert Babuska,et al.  Change detection using weighted features for image-based localization , 2021, Robotics Auton. Syst..

[10]  M. Vincze,et al.  Robust and Efficient Object Change Detection by Combining Global Semantic Information and Local Geometric Verification , 2020, IEEE/RJS International Conference on Intelligent RObots and Systems.

[11]  Andrew Davison,et al.  NodeSLAM: Neural Object Descriptors for Multi-View Shape Reconstruction , 2020, 2020 International Conference on 3D Vision (3DV).

[12]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[13]  Kai Xu,et al.  Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Marc Pollefeys,et al.  Convolutional Occupancy Networks , 2020, ECCV.

[15]  Andreas Geiger,et al.  Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Gordon Wetzstein,et al.  Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations , 2019, NeurIPS.

[17]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Sebastian Nowozin,et al.  Occupancy Networks: Learning 3D Reconstruction in Function Space , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Roland Siegwart,et al.  TSDF-based change detection for consistent long-term dense reconstruction and dynamic object discovery , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[21]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[22]  Rares Ambrus,et al.  Meta-rooms: Building and maintaining long term spatial models in a dynamic world , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  John J. Leonard,et al.  Toward lifelong object segmentation from change detection in dense RGB-D maps , 2013, 2013 European Conference on Mobile Robots.

[24]  Dieter Fox,et al.  Toward object discovery and modeling via 3-D scene comparison , 2011, 2011 IEEE International Conference on Robotics and Automation.

[25]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.