Optimized Deep Encoder-Decoder Methods for Crack Segmentation

Continuous maintenance of concrete infrastructure is an important task which is needed to continue safe operations of these structures. One kind of defect that occurs on surfaces in these structures are cracks. Automatic detection of those cracks poses a challenging computer vision task as background, shape, colour and size of cracks vary. In this work we propose optimized deep encoder-decoder methods consisting of a combination of techniques which yield an increase in crack segmentation performance. Specifically, we propose a new design for the decoder-part in encoder-decoder based deep learning architectures for semantic segmentation. We study its composition and how to achieve increased performance by exploring components such as deep supervision and upsampling strategies. Then we examine the optimal encoder to go in conjunction with this decoder and determine that pretrained encoders lead to an increase in performance. We propose a data augmentation strategy to increase the amount of available training data and carry out the performance evaluation of the designed architecture on four publicly available crack segmentation datasets. Additionally, we introduce two techniques into the field of surface crack segmentation, previously not used there: Generating results using test-time-augmentation and performing a statistical result analysis over multiple training runs. The former approach generally yields increased performance results, whereas the latter allows for more reproducible and better representability of a methods results. Using those aforementioned strategies with our proposed encoder-decoder architecture we are able to achieve new state of the art results in all datasets.

[1]  Lin Gao,et al.  Automated Pavement Crack Damage Detection Using Deep Multiscale Convolutional Features , 2020, Journal of Advanced Transportation.

[2]  Gordon Morison,et al.  A Convolutional Neural Network for Pavement Surface Crack Segmentation Using Residual Connections and Attention Gating , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[3]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[4]  Khurram Kamal,et al.  Pavement crack detection using the Gabor filter , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[5]  Kelvin C. P. Wang,et al.  Wavelet-Based Pavement Distress Image Edge Detection with À Trous Algorithm , 2007 .

[6]  Qian Wang,et al.  DeepCrack: Learning Hierarchical Convolutional Features for Crack Detection , 2019, IEEE Transactions on Image Processing.

[7]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Gordon Morison,et al.  A Deep Convolutional Neural Network for Semantic Pixel-Wise Segmentation of Road and Pavement Surface Cracks , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[9]  R. Siezen,et al.  others , 1999, Microbial Biotechnology.

[10]  Kelwin Fernandes,et al.  Pavement pathologies classification using graph-based features , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[12]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Bo Chen,et al.  MnasNet: Platform-Aware Neural Architecture Search for Mobile , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[15]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Gordon Morison,et al.  Segmentation of Surface Cracks Based on a Fully Convolutional Neural Network and Gated Scale Pooling , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[17]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Stefano Marrone,et al.  Reproducibility of Deep CNN for Biomedical Image Processing Across Frameworks and Architectures , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[19]  Sumathi Poobal,et al.  Crack detection using image processing: A critical review and analysis , 2017, Alexandria Engineering Journal.

[20]  Horst-Michael Groß,et al.  How to get pavement distress detection ready for deep learning? A systematic approach , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[21]  Katsuto Yamaki,et al.  Road deformation detection based sensor fusion , 2017, 2017 9th International Conference on Information Technology and Electrical Engineering (ICITEE).

[22]  Kun Qian,et al.  Automated Detection of Steel Defects via Machine Learning based on Real-Time Semantic Segmentation , 2019, ICVIP.

[23]  Paulo Lobato Correia,et al.  Automatic road crack segmentation using entropy and image dynamic thresholding , 2009, 2009 17th European Signal Processing Conference.

[24]  Li Li,et al.  DeepCrack: A deep hierarchical feature learning architecture for crack segmentation , 2019, Neurocomputing.

[25]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[26]  Vijayan K. Asari,et al.  Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation , 2018, ArXiv.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Quoc V. Le,et al.  EfficientDet: Scalable and Efficient Object Detection , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[31]  S. Chambon,et al.  Automatic Road Pavement Assessment with Image Processing: Review and Comparison , 2011 .

[32]  Peggy Subirats,et al.  Automation of Pavement Surface Crack Detection using the Continuous Wavelet Transform , 2006, 2006 International Conference on Image Processing.

[33]  Michael Kampffmeyer,et al.  ConnNet: A Long-Range Relation-Aware Pixel-Connectivity Network for Salient Segmentation , 2018, IEEE Transactions on Image Processing.

[34]  Zhun Fan,et al.  Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network , 2018, ArXiv.

[35]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Hui-li Zhao,et al.  Improvement of canny algorithm based on pavement edge detection , 2010, 2010 3rd International Congress on Image and Signal Processing.

[37]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[38]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[39]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[42]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Qi Chen,et al.  FPCNet: Fast Pavement Crack Detection Network Based on Encoder-Decoder Architecture , 2019, ArXiv.

[45]  Naoki Tanaka,et al.  A Crack Detection Method in Road Surface Images Using Morphology , 1998, MVA.

[46]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[47]  B. Koonce EfficientNet , 2021, Convolutional Neural Networks with Swift for Tensorflow.

[48]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[49]  Fan Yang,et al.  Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection , 2019, IEEE Transactions on Intelligent Transportation Systems.

[50]  Ying Chen,et al.  Ensemble of Deep Convolutional Neural Networks for Automatic Pavement Crack Detection and Measurement , 2020, Coatings.

[51]  Yoshihiko Hamamoto,et al.  A Method for Crack Detection on a Concrete Structure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[52]  Gordon Morison,et al.  Road crack detection using a single stage detector based deep neural network , 2018, 2018 IEEE Workshop on Environmental, Energy, and Structural Monitoring Systems (EESMS).

[53]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[54]  Guosheng Lin,et al.  RefineNet: Multi-Path Refinement Networks for Dense Prediction , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Zhaozheng Hu,et al.  Grid-based pavement crack analysis using deep learning , 2017, 2017 4th International Conference on Transportation Information and Safety (ICTIS).

[56]  Paulo Lobato Correia,et al.  Automatic Road Crack Detection and Characterization , 2013, IEEE Transactions on Intelligent Transportation Systems.

[57]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[58]  Hiroto Nagayoshi,et al.  Deployment Conscious Automatic Surface Crack Detection , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[59]  Yuan Xie,et al.  Instance-Level Salient Object Segmentation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[61]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[62]  Wang Chao,et al.  Research on Crack Detection Method of Airport Runway Based on Twice-Threshold Segmentation , 2015, 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC).

[63]  Shaojie Shen,et al.  Real-Time Dense Stereo Embedded in a UAV for Road Inspection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[64]  Bo Shaobo,et al.  Pavement Crack Detection and Analysis for High-grade Highway , 2007, 2007 8th International Conference on Electronic Measurement and Instruments.

[65]  Nikolaos Doulamis,et al.  Deep Convolutional Neural Networks for efficient vision based tunnel inspection , 2015, 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).