Comparison of Deep Transfer Learning Strategies for Digital Pathology

In this paper, we study deep transfer learning as a way of overcoming object recognition challenges encountered in the field of digital pathology. Through several experiments, we investigate various uses of pre-trained neural network architectures and different combination schemes with random forests for feature selection. Our experiments on eight classification datasets show that densely connected and residual networks consistently yield best performances across strategies. It also appears that network fine-tuning and using inner layers features are the best performing strategies, with the former yielding slightly superior results.

[1]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[2]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[3]  Yolanda T. Chong,et al.  Automated analysis of high‐content microscopy data with deep learning , 2017, Molecular systems biology.

[4]  Hamid R. Tizhoosh,et al.  A comparative study of CNN, BoVW and LBP for classification of histopathological images , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[5]  Bram van Ginneken,et al.  Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[6]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[7]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[8]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[9]  Hayit Greenspan,et al.  Chest pathology detection using deep learning with non-medical training , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[10]  Raphaël Marée,et al.  The Need for Careful Data Collection for Pattern Recognition in Digital Pathology , 2017, Journal of pathology informatics.

[11]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Yuanjie Zheng,et al.  Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model , 2017, Scientific Reports.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[17]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[18]  Janne Heikkilä,et al.  Transfer Learning for Cell Nuclei Classification in Histopathology Images , 2016, ECCV Workshops.

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Joel H. Saltz,et al.  Automatic histopathology image analysis with CNNs , 2016, 2016 New York Scientific Data Summit (NYSDS).

[21]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[22]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[23]  Bram van Ginneken,et al.  Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box , 2015, Medical Image Anal..

[24]  Hamid R. Tizhoosh,et al.  Convolutional neural networks for histopathology image classification: Training vs. Using pre-trained networks , 2017, 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA).

[25]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[26]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[27]  Gilles Louppe,et al.  Collaborative analysis of multi-gigapixel imaging data using Cytomine , 2016, Bioinform..

[28]  Raphaël Marée,et al.  Towards generic image classification using tree-based learning: An extensive empirical study , 2016, Pattern Recognit. Lett..

[29]  Jean-Christophe Olivo-Marin,et al.  An approach for detection of glomeruli in multisite digital pathology , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[30]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Hariharan Ravishankar,et al.  Understanding the Mechanisms of Deep Transfer Learning for Medical Images , 2016, LABELS/DLMIA@MICCAI.

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[34]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[35]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[36]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[37]  Bahram Parvin,et al.  Automated Histology Analysis: Opportunities for signal processing , 2015, IEEE Signal Processing Magazine.

[38]  Edward Kim,et al.  A deep semantic mobile application for thyroid cytopathology , 2016, SPIE Medical Imaging.

[39]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[40]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Quoc V. Le,et al.  Do Better ImageNet Models Transfer Better? , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[43]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[44]  Hongfei Lin,et al.  Deep Transfer Learning for Modality Classification of Medical Images , 2017, Inf..

[45]  Joseph Antony,et al.  Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).