Automatic Vertebra Labeling in Large-Scale 3D CT using Deep Image-to-Image Network with Message Passing and Sparsity Regularization

Automatic localization and labeling of vertebra in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. However, the unusual conditions of pathological cases, such as the abnormal spine curvature, bright visual imaging artifacts caused by metal implants, and the limited field of view, increase the difficulties of accurate localization. In this paper, we propose an automatic and fast algorithm to localize and label the vertebra centroids in 3D CT volumes. First, we deploy a deep image-to-image network (DI2IN) to initialize vertebra locations, employing the convolutional encoder-decoder architecture together with multi-level feature concatenation and deep supervision. Next, the centroid probability maps from DI2IN are iteratively evolved with the message passing schemes based on the mutual relation of vertebra centroids. Finally, the localization results are refined with sparsity regularization. The proposed method is evaluated on a public dataset of 302 spine CT volumes with various pathologies. Our method outperforms other state-of-the-art methods in terms of localization accuracy. The run time is around 3 seconds on average per case. To further boost the performance, we retrain the DI2IN on additional 1000+ 3D CT volumes from different patients. To the best of our knowledge, this is the first time more than 1000 3D CT volumes with expert annotation are adopted in experiments for the anatomic landmark detection tasks. Our experimental results show that training with such a large dataset significantly improves the performance and the overall identification rate, for the first time by our knowledge, reaches 90%.

[1]  K. Doi,et al.  Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules. , 2000, AJR. American journal of roentgenology.

[2]  Max Mignotte,et al.  3D/2D registration and segmentation of scoliotic vertebrae using statistical models. , 2003, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[3]  Xiaogang Wang,et al.  Structured Feature Learning for Pose Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Ben Glocker,et al.  Vertebrae Localization in Pathological Spine CT via Dense Classification from Sparse Annotations , 2013, MICCAI.

[5]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[6]  Hao Chen,et al.  Automatic Localization and Identification of Vertebrae in Spine CT via a Joint Learning Model with Deep Neural Networks , 2015, MICCAI.

[7]  David J. Kriegman,et al.  Dense Volume-to-Volume Vascular Boundary Detection , 2016, MICCAI.

[8]  Xiaogang Wang,et al.  End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[10]  Timothy F. Cootes,et al.  Vertebral Shape: Automatic Measurement with Dynamically Sequenced Active Appearance Models , 2005, MICCAI.

[11]  Hao Chen,et al.  3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes , 2016, MICCAI.

[12]  Nikos Komodakis,et al.  MRF Optimization via Dual Decomposition: Message-Passing Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Horst Bischof,et al.  Regressing Heatmaps for Multiple Landmark Localization Using CNNs , 2016, MICCAI.

[14]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[15]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Martial Hebert,et al.  Learning message-passing inference machines for structured prediction , 2011, CVPR 2011.

[17]  M. Nevitt,et al.  Vertebral fracture assessment using a semiquantitative technique , 1993, Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research.

[18]  Cristian Lorenz,et al.  Automated model-based vertebra detection, identification, and segmentation in CT images , 2009, Medical Image Anal..

[19]  Tomaz Slivnik,et al.  3-D/2-D registration of CT and MR to X-ray images , 2003, IEEE Transactions on Medical Imaging.

[20]  Yilong Yin,et al.  Information Processing in Medical Imaging , 2017, Lecture Notes in Computer Science.

[21]  Ronald M. Summers,et al.  A multi-center milestone study of clinical vertebral CT segmentation , 2016, Comput. Medical Imaging Graph..

[22]  Purang Abolmaesumi,et al.  Fast Automatic Vertebrae Detection and Localization in Pathological CT Scans - A Deep Learning Approach , 2015, MICCAI.

[23]  F. Langlotz,et al.  Accuracy of Computer‐Assisted Pedicle Screw Placement: An In Vivo Computed Tomography Analysis , 1997, Spine.

[24]  Stefan Jaeger,et al.  Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. , 2014, Quantitative imaging in medicine and surgery.

[25]  Nicholas Ayache,et al.  Geometric Variability of the Scoliotic Spine Using Statistics on Articulated Shape Models , 2008, IEEE Transactions on Medical Imaging.

[26]  Sebastian Nowozin,et al.  Structured Learning and Prediction in Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[27]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[28]  Clement J. McDonald,et al.  Preparing a collection of radiology examinations for distribution and retrieval , 2015, J. Am. Medical Informatics Assoc..

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  Sebastian P. M. Dries,et al.  Spine Detection and Labeling Using a Parts-Based Graphical Model , 2007, IPMI.

[31]  Ben Glocker,et al.  Automatic Localization and Identification of Vertebrae in Arbitrary Field-of-View CT Scans , 2012, MICCAI.