Instance Segmentation by Learning Pixel Neighbour Relations with a CNN

Instance segmentation is an important task in the context of automated vehicles. It allows to get precise object locations and shapes from camera images. In this paper, we propose an approach to instance segmentation based on deep learning. A convolutional neural network predicts relations between neighbour pixels. It also predicts which regions of the image lie inside instances. This representation is then decoded to get the object instances. Our approach achieves an average precision of 13.9 % over all classes on the validation dataset of the challenging Cityscapes benchmark. In contrast to other encoding based approaches, the presented approach can associate freestanding (unconnected) areas of the same instance. This problem can arise if an object is partially occluded. It is solved by learning long-range neighbour relations that do not only consider direct neighbour pixels. We show that these long-range neighbour relations result in higher accuracies for all classes.