Unsupervised discovery of activity correlations using latent topic models

Topic models such as probabilistic Latent Semantic Analysis (pLSA) and Latent Dirichlet Allocation (LDA) have been successfully used to discover individual activities in a scene. However these methods do not discover group activities which are commonly observed in real life videos of public places. In this paper we address the problem of discovering activities and their associations as a group activity in an unsupervised manner. We propose a method that uses a two layer hierarchical latent structure to correlate individual activities in lower layer with group activity in higher layer. Our model considers each scene to be composed of a mixture of group activities. Each group activity is in turn composed as a mixture of individual activities represented as multinomial distributions. Each individual activity is represented as a distribution over local visual features. We use a Gibbs sampling based algorithm to infer these activities. Our method can summarize not only the individual activities but also the common group activities in a video. We demonstrate the strength of our method by mining activities and the salient correlation amongst them in real life videos of crowded public scenes.