Hidden Markov models with graph densities for action recognition

Human action recognition in video streams is a fast developing field in pattern recognition and machine learning. Local image representations, e.g. space-time interest points [1], have proven to be the current most reliable choice of feature in sequences in which the region of interest is difficult to determine [2]. However, the question how to deal with more severe occlusions has large been ignored [2]. This work proposes a new approach which directly addresses heavy occlusions by modeling the skeleton-based features using a probability density functions (PDF) defined over graphs. We integrated the proposed density into an hidden Markov model (HMM) to model sequences of graphs of arbitrary sizes, i.e. occlusions setting may change over time. The approach is evaluated using a dataset embracing three action classes, studying six different types of occlusions (involving the removal of subgraphs from the graphical representation of action sequence). The presented study shows clearly that actions from even heavily occluded sequences can be reliably recognized.