Robust incremental optical flow

This thesis addresses the problem of recovering 2D image velocity, or optical flow, robustly over long image sequences. We develop a robust estimation framework for improving the reliability of motion estimates and an incremental minimization framework for recovering flow estimates over time. Attempts to improve the robustness of optical flow have focused on detecting and accounting for motion discontinuities in the optical flow field. We show that motion discontinuities are one example of a more general class of model violations and that by formulating the optical flow problem as one of robust estimation the problems posed by motion discontinuities can be reduced, and the violations can be detected. Additionally, robust estimation provides a powerful framework for early vision problems that generalizes the popular "line process" approaches. We formulate a temporal continuity constraint, which reflects the fact that the motion of a surface changes gradually over time. We exploit this constraint to develop a new incremental minimization framework and show how it is related to standard recursive estimation techniques. Within this framework we implement two incremental algorithms for minimizing non-convex objective functions over time; Incremental Stochastic Minimization (ISM) and Incremental Graduated Non-Convexity (IGNC). With this approach, motion estimates are always available; they are refined over time, the algorithm adapts to scene changes, and the amount of computation between frames is kept fixed. The psychophysical implications of temporal continuity are discussed and the power of the incremental minimization framework is demonstrated by extending image feature extraction over time.