Optical Flow Estimation with Channel Constancy

Large motions remain a challenge for current optical flow algorithms. Traditionally, large motions are addressed using multi-resolution representations like Gaussian pyramids. To deal with large displacements, many pyramid levels are needed and, if an object is small, it may be invisible at the highest levels. To address this we decompose images using a channel representation (CR) and replace the standard brightness constancy assumption with a descriptor constancy assumption. CRs can be seen as an over-segmentation of the scene into layers based on some image feature. If the appearance of a foreground object differs from the background then its descriptor will be different and they will be represented in different layers. We create a pyramid by smoothing these layers, without mixing foreground and background or losing small objects. Our method estimates more accurate flow than the baseline on the MPI-Sintel benchmark, especially for fast motions and near motion boundaries.

[1]  Laura Sevilla-Lara,et al.  Distribution fields for tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Hans Knutsson,et al.  Representation and learning of invariance , 1994, Proceedings of 1st International Conference on Image Processing.

[4]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[5]  Michael J. Black,et al.  Learning Optical Flow , 2008, ECCV.

[6]  Matthieu Guillaumin,et al.  Segmentation Propagation in ImageNet , 2012, ECCV.

[7]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[8]  Antti Oulasvirta,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[9]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Joachim Weickert,et al.  Scale-Space Theories in Computer Vision , 1999, Lecture Notes in Computer Science.

[11]  Gösta H. Granlund,et al.  An Associative Perception-Action Structure Using a Localized Space Variant Information Representation , 2000, AFPAC.

[12]  Daniel Cremers,et al.  Advanced Data Terms for Variational Optic Flow Estimation , 2009, VMV.

[13]  Jan J. Koenderink,et al.  Algebraic Frames for the Perception-Action Cycle , 1997, Lecture Notes in Computer Science.

[14]  Michael Felsberg,et al.  Efficient computation of channel-coded feature maps through piecewise polynomials , 2009, Image Vis. Comput..

[15]  David J. Fleet,et al.  Computing optical flow with physical models of brightness variation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Michael Felsberg,et al.  Accurate Interpolation in Appearance-Based Pose Estimation , 2007, SCIA.

[17]  Michael J. Black,et al.  A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them , 2013, International Journal of Computer Vision.

[18]  Andrea J. van Doorn,et al.  The Structure of Locally Orderless Images , 1999, International Journal of Computer Vision.

[19]  Jitendra Malik,et al.  Geometric blur for template matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[21]  Laura Sevilla-Lara,et al.  Distribution Fields with Adaptive Kernels for Large Displacement Image Alignment , 2013, BMVC.

[22]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 2005, International Journal of Computer Vision.

[24]  E. Learned-Miller,et al.  Distribution Fields , 2011 .

[25]  Michael Felsberg,et al.  Adaptive Filtering Using Channel Representations , 2012, Mathematical Methods for Signal and Image Analysis and Representation.

[26]  Michael Felsberg,et al.  Channel smoothing: efficient robust smoothing of low-level signal features , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Luc Florack,et al.  Mathematical Methods for Signal and Image Analysis and Representation , 2012, Computational Imaging and Vision.

[28]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[29]  Jan J. Koenderink,et al.  Discrimination thresholds for channel-coded systems , 1992, Biological Cybernetics.

[30]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[31]  Daniel Cremers,et al.  Large displacement optical flow computation withoutwarping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  Knut-Andreas Lie,et al.  Scale Space and Variational Methods in Computer Vision, Second International Conference, SSVM 2009, Voss, Norway, June 1-5, 2009. Proceedings , 2009, SSVM.

[33]  Shai Avidan,et al.  Locally Orderless Tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[35]  Bram van Ginneken,et al.  Applications of Locally Orderless Images , 2000, J. Vis. Commun. Image Represent..

[36]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[37]  Jan J. Koenderink,et al.  Proceedings of the Second International Workshop on Algebraic Frames for the Perception-Action Cycle , 1997 .

[38]  Leszek Wojnar,et al.  Image Analysis , 1998 .

[39]  Michael Felsberg,et al.  Spatio-Featural Scale-Space , 2009, SSVM.

[40]  Michel Barlaud,et al.  Two deterministic half-quadratic regularization algorithms for computed imaging , 1994, Proceedings of 1st International Conference on Image Processing.