Reflection methods for user-friendly submodular optimization

Recently, it has become evident that submodularity naturally captures widely occurring concepts in machine learning, signal processing and computer vision. Consequently, there is need for efficient optimization procedures for submodular functions, especially for minimization problems. While general submodular minimization is challenging, we propose a new method that exploits existing decomposability of submodular functions. In contrast to previous approaches, our method is neither approximate, nor impractical, nor does it need any cumbersome parameter tuning. Moreover, it is easy to implement and parallelize. A key component of our method is a formulation of the discrete submodular minimization problem as a continuous best approximation problem that is solved through a sequence of reflections, and its solution can be easily thresholded to obtain an optimal discrete solution. This method solves both the continuous and discrete formulations of the problem, and therefore has applications in learning, inference, and reconstruction. In our experiments, we illustrate the benefits of our method on two image segmentation tasks.

[1]  James B. Orlin,et al.  A Faster Strongly Polynomial Time Algorithm for Submodular Function Minimization , 2007, IPCO.

[2]  Pushmeet Kohli,et al.  Robust Higher Order Potentials for Enforcing Label Consistency , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[4]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Suvrit Sra,et al.  Fast Newton-type Methods for Total Variation Regularization , 2011, ICML.

[6]  James B. Orlin,et al.  A faster strongly polynomial time algorithm for submodular function minimization , 2007, Math. Program..

[7]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[9]  P. Stobbe,et al.  Convex Analysis for Minimizing and Learning Submodular Set Functions , 2013 .

[10]  Hui Lin,et al.  On fast approximate submodular minimization , 2011, NIPS.

[11]  Christoph Schnörr,et al.  A study of Nesterov's scheme for Lagrangian decomposition and MAP labeling , 2011, CVPR 2011.

[12]  Satoru Iwata,et al.  A network flow approach to cost allocation for rooted trees , 2004 .

[13]  Václav Hlavác,et al.  A Distributed Mincut/Maxflow Algorithm Combining Path Augmentation and Push-Relabel , 2011, International Journal of Computer Vision.

[14]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Antonin Chambolle,et al.  On Total Variation Minimization and Surface Evolution Using Parametric Maximum Flows , 2009, International Journal of Computer Vision.

[16]  Dorit S. Hochbaum,et al.  About strongly polynomial time algorithms for quadratic optimization over submodular constraints , 1995, Math. Program..

[17]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Christoph Schnörr,et al.  Efficient MRF Energy Minimization via Adaptive Diminishing Smoothing , 2012, UAI.

[19]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[20]  Tommi S. Jaakkola,et al.  Convergence Rate Analysis of MAP Coordinate Minimization Algorithms , 2012, NIPS.

[21]  Satoru Iwata,et al.  Submodular function minimization , 2007, Math. Program..

[22]  Heinz H. Bauschke,et al.  Finding best approximation pairs relative to two closed convex sets in Hilbert spaces , 2004, J. Approx. Theory.

[23]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[24]  VekslerOlga,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001 .

[25]  Andreas Krause,et al.  Submodularity and its applications in optimized information gathering , 2011, TIST.

[26]  藤重 悟 Submodular functions and optimization , 1991 .

[27]  Julien Mairal,et al.  Proximal Methods for Hierarchical Sparse Coding , 2010, J. Mach. Learn. Res..

[28]  Fabián A. Chudak,et al.  Efficient solutions to relaxations of combinatorial problems with submodular penalties via the Lovász extension and non-smooth convex optimization , 2007, SODA '07.

[29]  Andreas Krause,et al.  Efficient Minimization of Decomposable Submodular Functions , 2010, NIPS.

[30]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[31]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .

[32]  H. Groenevelt Two algorithms for maximizing a separable concave function over a polymatroid feasible region , 1991 .

[33]  Robert E. Tarjan,et al.  Balancing Applied to Maximum Network Flow Problems , 2006, ESA.

[34]  Nikos Komodakis,et al.  MRF Energy Minimization and Beyond via Dual Decomposition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jack Edmonds,et al.  Submodular Functions, Matroids, and Certain Polyhedra , 2001, Combinatorial Optimization.

[36]  Satoru Fujishige,et al.  Lexicographically Optimal Base of a Polymatroid with Respect to a Weight Vector , 1980, Math. Oper. Res..

[37]  Vladimir Kolmogorov,et al.  Minimizing a sum of submodular functions , 2010, Discret. Appl. Math..

[38]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[39]  Vahab S. Mirrokni,et al.  Maximizing Non-Monotone Submodular Functions , 2011, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[40]  Francis R. Bach,et al.  Learning with Submodular Functions: A Convex Optimization Perspective , 2011, Found. Trends Mach. Learn..

[41]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[42]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[43]  Satoru Iwata,et al.  A network flow approach to cost allocation for rooted trees , 2004, Networks.

[44]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[45]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[46]  S. Fujishige,et al.  A Submodular Function Minimization Algorithm Based on the Minimum-Norm Base ⁄ , 2009 .

[47]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..