Fast projection onto the simplex and the $$\pmb {l}_\mathbf {1}$$l1 ball

A new algorithm is proposed to project, exactly and in finite time, a vector of arbitrary size onto a simplex or an $$l_1$$l1-norm ball. It can be viewed as a Gauss–Seidel-like variant of Michelot’s variable fixing algorithm; that is, the threshold used to fix the variables is updated after each element is read, instead of waiting for a full reading pass over the list of non-fixed elements. This algorithm is empirically demonstrated to be faster than existing methods.

[1]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[2]  Manuel Blum,et al.  Time Bounds for Selection , 1973, J. Comput. Syst. Sci..

[3]  Philip Wolfe,et al.  Validation of subgradient optimization , 1974, Math. Program..

[4]  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of ∝n , 1986 .

[5]  Jon Louis Bentley,et al.  Engineering a sort function , 1993, Softw. Pract. Exp..

[6]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[7]  Krzysztof C. Kiwiel.  On Floyd and Rivest's SELECT algorithm , 2005, Theor. Comput. Sci..

[8]  K. Kiwiel.  Variable Fixing Algorithms for the Continuous Quadratic Knapsack Problem , 2008 .

[9]  Michael Patriksson,et al.  A survey on the continuous nonlinear resource allocation problem , 2008, Eur. J. Oper. Res..

[10]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[11]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[12]  Krzysztof C. Kiwiel.  Breakpoint searching algorithms for the continuous quadratic knapsack problem , 2008, Math. Program..

[13]  Jun Liu,et al.  Efficient Euclidean projections in linear time , 2009, ICML '09.

[14]  I. Daubechies,et al.  Sparse and stable Markowitz portfolios , 2007, Proceedings of the National Academy of Sciences.

[15]  Mohamed-Jalal Fadili,et al.  Total Variation Projection With First Order Schemes , 2009, IEEE Transactions on Image Processing.

[16]  A. Fiacco.  A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R " , 2009 .

[17]  Jing Yuan,et al.  Convex Multi-class Image Labeling by Simplex-Constrained Total Variation , 2009, SSVM.

[18]  Changshui Zhang,et al.  Efficient Euclidean projections via Piecewise Root Finding and its application in gradient projection , 2011, Neurocomputing.

[19]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Naonori Ueda,et al.  Large-Scale Multiclass Support Vector Machine Training via Euclidean Projection onto the Simplex , 2014, 2014 22nd International Conference on Pattern Recognition.

[21]  Roberto Cominetti,et al.  A Newton’s method for the continuous quadratic knapsack problem , 2014, Math. Program. Comput..

[22]  Michael Patriksson,et al.  Algorithms for the continuous nonlinear resource allocation problem - New implementations and numerical studies , 2015, Eur. J. Oper. Res..