Online gradient descent for least squares regression: Non-asymptotic bounds and application to bandits

We improve the computational complexity of online learning algorithms that require to often recompute least squares regression estimates of parameters. We propose two online gradient descent schemes with randomisation of samples in order to efficiently track the true solutions of the regression problems. Both algorithms result in an O(d) improvement in complexity, where d is the dimension of the data. This makes it attractive to implement these algorithms in the big data setting where d is large. The first algorithm assumes strong convexity in the regression problem, and we provide bounds on the error both in expectation and high probability (the latter is often needed to provide theoretical guarantees for higher level algorithms). The second algorithm deals with cases where strong convexity of the regression problem cannot be guaranteed and uses adaptive regularisation. We combine the first algorithm with the PEGE linear bandit algorithm Rusmevichientong and Tsitsiklis [2010] and show that we lose only logarithmic factors in the regret performance of PEGE. We empirically test the second in a news article recommendation application, which uses the large scale news recommendation dataset from Yahoo! front page. These experiments show a large gain in computational complexity, with no appreciable loss in performance.