Learning algorithms for tracking changing concepts and an investigation into the error surfaces of single artificial neurons

This thesis is divided into two distinct parts. In part I, we present on-line learning algorithms which are designed for combining classifiers or experts in nonstationary environments. The motivating assumption of our work is as follows: given a set of classifiers, and a data sequence whose underlying generative process is changing over time, we expect that different classifiers will perform well over different segments of the sequence. We give an algorithm whose performance is an additive term larger than the performance of the “best” sequence of classifiers. The additive term measures the degree of nonstationarity in the sequence of classifiers. In Part II, we consider some simple results which characterize certain properties of the error surface of an artificial neuron. We prove three results: one result is a sufficient condition for local minima (in fact “exponentially” many local minima), and two results are sufficient conditions for no local minima. These results hold for a wide variety of artificial neurons.