Linear neural networks which minimize the output variance

The authors analyze constrained linear architectures which learn according to Hebb's rule to minimize the output energy. They study the conditions under which such networks act as decorrelating (square-root) filters. In particular, it is shown how constrained architectures can decorrelate efficiently by simply using Hebb's rule. The authors extend the analysis to networks with arbitrary interconnections. The purpose is the design of useful architectures and the understanding of the functionality of patterns of connectivity observed in biological systems. The authors deal only with linear neurons performing simple linear combinations. The authors restrict attention to decorrelating networks which do not use the output variance to compress the input space into a new space with a smaller number of dimensions.<<ETX>>