Non linear neurons in the low noise limit : a factorial code maximizes information transferJean

We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focussing on the case of non linear transfer functions. We assume that both receptive elds (synaptic eecacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker'sinfomax principle) leads to a factorial code-hence to the same solution as required by the redundancy reduction principle of Barlow. We show also that this result is valid for linear, more generally unbounded, transfer functions, provided optimization is performed under an additive constraint, that is which can be written as a sum of terms, each one being speciic to one output neuron. Finally we study the eeect of a non zero input noise. We nd that, at rst order in the input noise, assumed to be small as compared to the-small-output noise, the above results are still valid, provided the output noise is uncorrelated from one neuron to the other. P.A.C.S. 87.30 Biophysics of neurophysiological processes Short title: Information maximization with non linear neurons To appear in NETWORK INDEX: nadalparga.infomaxredred.ps.Z nadal@physique.ens.fr 19 pages Infomax applied to non linear neurons, in the low noise limit, leads to redundancy reduction.