Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks

We introduce a new form of computational unit for feedforward learning networks of the backpropagation type. Instead of calculating a weighted sum this unit calculates a weighted product, where each input is raised to a power determined by a variable weight. Such a unit can learn an arbitrary polynomial term, which would then feed into higher level standard summing units. We show how learning operates with product units, provide examples to show their efficiency for various types of problems, and argue that they naturally extend the family of theoretical feedforward net structures. There is a plausible neurobiological interpretation for one interesting configuration of product and summing units.

[1]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[2]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[3]  Robert O. Winder,et al.  Enumeration of Seven-Argument Threshold Functions , 1965, IEEE Trans. Electron. Comput..

[4]  D. Howard Giles, and Y. , 1973 .

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[7]  T. Bliss,et al.  NMDA receptors - their role in long-term potentiation , 1987, Trends in Neurosciences.

[8]  Stanley Munsat,et al.  Keeping representations at bay , 1990, Behavioral and Brain Sciences.

[9]  W. Maki Toward a unification of conditioning and cognition in animal learning , 1990, Behavioral and Brain Sciences.

[10]  James A. Hendler,et al.  But what is the substance of connectionist representation? , 1990, Behavioral and Brain Sciences.

[11]  John E. Rager,et al.  The analysis of the learning needs to be deeper , 1990, Behavioral and Brain Sciences.

[12]  William A. Phillips,et al.  Realistic neural nets need to learn iconic representations , 1990, Behavioral and Brain Sciences.

[13]  Steven José Hanson,et al.  Learning and representation: Tensions at the interface , 1990, Behavioral and Brain Sciences.

[14]  Bill Baird,et al.  Bifurcation and category learning in network models of oscilating cortex , 1990 .

[15]  Karl Haberlandt,et al.  Expose hidden assumptions in network theory , 1990, Behavioral and Brain Sciences.

[16]  G. Toulouse Advances in neural network theory , 1990, Behavioral and Brain Sciences.

[17]  Shabtai Barash,et al.  Relatively local neurons in a distributed representation: A neurophysiological perspective , 1990, Behavioral and Brain Sciences.

[18]  Tim van Gelder,et al.  Connectionist models learn what? , 1990, Behavioral and Brain Sciences.

[19]  Richard M. Golden,et al.  Are connectionist models just statistical pattern classifiers? , 1990, Behavioral and Brain Sciences.

[20]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[21]  William Timberlake,et al.  What connectionist models learn: Learning and representation in connectionist networks , 1990, Behavioral and Brain Sciences.

[22]  Patrick Suppes,et al.  Problems of extension, representation, and computational irreducibility , 1990, Behavioral and Brain Sciences.

[23]  Stephen José Hanson,et al.  What connectionist models learn: Learning and representation in connectionist networks , 1990, Behavioral and Brain Sciences.

[24]  John K. Kruschke,et al.  How connectionist models learn: The course of learning in connectionist networks , 1990, Behavioral and Brain Sciences.

[25]  P. Langley Approaches to learning and representation , 1990, Behavioral and Brain Sciences.

[26]  N. Sharkey There is more to learning then meeth the eye (or ear) , 1990, Behavioral and Brain Sciences.

[27]  Misha Pavel,et al.  Learning from learned networks , 1990, Behavioral and Brain Sciences.

[28]  Nick Chater,et al.  Connectionism and classical computation , 1990, Behavioral and Brain Sciences.

[29]  Géry d'Ydewalle,et al.  What can psychologists learn from hidden-unit nets? , 1990, Behavioral and Brain Sciences.

[30]  Gordon D. A. Brown,et al.  Representational systems and symbolic systems , 1990, Behavioral and Brain Sciences.

[31]  Michael I. Jordan A non-empiricist perspective on learning in layered networks , 1990, Behavioral and Brain Sciences.

[32]  Stephen Kaplan,et al.  Connectionist learning and the challenge of real environments , 1990, Behavioral and Brain Sciences.

[33]  B. Bridgeman What connectionists learn: Comparisons of model and neural nets , 1990, Behavioral and Brain Sciences.