An analog neural network processor with programmable topology

The architecture, implementation, and applications of a special-purpose neural network processor are described. The chip performs over 2000 multiplications and additions simultaneously. Its data path is particularly suitable for the convolutional topologies that are typical in classification networks, but can also be configured for fully connected or feedback topologies. Resources can be multiplexed to permit implementation of networks with several hundreds of thousands of connections on a single chip. Computations are performed with 6 b accuracy for the weights and 3 b for the neuron states. Analog processing is used internally for reduced power dissipation and higher density, but all input/output is digital to simplify system integration. The practicality of the chip is demonstrated with an implementation of a neural network for optical character recognition. This network contains over 130000 connections and was evaluated in 1 ms. >

[1]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[2]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[3]  S. P. Pekarich,et al.  The DSP32C: AT&Ts second generation floating point digital signal processor , 1988, IEEE Micro.

[4]  Richard Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[5]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[6]  W. Groeneveld,et al.  A self calibration technique for monolithic high-resolution D/A converters , 1989, IEEE International Solid-State Circuits Conference, 1989 ISSCC. Digest of Technical Papers.

[7]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[8]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[9]  H.P. Graf,et al.  A reconfigurable CMOS neural network , 1990, 1990 37th IEEE International Conference on Solid-State Circuits.

[10]  Y. Tamura,et al.  A BiCMOS analog neural network with dynamically updated weights , 1992, 1990 37th IEEE International Conference on Solid-State Circuits.

[11]  Isabelle Guyon,et al.  Neural Network Implementation of Admission Control , 1990, NIPS.

[12]  Steven B. Bibyk,et al.  Programmable current-mode neural network for implementation in analogue MOS VLSI , 1990 .

[13]  Robert B. Allen,et al.  Relaxation Networks for Large Supervised Learning Problems , 1990, NIPS.

[14]  Bernard Widrow,et al.  Sensitivity of feedforward neural networks to weight errors , 1990, IEEE Trans. Neural Networks.

[15]  Y. Le Cun,et al.  VLSI implementations of electronic neural networks: an example in character recognition , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[16]  Y. Arima,et al.  A 336-neuron 28k-synapse Self-learning Neural Network Chip With Branch-neuron-unit Architecture , 1991, 1991 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.

[17]  L.D. Jackel,et al.  An analog neural network processor and its application to high-speed character recognition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[18]  Isabelle Guyon,et al.  Design of a neural network character recognizer for a touch terminal , 1991, Pattern Recognit..