Algorithmic mapping of neural network models onto parallel SIMD machines

The authors consider parallel implementation of neural network computations of fine grain SIMD machines. The authors show a mapping of a neural network having n nodes and e connections onto a parallel machine having (n+e) PEs arranged in an array of square root n+e* square root n+e PEs such that routing for each update iteration of the recall phase can be performed in 24( square root n+e-1) elemental data shifts. The array uses simple PEs and few local registers to perform the routing and computations. The method is simple and is well suited for implementation of various classes of neural networks on many currently available parallel machines.<<ETX>>

[1]  D. S. Touretzky,et al.  Neural network simulation at Warp speed: how we got 17 million connections per second , 1988, IEEE 1988 International Conference on Neural Networks.

[2]  Sartaj Sahni,et al.  Data broadcasting in SIMD computers , 1981, IEEE Transactions on Computers.

[3]  Sartaj Sahni,et al.  Optimal BPC Permutations on a Cube Connected SIMD Computer , 1982, IEEE Transactions on Computers.

[4]  K. W. Przytula,et al.  Mapping of neural networks onto programmable parallel machines , 1990, IEEE International Symposium on Circuits and Systems.

[5]  S. Tomboulian.  Overview and extensions of a system for routing directed graphs on SIMD architectures , 1988, Proceedings., 2nd Symposium on the Frontiers of Massively Parallel Computation.

[6]  Nam Ling,et al.  Systolic architectures for artificial neural nets , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[7]  Geoffrey E. Hinton,et al.  Connectionist Architectures for Artificial Intelligence , 1990, Computer.

[8]  S. Y. Kung,et al.  Parallel architectures for artificial neural nets , 1988, IEEE 1988 International Conference on Neural Networks.

[9]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[10]  Charles Clos,et al.  A study of non-blocking switching networks , 1953 .