Temporally continuous vs. clocked networks

We discuss advantages and disadvantages of temporally continuous neural networks in contrast to clocked ones, and continue with some “tricks of the trade” of continuous time and recurrent neural networks.

[1]  Patrice Y. Simard,et al.  Shaping the State Space Landscape in Recurrent Networks , 1990, NIPS.

[2]  Terrence J. Sejnowski,et al.  A Dynamic Neural Network Model of Sensorimotor Transformations in the Leech , 1990, Neural Computation.

[3]  A. Lapedes,et al.  Nonlinear signal processing using neural networks: Prediction and system modelling , 1987 .

[4]  Ulrich Bodenhausen.  Learning internal representations of pattern sequences in a neural network with adaptive time-delays , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[5]  Yann LeCun,et al.  Improving the convergence of back-propagation learning with second-order methods , 1989 .

[6]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[7]  Anthony J. Robinson,et al.  Static and Dynamic Error Propagation Networks with Application to Speech Coding , 1987, NIPS.

[8]  M. Gori,et al.  BPS: a learning algorithm for capturing the dynamic nature of speech , 1989, International 1989 Joint Conference on Neural Networks.

[9]  Michael R. Davenport,et al.  Continuous-time temporal back-propagation with adaptable time delays , 1993, IEEE Trans. Neural Networks.

[10]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[11]  James P. Crutchfield,et al.  Equations of Motion from a Data Series , 1987, Complex Syst..

[12]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[14]  Terrence J. Sejnowski,et al.  Faster Learning for Dynamic Recurrent Backpropagation , 1990, Neural Computation.

[15]  David S. Touretzky,et al.  Advances in neural information processing systems 2 , 1989 .

[16]  Marco Saerens,et al.  Hopfield net generation, encoding and classification of temporal trajectories , 1994, IEEE Trans. Neural Networks.

[17]  Luís B. Almeida,et al.  A learning rule for asynchronous perceptrons with feedback in a combinatorial environment , 1990 .

[18]  Terrence J. Sejnowski,et al.  Neural Network Analysis of Distributed Representations of Dynamical Sensory-Motor Transformations in the Leech , 1989, NIPS 1989.

[19]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[20]  Geoffrey E. Hinton.  Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space , 1989, Neural Computation.

[21]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[22]  José Carlos Príncipe,et al.  A Theory for Neural Networks with Time Delays , 1990, NIPS.

[23]  Robert A. Jacobs,et al.  Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[24]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[25]  Barak A. Pearlmutter.  Dynamic recurrent neural networks , 1990 .

[26]  S. Lockery,et al.  Distributed processing of sensory information in the leech. II. Identification of interneurons contributing to the local bending reflex , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[28]  Thomas L. Griffiths,et al.  Advances in Neural Information Processing Systems 21 , 1993, NIPS 2009.

[29]  W. Kristan,et al.  Distributed processing of sensory information in the leech. I. Input- output relations of the local bending reflex , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[30]  A. E. Bryson,et al.  A Steepest-Ascent Method for Solving Optimum Programming Problems , 1962 .

[31]  Raymond L. Watrous,et al.  Complete gradient optimization of a recurrent network applied to /b/,/d/,/g/ discrimination , 1988 .

[32]  Geoffrey E. Hinton,et al.  Adaptive Soft Weight Tying using Gaussian Mixtures , 1991, NIPS.

[33]  David Zipser,et al.  Subgrouping Reduces Complexity and Speeds Up Learning in Recurrent Networks , 1989, NIPS.

[34]  Pierre Baldi,et al.  Contrastive Learning and Neural Oscillations , 1991, Neural Computation.

[35]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[36]  David Zipser,et al.  A Subgrouping Strategy that Reduces Complexity and Speeds Up Learning in Recurrent Networks , 1989, Neural Computation.

[37]  M. Gherrity,et al.  A learning algorithm for analog, fully recurrent neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[38]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.