We proposed a model of Time Warping Invariant Neural Networks (TWINN) to handle the time warped continuous signals. Although TWINN is a simple modification of well known recurrent neural network, analysis has shown that TWINN completely removes time warping and is able to handle difficult classification problem. It is also shown that TWINN has certain advantages over the current available sequential processing schemes: Dynamic Programming(DP)[1], Hidden Markov Model(HMM)[2], Time Delayed Neural Networks(TDNN) [3] and Neural Network Finite Automata(NNFA)[4].
We also analyzed the time continuity employed in TWINN and pointed out that this kind of structure can memorize longer input history compared with Neural Network Finite Automata (NNFA). This may help to understand the well accepted fact that for learning grammatical reference with NNFA one had to start with very short strings in training set.
The numerical example we used is a trajectory classification problem. This problem, making a feature of variable sampling rates, having internal states, continuous dynamics, heavily time-warped data and deformed phase space trajectories, is shown to be difficult to other schemes. With TWINN this problem has been learned in 100 iterations. For benchmark we also trained the exact same problem with TDNN and completely failed as expected.
[1]
S. Chiba,et al.
Dynamic programming algorithm optimization for spoken word recognition
,
1978
.
[2]
Geoffrey E. Hinton,et al.
Learning internal representations by error propagation
,
1986
.
[3]
C. Lee Giles,et al.
Higher Order Recurrent Networks and Grammatical Inference
,
1989,
NIPS.
[4]
Geoffrey E. Hinton,et al.
Phoneme recognition using time-delay neural networks
,
1989,
IEEE Trans. Acoust. Speech Signal Process..
[5]
Ronald J. Williams,et al.
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
,
1989,
Neural Computation.