Long Short-Term Memory Learns Context Free and Context Sensitive Languages

Previous work on learning regular languages from exemplary training sequences showed that Long Short-Term Memory (\mbox{LSTM}) outperforms traditional recurrent neural networks (\mbox{RNNs}). Here we demonstrate \mbox{LSTM''s} superior performance on context free language \mbox{(CFL)} benchmarks, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a context {\em sensitive} language (\mbox{CSL}), namely $a^nb^n c^n$.