Deep Learning of Grammatically-Interpretable Representations Through Question-Answering

We introduce an architecture in which internal representations— learned by end-to-end optimization in a deep neural network performing a textual question-answering task—can be interpreted using basic concepts from linguistic theory. This interpretability comes at a cost of only a few percentage-point reduction in accuracy relative to the original model on which the new one is based (BiDAF [1]). The internal representation that is interpreted is a Tensor Product Representation: for each input word, the model selects a symbol to encode the word, and a role in which to place the symbol, and binds the two together. The selection is via soft attention. The overall interpretation is built from interpretations of the symbols, as recruited by the trained model, and interpretations of the roles as used by the model. We find support for our initial hypothesis that symbols can be interpreted as lexical-semantic word meanings, while roles can be interpreted as approximations of grammatical roles (or categories) such as subject, wh-word, determiner, etc. Through extremely detailed, fine-grained analysis, we find specific correspondences between the learned roles and parts of speech as assigned by a standard parser [2], and find several discrepancies in the model’s favor. In this sense, the model learns significant aspects of grammar, after having been exposed solely to linguistically unannotated text, questions, and answers: no prior linguistic knowledge is given to the model. What is given is the means ∗This work was carried out while PS was on leave from Johns Hopkins University. LD is currently at Citadel. 1 ar X iv :1 70 5. 08 43 2v 1 [ cs .C L ] 2 3 M ay 2 01 7 to represent using symbols and roles and an inductive bias favoring use of these in an approximately discrete manner.