Grammar-based connectionist approaches to language

This article describes an approach to connectionist language research which relies on the development of grammar formalisms rather than computer models. From formulations of the fundamental theoretical commitments of connectionism and of generative grammar, it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms — Harmonic Grammar (Legendre, Miyata and Smolensky, 1990ab) and Optimality Theory (Prince and Smolensky, 1991, 1993) — are briefly introduced to illustrate grammar-based approaches to connectionist language research. The strengths and weaknesses of grammar-based research and more traditional model-based research are argued to be complementary, suggesting a significant role for both strategies in the spectrum of connectionist language research. This article is addressed to basic methodological issues arising in connectionist research on language. I will attempt to briefly sketch a lengthy argument begun in Smolensky, Legendre, and Miyata (1992) and presented in detail in Smolensky and Legendre (in progress). In many places, I will try to articulate personal viewpoints and, to a very limited degree, justify them. The focus is on two main claims. The first is that there are two general styles of research that both deserve a central place in connectionist approaches to language. The first, model-based research, is well-established. The second, grammar-based research, is less so. Each approach, I will argue, has important strengths that are lacking in the other. The second main claim is that the time has come to stop regarding generative grammar and connectionist approaches to language as incompatible research paradigms. Each has significant potential for contributing to the other. I will suggest a view of the core theoretical commitments of the two paradigms, connectionism and generative linguistics, and argue that these commitments combine to support a coherent and fruitful research program in connectionist-grounded generative grammar. It is my belief, although I will not attempt to justify it in detail here, that the core commitments I identify are indeed consensus beliefs of the connectionist and generative linguistics research communities. Going beyond the core commitments, individual researchers have further commitments which are often not mutually compatible, and these competing scientific hypotheses must of course be adjudicated by theoretical and empirical arguments. But at this level, competition between incompatible hypotheses is readily found among generative grammarians themselves, or among connectionists themselves, as well as between generative linguists and connectionists. Thus it seems to me more accurate to regard the current scientific debates about language as individual conflicts between individual hypotheses, rather than a war between two unified paradigms, Connectionism and Generative Grammar. 1. Commitments of connectionism The Parallel Distributed Processing (PDP) school of connectionism is founded, it seems to me, on the following general principles (Rumelhart, McClelland, and the PDP Research Group, 1986):