Exploiting deep semantics and compositionality of natural language for Human-Robot-Interaction

We are developing a natural language interface for human robot interaction that implements reasoning about deep semantics in natural language. To realize the required deep analysis, we employ methods from cognitive linguistics, namely the modular and compositional framework of Embodied Construction Grammar (ECG) [18]. Using ECG, robots are able to solve fine-grained reference resolution problems and other issues related to deep semantics and compositionality of natural language. This also includes verbal interaction with humans to clarify commands and queries that are too ambiguous to be executed safely. We implement our NLU framework as a ROS package and present proof-of-concept scenarios with different robots, as well as a survey on the state of the art in knowledge-based language HRI.

[1]  Luke S. Zettlemoyer,et al.  Learning to Parse Natural Language Commands to a Robot Control System , 2012, ISER.

[2]  Stefanie Tellex,et al.  Clarifying commands with information-theoretic human-robot dialog , 2013, HRI 2013.

[3]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[4]  Matthias Scheutz,et al.  Going Beyond Literal Command-Based Instructions: Extending Robotic Natural Language Interaction Capabilities , 2015, AAAI.

[5]  Charles J. Fillmore,et al.  Frames and the semantics of understanding , 1985 .

[6]  Benjamin K. Bergen,et al.  Embodied Construction Grammar , 2013 .

[7]  John E. Laird,et al.  Toward Integrating Cognitive Linguistics and Cognitive Language Processing , 2016 .

[8]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[9]  Geoffrey K. Pullum,et al.  Natural languages and context-free languages , 1982 .

[10]  Luc Steels,et al.  The Origins of Ontologies and Communication Conventions in Multi-Agent Systems , 2004, Autonomous Agents and Multi-Agent Systems.

[11]  Johan Bos,et al.  A spoken language interface with a mobile robot , 2006, Artificial Life and Robotics.

[12]  Roberto Basili,et al.  Effective and Robust Natural Language Understanding for Human-Robot Interaction , 2014, ECAI.

[13]  Daniel Bonevac Discourse Representation Theory , 2012 .

[14]  G. Lakoff,et al.  Metaphors We Live by , 1982 .

[15]  Matthias Scheutz,et al.  Towards a Framework for Integrated Natural Language Processing Architectures for Social Robots , 2008, NLPCS.

[16]  Roberto Basili,et al.  Textual Inference and Meaning Representation in Human Robot Interaction , 2013, JSSP.

[17]  Daniele Nardi,et al.  Knowledgeable Talking Robots , 2013, AGI.

[18]  Gideon Borensztajn,et al.  Luc Steels , the Talking Heads Experiment and Cognitive Philosophy , 2006 .

[19]  Luc Steels,et al.  Fluid Construction Grammar on Real Robots , 2012, Language Grounding in Robots.

[20]  Henrik I. Christensen,et al.  Situated Dialogue and Spatial Organization: What, Where… and Why? , 2007 .

[21]  Roberto Basili,et al.  HuRIC: a Human Robot Interaction Corpus , 2014, LREC.

[22]  Miriam R. L. Petruck FRAME SEMANTICS , 1996 .

[23]  Luc Steels,et al.  The Talking Heads experiment , 2015 .

[24]  Ellen Dodge,et al.  A Neural Theory of Language and Embodied Construction Grammar , 2008 .

[25]  Raymond J. Mooney,et al.  Learning to Interpret Natural Language Navigation Instructions from Observations , 2011, Proceedings of the AAAI Conference on Artificial Intelligence.

[26]  Guido Boella,et al.  Normative framework for normative system change , 2009, AAMAS 2009.

[27]  Matthew R. Walter,et al.  Learning Semantic Maps from Natural Language Descriptions , 2013, Robotics: Science and Systems.

[28]  Mark Steedman,et al.  The syntactic process , 2004, Language, speech, and communication.

[29]  Stefanie Tellex,et al.  Toward Information Theoretic Human-Robot Dialog , 2012, Robotics: Science and Systems.

[30]  Evan A. Krause,et al.  Novel Mechanisms for Natural Human-Robot Interactions in the DIARC Architecture , 2013 .

[31]  Matthias Scheutz,et al.  Toward Humanlike Task-Based Dialogue Processing for Human Robot Interaction , 2011, AI Mag..

[32]  Katrien Beuls,et al.  Diagnostics and Repairs in Fluid Construction Grammar , 2012, Language Grounding in Robots.

[33]  Huda Khayrallah,et al.  Natural Language For Human Robot Interaction , 2015 .

[34]  Jerome A. Feldman,et al.  Natural Language Understanding and Communication for Multi-Agent Systems , 2015, AAAI Fall Symposia.

[35]  Laurence R. Horn,et al.  The handbook of pragmatics , 2004 .

[36]  Steven Kumar Sinha,et al.  Answering Questions about Complex Events , 2008 .

[37]  G. Lakoff Philosophy in the flesh , 1999 .

[38]  Franziska Frankfurter,et al.  Constructions: A construction grammar approach to argument structure: Adele E. Goldberg, Chicago, IL: The University of Chicago Press, 1995. xi + 265 pp , 1998 .

[39]  Jerome A. Feldman,et al.  Best-fit constructional analysis , 2008 .

[40]  Ellen Dodge,et al.  MetaNet: Deep semantic automatic metaphor analysis , 2015 .

[41]  Mehul Bhatt,et al.  Robust Natural Language Processing - Combining Reasoning, Cognitive Semantics, and Construction Grammar for Spatial Language , 2016, IJCAI.

[42]  Luca Iocchi,et al.  RoboCup@Home: Scientific Competition and Benchmarking for Domestic Service Robots , 2009 .

[43]  Luc Steels,et al.  Co-Acquisition of Syntax and Semantics - An Investigation in Spatial Language , 2015, IJCAI.

[44]  Mehul Bhatt,et al.  Approximate Epistemic Planning with Postdiction as Answer-Set Programming , 2013, LPNMR.

[45]  Changsong Liu,et al.  Collaborative Effort towards Common Ground in Situated Human-Robot Dialogue , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[46]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[47]  Matthias Scheutz,et al.  Robust spoken instruction understanding for HRI , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[48]  Jeffrey Mark Siskind,et al.  Robot Language Learning, Generation, and Comprehension , 2015, ArXiv.

[49]  Mehul Bhatt,et al.  A history based approximate epistemic action theory for efficient postdictive reasoning , 2015, J. Appl. Log..

[50]  Pierre Lison,et al.  Situated Dialogue Processing for Human-Robot Interaction , 2010, Cognitive Systems.