Hybrid Behavior Co-evolution and Structure Learning in Behavior-based Systems

Designing an intelligent situated agent is a difficult task as the designer must see the problem from the agent's standpoint considering all its sensors and actuators. We have devised a co-evolutionary/reinforcement learning hybrid method to automate the design of hierarchical behavior-based systems. In our approach, the design problem is decomposed into two separate parts: developing a repertoire of behaviors and organizing those behaviors in a structure. Mathematical formulation shows how to decompose the value of the structure to simpler components. These components can be estimated and used to find the optimal organization of behaviors during the agent's lifetime. Moreover, a novel co-evolutionary mechanism is suggested that evolves each type of behavior separately in their own genetic pool. Our method is applied to the decentralized multi-robot object lifting task which results in human-competitive performance.

[1]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[2]  Rodney A. Brooks,et al.  A Robot that Walks; Emergent Behaviors from a Carefully Evolved Network , 1989, Neural Computation.

[3]  Rodney A. Brooks,et al.  A robot that walks; emergent behaviors from a carefully evolved network , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[4]  Rodney A. Brooks,et al.  Learning to Coordinate Behaviors , 1990, AAAI.

[5]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[6]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[7]  John R. Koza,et al.  Evolution of a subsumption architecture that performs a wall following task for an autonomous mobile robot , 1994, COLT 1994.

[8]  Eiji Nakano,et al.  Realizing cooperative object manipulation using multiple behaviour-based robots , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[9]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[10]  Lynne E. Parker,et al.  ALLIANCE: an architecture for fault tolerant multirobot cooperation , 1998, IEEE Trans. Robotics Autom..

[11]  M. Matarić.  Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior , 1998, Trends in Cognitive Sciences.

[12]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[13]  Maja J. Mataric,et al.  Learning in behavior-based multi-robot systems: policies, models, and other agents , 2001, Cognitive Systems Research.

[14]  Majid Nili Ahmadabadi,et al.  A "constrain and move" approach to distributed object manipulation , 2001, IEEE Trans. Robotics Autom..

[15]  Pablo Moscato,et al.  A Gentle Introduction to Memetic Algorithms , 2003, Handbook of Metaheuristics.

[16]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[17]  Julian Togelius,et al.  Evolution of a subsumption architecture neurocontroller , 2004, J. Intell. Fuzzy Syst..

[18]  Manuela M. Veloso,et al.  An evolutionary approach to gait learning for four-legged robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  Majid Nili Ahmadabadi,et al.  Behavior hierarchy learning in a behavior-based system using reinforcement learning , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).