Dreaming with ARC

Current machine learning algorithms are highly specialized to whatever it is they are meant to do –– e.g. playing chess, picking up objects, or object recognition. How can we extend this to a system that could solve a wide range of problems? We argue that this can be achieved by a modular system –– one that can adapt to solving different problems by changing only the modules chosen and the order in which those modules are applied to the problem. The recently introduced ARC (Abstraction and Reasoning Corpus) dataset serves as an excellent test of abstract reasoning. Suited to the modular approach, the tasks depend on a set of human Core Knowledge inbuilt priors. In this paper we implement these priors as the modules of our system. We combine these modules using a neural-guided program synthesis.

[1]  Xavier Boix,et al.  Do Neural Networks for Segmentation Understand Insideness? , 2020, Neural Computation.

[2]  Armando Solar-Lezama,et al.  DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning , 2020, Philosophical Transactions of the Royal Society A.

[3]  Guillaume Lample,et al.  Deep Learning for Symbolic Mathematics , 2019, ICLR.

[4]  Roger B. Grosse,et al.  Neural Theorem Proving on Inequality Problems , 2020 .

[5]  Alberto Camacho,et al.  Towards Neural-Guided Program Synthesis for Linear Temporal Logic Specifications , 2019, ArXiv.

[6]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[7]  Jianfeng Gao,et al.  Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving , 2019, ArXiv.

[8]  Pushmeet Kohli,et al.  Analysing Mathematical Reasoning Abilities of Neural Models , 2019, ICLR.

[9]  Taehoon Kim,et al.  Quantifying Generalization in Reinforcement Learning , 2018, ICML.

[10]  Dawn Song,et al.  Execution-Guided Neural Program Synthesis , 2018, ICLR.

[11]  Yoshua Bengio,et al.  Modularity Matters: Learning Invariant Relational Reasoning Tasks , 2018, ArXiv.

[12]  Cezary Kaliszyk,et al.  Reinforcement Learning of Theorem Proving , 2018, NeurIPS.

[13]  Sumit Gulwani,et al.  Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples , 2018, ICLR.

[14]  Christopher D. Manning,et al.  Compositional Attention Networks for Machine Reasoning , 2018, ICLR.

[15]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[16]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[17]  Dawn Xiaodong Song,et al.  Making Neural Programming Architectures Generalize via Recursion , 2017, ICLR.

[18]  Ohad Shamir,et al.  Failures of Gradient-Based Deep Learning , 2017, ICML.

[19]  Cezary Kaliszyk,et al.  Deep Network Guided Proof Search , 2017, LPAR.

[20]  Lihong Li,et al.  Neuro-Symbolic Program Synthesis , 2016, ICLR.

[21]  Dan Klein,et al.  Learning to Compose Neural Networks for Question Answering , 2016, NAACL.

[22]  Lukasz Kaiser,et al.  Neural GPUs Learn Algorithms , 2015, ICLR.

[23]  Dan Klein,et al.  Deep Compositional Question Answering with Neural Module Networks , 2015, ArXiv.

[24]  Tomas Mikolov,et al.  Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets , 2015, NIPS.

[25]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[26]  N. McGlynn.  Thinking fast and slow. , 2014, Australian veterinary journal.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  A. Tate.  A measure of intelligence , 2012 .

[29]  Jonathan Evans.  In two minds: dual-process accounts of reasoning , 2003, Trends in Cognitive Sciences.

[30]  S. Sloman.  The empirical case for two systems of reasoning. , 1996 .