A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27

How could machines learn as efficiently as humans and animals? How could machines learn to reason and plan? How could machines learn representations of percepts and action plans at multiple levels of abstraction, enabling them to reason, predict, and plan at multiple time horizons? This position paper proposes an architecture and training paradigms with which to construct autonomous intelligent agents. It combines concepts such as configurable predictive world model, behavior driven through intrinsic motivation, and hierarchical joint embedding architectures trained with self-supervised learning.

[1]  Ian S. Fischer,et al.  Deep Hierarchical Planning from Pixels , 2022, NeurIPS.

[2]  Tarek R. Besold,et al.  Lessons from infant learning for unsupervised machine learning , 2022, Nature Machine Intelligence.

[3]  Jakob Drachmann Havtorn,et al.  Self-Supervised Speech Representation Learning: A Review , 2022, ArXiv.

[4]  Mark K. Ho,et al.  People construct simplified mental representations to plan. , 2022, Nature.

[5]  Kyunghyun Cho,et al.  Separating the World and Ego Models for Self-Driving , 2022, ArXiv.

[6]  M. Lengyel,et al.  Planning in the brain , 2022, Neuron.

[7]  Lerrel Pinto,et al.  The Surprising Effectiveness of Representation Learning for Visual Imitation , 2021, Robotics: Science and Systems.

[8]  Kwan Ho Ryan Chan,et al.  CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction , 2021, Entropy.

[9]  Yann LeCun,et al.  VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning , 2021, ICLR.

[10]  Yann LeCun,et al.  Sparse Coding with Multi-Layer Decoders using Variance Regularization , 2021, Trans. Mach. Learn. Res..

[11]  Sergey Levine,et al.  Understanding the World Through Action , 2021, CoRL.

[12]  Andreas Krause,et al.  Hierarchical Skills for Efficient Exploration , 2021, NeurIPS.

[13]  Doina Precup,et al.  Reward is enough , 2021, Artif. Intell..

[14]  Sergey Levine,et al.  Offline Reinforcement Learning as One Big Sequence Modeling Problem , 2021, NeurIPS.

[15]  Yann LeCun,et al.  Barlow Twins: Self-Supervised Learning via Redundancy Reduction , 2021, ICML.

[16]  Aäron van den Oord,et al.  Predicting Video with VQVAE , 2021, ArXiv.

[17]  Mohammad Norouzi,et al.  Mastering Atari with Discrete World Models , 2020, ICLR.

[18]  Nicu Sebe,et al.  Whitening for Self-Supervised Representation Learning , 2020, ICML.

[19]  R. Fergus,et al.  Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels , 2020, ICLR.

[20]  Yann LeCun,et al.  Implicit Rank-Minimizing Autoencoder , 2020, NeurIPS.

[21]  B. Lake,et al.  Self-supervised learning through the eyes of a child , 2020, NeurIPS.

[22]  Abdel-rahman Mohamed,et al.  wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations , 2020, NeurIPS.

[23]  Geoffrey E. Hinton,et al.  Big Self-Supervised Models are Strong Semi-Supervised Learners , 2020, NeurIPS.

[24]  Julien Mairal,et al.  Unsupervised Learning of Visual Features by Contrasting Cluster Assignments , 2020, NeurIPS.

[25]  Pierre H. Richemond,et al.  Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning , 2020, NeurIPS.

[26]  Lantao Yu,et al.  MOPO: Model-based Offline Policy Optimization , 2020, NeurIPS.

[27]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[28]  Diego de Las Casas,et al.  Transformation-based Adversarial Video Prediction on Large-Scale Data , 2020, ArXiv.

[29]  Kaiming He,et al.  Improved Baselines with Momentum Contrastive Learning , 2020, ArXiv.

[30]  Laurens van der Maaten,et al.  Self-Supervised Learning of Pretext-Invariant Representations , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Ross B. Girshick,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Jean Pierre Mercat,et al.  Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Ali Razavi,et al.  Data-Efficient Image Recognition with Contrastive Predictive Coding , 2019, ICML.

[34]  S. Kolassa.  Two Cheers for Rebooting AI: Building Artificial Intelligence We Can Trust , 2020 .

[35]  Yann LeCun,et al.  Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic , 2019, ICLR.

[36]  Ruben Villegas,et al.  Learning Latent Dynamics for Planning from Pixels , 2018, ICML.

[37]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[38]  Jürgen Schmidhuber,et al.  Recurrent World Models Facilitate Policy Evolution , 2018, NeurIPS.

[39]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[40]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[41]  Allan Jabri,et al.  Universal Planning Networks , 2018, ICML.

[42]  Yann LeCun,et al.  Predicting Future Instance Segmentations by Forecasting Convolutional Features , 2018, ECCV.

[43]  Rob Fergus,et al.  Stochastic Video Generation with a Learned Prior , 2018, ICML.

[44]  Sergey Levine,et al.  Stochastic Variational Video Prediction , 2017, ICLR.

[45]  Sergey Levine,et al.  Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[46]  Wenlong Fu,et al.  Model-based reinforcement learning: A survey , 2018 .

[47]  D. Bertsekas.  Reinforcement Learning and Optimal ControlA Selective Overview , 2018 .

[48]  Oriol Vinyals,et al.  Neural Discrete Representation Learning , 2017, NIPS.

[49]  S. Dehaene,et al.  What is consciousness, and could machines have it? , 2017, Science.

[50]  Adam Wierman,et al.  Thinking Fast and Slow , 2017, SIGMETRICS Perform. Evaluation Rev..

[51]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[52]  Yann LeCun,et al.  Predicting Deeper into the Future of Semantic Segmentation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[53]  Jason Weston,et al.  Tracking the World State with Recurrent Entity Networks , 2016, ICLR.

[54]  Sergey Levine,et al.  Deep visual foresight for planning robot motion , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[55]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[56]  Jitendra Malik,et al.  Learning to Poke by Poking: Experiential Learning of Intuitive Physics , 2016, NIPS.

[57]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[58]  Sergey Levine,et al.  Unsupervised Learning for Physical Interaction through Video Prediction , 2016, NIPS.

[59]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[60]  Rob Fergus,et al.  Learning Physical Intuition of Block Towers by Example , 2016, ICML.

[61]  Jitendra Malik,et al.  Learning Visual Predictive Models of Physics for Playing Billiards , 2015, ICLR.

[62]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[63]  Honglak Lee,et al.  Action-Conditional Video Prediction using Deep Networks in Atari Games , 2015, NIPS.

[64]  Yann LeCun,et al.  Learning to Linearize Under Uncertainty , 2015, NIPS.

[65]  Jason Weston,et al.  Large-scale Simple Question Answering with Memory Networks , 2015, ArXiv.

[66]  Jonathan Tompson,et al.  Unsupervised Feature Learning from Temporal Data , 2015, ICLR.

[67]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[68]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[69]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[70]  L. F. Abbott,et al.  Hierarchical Control Using Networks Trained with Higher-Level Forward Models , 2014, Neural Computation.

[71]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[72]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[73]  Pierre-Yves Oudeyer,et al.  Information-seeking, curiosity, and attention: computational and neural mechanisms , 2013, Trends in Cognitive Sciences.

[74]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[75]  Yann LeCun,et al.  Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields , 2010, ArXiv.

[76]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[77]  Michael S. Lewicki,et al.  Robust Coding Over Noisy Overcomplete Channels , 2007, IEEE Transactions on Image Processing.

[78]  Katherine D. Kinzler,et al.  Core knowledge. , 2007, Developmental science.

[79]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[80]  Fu Jie Huang,et al.  A Tutorial on Energy-Based Learning , 2006 .

[81]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[82]  Miguel Á. Carreira-Perpiñán,et al.  On Contrastive Divergence Learning , 2005, AISTATS.

[83]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[84]  G. Murphy,et al.  The Big Book of Concepts , 2002 .

[85]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[86]  A. Gopnik,et al.  The Scientist in the Crib: What Early Learning Tells Us About the Mind , 2000 .

[87]  S. Carey.  The Origin of Concepts , 2000 .

[88]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[89]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[90]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[91]  A. Gopnik,et al.  Words, thoughts, and theories , 1997 .

[92]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[93]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[94]  Eduardo D. Sontag,et al.  Neural Networks for Control , 1993 .

[95]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[96]  Geoffrey E. Hinton,et al.  Self-organizing neural network that discovers surfaces in random-dot stereograms , 1992, Nature.

[97]  Richard S. Sutton,et al.  Dyna, an integrated architecture for learning, planning, and reacting , 1990, SGAR.

[98]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[99]  J. Richalet,et al.  Model predictive heuristic control: Applications to industrial processes , 1978, Autom..

[100]  J. Meditch,et al.  Applied optimal control , 1972, IEEE Transactions on Automatic Control.

[101]  W. H. F. Barnes.  The Nature of Explanation , 1944, Nature.