Analogy-Related Information Can Be Accessed by Simple Addition and Subtraction of fMRI Activation Patterns, Without Participants Performing any Analogy Task

Abstract Analogical reasoning, for example, inferring that teacher is to chalk as mechanic is to wrench, plays a fundamental role in human cognition. However, whether brain activity patterns of individual words are encoded in a way that could facilitate analogical reasoning is unclear. Recent advances in computational linguistics have shown that information about analogical problems can be accessed by simple addition and subtraction of word embeddings (e.g., wrench = mechanic + chalk − teacher). Critically, this property emerges in artificial neural networks that were not trained to produce analogies but instead were trained to produce general-purpose semantic representations. Here, we test whether such emergent property can be observed in representations in human brains, as well as in artificial neural networks. fMRI activation patterns were recorded while participants viewed isolated words but did not perform analogical reasoning tasks. Analogy relations were constructed from word pairs that were categorically or thematically related, and we tested whether the predicted fMRI pattern calculated with simple arithmetic was more correlated with the pattern of the target word than other words. We observed that the predicted fMRI patterns contain information about not only the identity of the target word but also its category and theme (e.g., teaching-related). In summary, this study demonstrated that information about analogy questions can be reliably accessed with the addition and subtraction of fMRI patterns, and that, similar to word embeddings, this property holds for task-general patterns elicited when participants were not explicitly told to perform analogical reasoning.

[1]  Massimo Poesio,et al.  Discriminating Taxonomic Categories and Domains in Mental Simulations of Concepts of Varying Concreteness , 2014, Journal of Cognitive Neuroscience.

[2]  Dedre Gentner,et al.  Structure-Mapping: A Theoretical Framework for Analogy , 1983, Cogn. Sci..

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Tilo Kircher,et al.  Categorical and thematic knowledge representation in the brain: Neural correlates of taxonomic and thematic conceptual relations , 2008, Neuropsychologia.

[5]  Sara C. Sereno,et al.  The Glasgow Norms: Ratings of 5,500 words on nine scales , 2018, Behavior Research Methods.

[6]  Jeffrey N. Chiang,et al.  Distributed Code for Semantic Relations Predicts Neural Similarity during Analogical Reasoning , 2020, Journal of Cognitive Neuroscience.

[7]  Nancy Kanwisher,et al.  Toward a universal decoder of linguistic meaning from brain activation , 2018, Nature Communications.

[8]  A. Caramazza,et al.  Conceptual Object Representations in Human Anterior Temporal Cortex , 2012, The Journal of Neuroscience.

[9]  Edmund T. Rolls,et al.  Implementation of a new parcellation of the orbitofrontal cortex in the automated anatomical labeling atlas , 2015, NeuroImage.

[10]  Ryo Ishibashi,et al.  The neural network for tool-related cognition: An activation likelihood estimation meta-analysis of 70 neuroimaging contrasts , 2016, Cognitive neuropsychology.

[11]  Nikolaus Kriegeskorte,et al.  Representational Similarity Analysis – Connecting the Branches of Systems Neuroscience , 2008, Frontiers in systems neuroscience.

[12]  Wei Wu,et al.  Organizational Principles of Abstract Words in the Human Brain , 2018, Cerebral cortex.

[13]  Grant M. Walker,et al.  Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain , 2011, Proceedings of the National Academy of Sciences.

[14]  Robert Vargas,et al.  Neural Representations of Abstract Concepts: Identifying Underlying Neurosemantic Dimensions. , 2020, Cerebral cortex.

[15]  K. Holyoak,et al.  A System for Relational Reasoning in Human Prefrontal Cortex , 1999 .

[16]  Jiajun Zhang,et al.  Fine-grained neural decoding with distributed word representations , 2020, Inf. Sci..

[17]  Jia-Hong Gao,et al.  Doctor, Teacher, and Stethoscope: Neural Representation of Different Types of Semantic Relations , 2018, The Journal of Neuroscience.

[18]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[19]  Dawn Chen,et al.  Bayesian analogy with relational transformations. , 2012, Psychological review.

[20]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[21]  Jing Wang,et al.  Predicting the brain activation pattern associated with the propositional content of a sentence: Modeling neural representations of events and states , 2017, Human brain mapping.

[22]  M. Kronbichler,et al.  Words in Context: The Effects of Length, Frequency, and Predictability on Brain Responses During Natural Reading , 2016, Cerebral cortex.

[23]  Massimo Poesio,et al.  Reading visually embodied meaning from the brain: Visually grounded computational models decode visual-object mental imagery induced by written text , 2015, NeuroImage.

[24]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[25]  Friedemann Pulvermüller,et al.  Representational Similarity Mapping of Distributional Semantics in Left Inferior Frontal, Middle Temporal, and Motor Cortex , 2017, Cerebral cortex.

[26]  Jack L. Gallant,et al.  A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain , 2012, Neuron.

[27]  Tom Michael Mitchell,et al.  A Neurosemantic Theory of Concrete Noun Representation Based on the Underlying Brain Codes , 2010, PloS one.

[28]  Wanxiang Che,et al.  Learning Semantic Hierarchies via Word Embeddings , 2014, ACL.

[29]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[30]  David Poeppel,et al.  The neural bases of taxonomic and thematic conceptual relations: An MEG study , 2015, Neuropsychologia.

[31]  Jeffrey R. Binder,et al.  An Integrated Neural Decoder of Linguistic and Experiential Meaning , 2019, The Journal of Neuroscience.

[32]  Massimo Poesio,et al.  Visually Grounded and Textual Semantic Models Differentially Decode Brain Activity Associated with Concrete and Abstract Nouns , 2017, TACL.

[33]  Nancy Kanwisher,et al.  Decoding of generic mental representations from functional MRI data using word embeddings , 2016, bioRxiv.

[34]  R. Levy,et al.  General and specialized brain correlates for analogical reasoning: A meta‐analysis of functional imaging studies , 2016, Human brain mapping.

[35]  T. Rogers,et al.  The neural and computational bases of semantic cognition , 2016, Nature Reviews Neuroscience.

[36]  Eric P. Xing,et al.  Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2014, ACL 2014.

[37]  Monica Baciu,et al.  The sensory-motor specificity of taxonomic and thematic conceptual relations: A behavioral and fMRI study , 2009, NeuroImage.

[38]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[39]  Benjamin D. Zinszer,et al.  Representational similarity encoding for fMRI: Pattern-based synthesis to predict brain activity using stimulus-model-similarities , 2016, NeuroImage.

[40]  Jaimie F. Veale,et al.  Edinburgh Handedness Inventory – Short Form: A revised version based on confirmatory factor analysis , 2014, Laterality.

[41]  John-Dylan Haynes,et al.  Valid population inference for information-based imaging: From the second-level t-test to prevalence inference , 2015, NeuroImage.

[42]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.