The neural coding of face and body orientation in occipitotemporal cortex

Face and body orientation convey important information for us to understand other people's actions, intentions and social interactions. It has been shown that several occipitotemporal areas respond differently to faces or bodies of different orientations. However, whether face and body orientation are processed by partially overlapping or completely separate brain networks remains unclear, as the neural coding of face and body orientation is often investigated separately. Here, we recorded participants' brain activity using fMRI while they viewed faces and bodies shown from three different orientations, while attending to either orientation or identity information. Using multivoxel pattern analysis we investigated which brain regions process face and body orientation respectively, and which regions encode both face and body orientation in a stimulus-independent manner. We found that patterns of neural responses evoked by different stimulus orientations in the occipital face area, extrastriate body area, lateral occipital complex and right early visual cortex could generalise across faces and bodies, suggesting a stimulus-independent encoding of person orientation in occipitotemporal cortex. This finding was consistent across functionally defined regions of interest and a whole-brain searchlight approach. The fusiform face area responded to face but not body orientation, suggesting that orientation responses in this area are face-specific. Moreover, neural responses to orientation were remarkably consistent regardless of whether participants attended to the orientation of faces and bodies or not. Together, these results demonstrate that face and body orientation are processed in a partially overlapping brain network, with a stimulus-independent neural code for face and body orientation in occipitotemporal cortex.

[1]  Joris Vangeneugden,et al.  Distinct Neural Mechanisms for Body Form and Body Motion Discriminations , 2014, The Journal of Neuroscience.

[2]  Michael J. Black,et al.  Separated and overlapping neural coding of face and body identity , 2021, bioRxiv.

[3]  Alison J. Wiggett,et al.  Behavioral / Systems / Cognitive Functional Magnetic Resonance Imaging Investigation of Overlapping Lateral Occipitotemporal Activations Using Multi-Voxel Pattern Analysis , 2006 .

[4]  Katrin Amunts,et al.  Julich-Brain: A 3D probabilistic atlas of the human brain’s cytoarchitecture , 2020, Science.

[5]  Justin L. Gardner,et al.  Task-dependent enhancement of facial expression and identity representations in human cortex , 2018, NeuroImage.

[6]  Johan D. Carlin,et al.  A Head View-Invariant Representation of Gaze Direction in Anterior Superior Temporal Sulcus , 2011, Current Biology.

[7]  J. Brooks Counterbalancing for serial order carryover effects in experimental condition orders. , 2012, Psychological methods.

[8]  Michael J. Black,et al.  Learning a model of facial shape and expression from 4D scans , 2017, ACM Trans. Graph..

[9]  A. J. Mistlin,et al.  Visual cells in the temporal cortex sensitive to face view and gaze direction , 1985, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[10]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2015, ACM Trans. Graph..

[11]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[12]  Celia Foster The neural coding of properties shared by faces, bodies and objects , 2020 .

[13]  Irving Biederman,et al.  Predicting the psychophysical similarity of faces and non-face complex shapes by image-based measures , 2012, Vision Research.

[14]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[15]  Martin N. Hebart,et al.  The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data , 2015, Front. Neuroinform..

[16]  J. S. Guntupalli,et al.  Disentangling the Representation of Identity from Head View Along the Human Face Processing Pathway , 2016, bioRxiv.

[17]  J. Gibson,et al.  Perception of another person's looking behavior. , 1963, The American journal of psychology.

[18]  L. Shah,et al.  Functional magnetic resonance imaging. , 2010, Seminars in roentgenology.

[19]  Andreas Bartels,et al.  Invariance of surface color representations across illuminant changes in the human cortex , 2017, NeuroImage.

[20]  Kalanit Grill-Spector,et al.  The representation of object viewpoint in human visual cortex , 2009, NeuroImage.

[21]  Bart Krekelberg,et al.  The Statistical Analysis of Multi-Voxel Patterns in Functional Imaging , 2013, PloS one.

[22]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[23]  Jonathan R. McDaniel,et al.  Face Adaptation without a Face , 2010, Current Biology.

[24]  H. Bülthoff,et al.  What the Human Brain Likes About Facial Motion , 2012, Cerebral cortex.

[25]  Frank Tong,et al.  Prevalence of Selectivity for Mirror-Symmetric Views of Faces in the Ventral and Dorsal Visual Pathways , 2012, The Journal of Neuroscience.

[26]  Steven L Franconeri,et al.  The Head of the Table: Marking the “Front” of An Object Is Tightly Linked with Selection , 2012, The Journal of Neuroscience.

[27]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[28]  Peter W Dicke,et al.  Following Eye Gaze Activates a Patch in the Posterior Temporal Cortex That Is not Part of the Human “Face Patch” System , 2017, eNeuro.

[29]  Katie L. H. Gray,et al.  Why are social interactions found quickly in visual search tasks? , 2020, Cognition.

[30]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[31]  Rainer Goebel,et al.  A Probabilistic Functional Atlas of Human Occipito-Temporal Visual Cortex , 2020, bioRxiv.

[32]  Andreas Bartels,et al.  Face processing regions are sensitive to distinct aspects of temporal sequence in facial dynamics , 2014, NeuroImage.

[33]  Luca Passamonti,et al.  Changes in “Top-Down” Connectivity Underlie Repetition Suppression in the Ventral Visual Pathway , 2011, The Journal of Neuroscience.

[34]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[35]  Rebecca F. Schwarzlose,et al.  Separate face and body selectivity on the fusiform gyrus. , 2010, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[36]  Michael J. Black,et al.  Decoding subcategories of human bodies from both body- and face-responsive cortical regions , 2019, NeuroImage.

[37]  P. Downing,et al.  The effect of viewpoint on body representation in the extrastriate body area , 2004, Neuroreport.

[38]  Timothy J. Andrews,et al.  fMR-adaptation reveals a distributed representation of inanimate objects and places in human visual cortex , 2005, NeuroImage.

[39]  Kami Koldewyn,et al.  Dyadic interaction processing in the posterior temporal cortex , 2019, NeuroImage.

[40]  G. Yovel,et al.  Hierarchical Processing of Face Viewpoint in Human Visual Cortex , 2012, The Journal of Neuroscience.

[41]  Ethan M. Meyers,et al.  Contextually Evoked Object-Specific Responses in Human Visual Cortex , 2004, Science.

[42]  R. Tootell,et al.  A problem of overlap , 2015, Visual Neuroscience.

[43]  Andreas Bartels,et al.  Invariance of surface color representations across illuminant changes in the human cortex , 2015, NeuroImage.

[44]  Radoslaw Martin Cichy,et al.  Imagery and perception share cortical representations of content and location. , 2012, Cerebral Cortex.

[45]  Liuba Papeo,et al.  The Representation of Two-Body Shapes in the Human Visual Cortex , 2019, The Journal of Neuroscience.

[46]  Radoslaw Martin Cichy,et al.  The Neural Code for Face Orientation in the Human Fusiform Face Area , 2014, The Journal of Neuroscience.

[47]  Fernando M Ramírez,et al.  Orientation Encoding and Viewpoint Invariance in Face Recognition: Inferring Neural Properties from Large-Scale Signals , 2018, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[48]  A. Caramazza,et al.  Decoding representations of face identity that are tolerant to rotation. , 2014, Cerebral cortex.

[49]  Alice J. O'Toole,et al.  Dissociable Neural Patterns of Facial Identity across Changes in Viewpoint , 2010, Journal of Cognitive Neuroscience.

[50]  Johan D. Carlin,et al.  The neural basis of eye gaze processing , 2013, Current Opinion in Neurobiology.

[51]  Rufin Vogels,et al.  Transformation of Visual Representations Across Ventral Stream Body-selective Patches , 2019, Cerebral cortex.

[52]  William Hyde Wollaston,et al.  XIII. On the apparent direction of eyes in a portrait , 1824, Philosophical Transactions of the Royal Society of London.

[53]  S. Edelman,et al.  Differential Processing of Objects under Various Viewing Conditions in the Human Lateral Occipital Complex , 1999, Neuron.

[54]  Rebecca F. Schwarzlose,et al.  Separate Face and Body Selectivity on the Fusiform Gyrus , 2005, The Journal of Neuroscience.

[55]  P. Downing,et al.  The neural basis of visual body perception , 2007, Nature Reviews Neuroscience.

[56]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[57]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[58]  Mona Rosenke,et al.  A Probabilistic Functional Atlas of Human Occipito-Temporal Visual Cortex , 2020, Cerebral cortex.

[59]  N. Kanwisher,et al.  A Cortical Area Selective for Visual Processing of the Human Body , 2001, Science.

[60]  Alison J. Wiggett,et al.  fMRI-adaptation studies of viewpoint tuning in the extrastriate and fusiform body areas. , 2010, Journal of neurophysiology.

[61]  Filip Germeys,et al.  Perceiving where another person is looking: the integration of head and body information in estimating another person’s gaze , 2015, Front. Psychol..

[62]  N. Kanwisher,et al.  Location and spatial profile of category‐specific regions in human extrastriate cortex , 2006, Human brain mapping.

[63]  A. Calder,et al.  Neural mechanisms of social attention , 2009, Trends in Cognitive Sciences.

[64]  A. Bartels,et al.  Decoding the Yellow of a Gray Banana , 2013, Current Biology.

[65]  Peter Janssen,et al.  Effective Connectivity Reveals Largely Independent Parallel Networks of Face and Body Patches , 2016, Current Biology.

[66]  Daniel D. Dilks,et al.  Differential selectivity for dynamic versus static information in face-selective cortical regions , 2011, NeuroImage.

[67]  D. Perrett,et al.  Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque. , 1994, Cerebral cortex.

[68]  Richard J. Harris,et al.  Symmetrical Viewpoint Representations in Face-Selective Regions Convey an Advantage in the Perception and Recognition of Faces , 2019, The Journal of Neuroscience.

[69]  Doris Y. Tsao,et al.  Single-Unit Recordings in the Macaque Face Patch System Reveal Limitations of fMRI MVPA , 2015, The Journal of Neuroscience.