Shared neural codes for visual and semantic information about familiar faces in a common

0


Face recognition is essential for effective social interactions. When we see a familiar face, we spontaneously retrieve person knowledge and the position occupied by that familiar individual in our social network. This information sets us up for the most appropriate behavior with that specific individual. The importance of familiar faces for social interactions is reflected in the way the human brain processes these stimuli. Familiar faces are processed in a prioritized way with faster detection even in suboptimal conditions (1, 2). Familiarity associated with faces warps their visual representation (3) and can result in a more homogenous representation across the visual field in areas with retinotopic organization (4). Recognition of familiar faces entails processing not only their visual appearance but also retrieval of person knowledge and an emotional response (510). Different parts of the distributed neural system for face perception contribute to these processes (5, 10, 11). The core system for face perception processes visual appearance, resulting in view-invariant representations of identity in anterior temporal and inferior frontal face areas (1214). The extended system for face perception plays a role in extracting semantic information from faces as well as emotional responses (5, 11, 15, 16).

Here, we investigated the neural codes for high-level visual and semantic information about personally familiar faces. Specifically, we asked whether these codes are supported by a common set of basis functions that are shared across people who are personally familiar with the same individuals. We measured patterns of brain activity with functional magnetic resonance imaging (fMRI) while participants viewed images of personally familiar faces and faces of strangers who were only visually familiar. We used hyperalignment to derive a set of basis functions that align brain response patterns in a common, high-dimensional information space (1719). Hyperalignment transformation parameters were based on participants’ brain activity measured while watching The Grand Budapest Hotel (20), an engaging comedy-drama with rich characterizations of several individuals. We found that these basis functions capture shared representations of visual appearance in the core system for both personally familiar faces and visually familiar faces of strangers. Surprisingly, we also found basis functions that capture shared representations of personally familiar others, but not visually familiar strangers, in extended system areas that are associated with representation of person knowledge, theory of mind, and emotion. Importantly, these basis functions are derived from brain responses to the movie and are, thus, not specific to the familiar individuals whose faces were the experimental stimuli. These results show that the face processing system encodes both visual and nonvisual high-level semantic information about personally familiar others in a neural information space that is not specific to a given set of faces and that is shared across brains.

Results

We measured patterns of brain responses to images of the faces of four personally familiar individuals and four individuals who were only visually familiar (each presented in five different head views; Fig. 1A) in a group of 14 participants who had known each other for over 2 y. Participants were graduate students, and the personally familiar faces were four other students in the same PhD program. Personally familiar individuals were rated as highly familiar by the participants (see SI Appendix, Fig. S1 and Supplementary Information for quantitative metrics of familiarity). The four individuals who were only visually familiar were previously unknown. Participants became visually familiar with these faces during an extensive behavioral training session a day prior the fMRI experiment (average face recognition accuracy during the training session was 97.9% [95% bootstrapped CI: 96.9, 98.6]; see Materials and Methods for more details and SI Appendix, Fig. S1).

We used fMRI data collected while participants watched a movie (The Grand Budapest Hotel) to derive a common model of information spaces with hyperalignment (17). For each participant, hyperalignment calculates transformations that remix that individual’s cortical vertices into the model space dimensions. These dimensions capture response basis functions that are shared across brains, affording markedly stronger between-subject decoding of brain response patterns (Fig. 2 and SI Appendix, Fig. S2). Because these transformations are performed on cortical vertices, they can be applied to new responses in the same participant to model responses for new stimuli that were not used in their derivation. These transformations derived from independent movie-viewing data were used to project brain responses to the face images in the familiar face perception experiment into the shared model information space.

Fig. 2.
Fig. 2.

Decoding identity across participants with surface-searchlight, between-subject MVPC after hyperalignment. (A) Brain map showing decoding accuracy for personally familiar faces. (B) Brain map showing decoding accuracy for visually familiar faces. Both maps are thresholded at P < 0.05, one-sided, after permutation testing and FDR correction (Benjamini-Hochberg). “NS” on the colorbar indicates the range of nonsignificant accuracy values greater than 25%. (C) Difference in decoding accuracy between personally familiar faces and visually familiar faces. Red vertices indicate higher decoding accuracy for personally familiar faces (Personal > Visual). Blue vertices indicate higher decoding accuracy for visually familiar faces (Visual > Personal). The map is thresholded at P < 0.05, two-sided, after permutation testing and FDR correction (Benjamini-Hochberg). For personally familiar faces, significant between-subject decoding accuracy was present across the core system: bilateral OFA and fusiform gyrus, right ATL (ventral core system); right pSTS, mSTS, and anterior STS (dorsal core system); and right IFG (anterior core system). Significant decoding accuracy was also present in areas of the extended system: right TPJ and MPFC(theory of mind areas), bilateral precuneus, and right insula. For visually familiar faces, significant between-subject decoding accuracy was limited to areas of the core system: bilateral OFA and fusiform gyrus, right mSTS, and right IFG. Between-subject decoding accuracy for personally familiar faces was higher than for visually familiar faces in large portions of the face processing network. Higher decoding accuracy for personally familiar faces was present in areas of the core system such as bilateral lateral occipital cortex (LOC), right mFFA, and right IFG, and areas of the extended system such as right TPJ, precuneus, MPFC, and insula.

We performed separate between-subject multivariate pattern classifications (MVPC) of responses to…



Read More:Shared neural codes for visual and semantic information about familiar faces in a common

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments