Can ChatGPT help researchers understand how the human brain handles language?

The ventral visual stream underlies key human visual object recognition abilities. However, neural encoding in the higher areas of the ventral stream remains poorly understood. Here, we describe a modeling approach that yields a quantitatively accurate model of inferior temporal (IT) cortex, the highest ventral cortical area. Using high-throughput computational techniques, we discovered that, within a class of biologically plausible hierarchical neural network models, there is a strong correlation between a model's categorization performance and its ability to predict individual IT neural unit response data. To pursue this idea, we then identified a high-performing neural network that matches human performance on a range of recognition tasks. Critically, even though we did not constrain this model to match neural data, its top output layer turns out to be highly predictive of IT spiking responses to complex naturalistic images at both the single site and population levels. Moreover, the model's intermediate layers are highly predictive of neural responses in the V4 cortex, a midlevel visual area that provides the dominant cortical input to IT. These results show that performance optimization--applied in a biologically appropriate model class--can be used to build quantitative predictive models of neural processing.

The meaning of language is represented in regions of the cerebral cortex collectively known as the “semantic system”. However, little of the semantic system has been mapped comprehensively, and the semantic selectivity of most regions is unknown. Here we systematically map semantic selectivity across the cortex using voxel-wise modeling of fMRI data collected while subjects listened to hours of narrative stories. We show that the semantic system is organized into intricate patterns that appear consistent across individuals. We then use a novel generative model to create a detailed semantic atlas. Our results suggest that most areas within the semantic system represent information about specific semantic domains, or groups of related concepts, and our atlas shows which domains are represented in each area. This study demonstrates that data-driven methods—commonplace in studies of human neuroanatomy and functional connectivity—provide a powerful and efficient means for mapping functional representations in the brain.

Neuroscientists have debated for centuries whether some regions of the human brain are selectively engaged in specific high-level mental functions or whether, instead, cognition is implemented in multifunctional brain regions. For the critical case of language, conflicting answers arise from the neuropsychological literature, which features striking dissociations between deficits in linguistic and nonlinguistic abilities, vs. the neuroimaging literature, which has argued for overlap between activations for linguistic and nonlinguistic processes, including arithmetic, domain general abilities like cognitive control, and music. Here, we use functional MRI to define classic language regions functionally in each subject individually and then examine the response of these regions to the nonlinguistic functions most commonly argued to engage these regions: arithmetic, working memory, cognitive control, and music. We find little or no response in language regions to these nonlinguistic functions. These data support a clear distinction between language and other cognitive processes, resolving the prior conflict between the neuropsychological and neuroimaging literatures.