Reading patterns

There is a paper (citation below) that takes a different look at language. It attempts to examine what happens in the brain when we read a story. There is the act of reading, the processing of the language, and the engagement in the story, all going on at the same time.

One of the main questions in the study of language processing in the brain is to understand the role of the multiple regions that are activated in response to reading. A network of multiple brain regions have been implicated in language, and while the view of the field started with a simplistic dissociation between the roles of Broca’s area and Wernicke’s area, the current theories about language comprehension are more complex and most of them involve different streams of information that involve multiple regions (including Broca’s and Wernicke’s).” By studying sub-processes in isolation, previous studies have resulted in a confused picture. The researchers changed the method and looked at all parts of the brain at the same time in a normal natural reading situation (reading a chapter of a Harry Potter book). “We extract from the words of the chapter very diverse features and properties (such as semantic and syntactic properties, visual properties, discourse level features) and then examine which brain areas have activity that is modulated by the different types of features, leading us to distinguish between brain areas on the basis of which type of information they represent.” This is unlike the usual method of finding the areas of the brain with the most change (those that ‘light up’ or ‘go dark’) during some activity or process. Here what is being noted is changes in pattern. They used a program that had been trained to predict the fMRI activation pattern for a piece of text from training with passages that had each word tagged with 195 features (size, part of speech, role in parsed sentence, emotion, involved with a particular character and the like). The program uses brain-wide patterns, not the activity of individual areas. “The model makes predictions of the fMRI activation for an arbitrary text passage, by capturing how this diverse set of information contributes to the neural activity, then combining these diverse neural encodings into a single prediction of brain-wide fMRI activity over time. Our model not only accounts for the different levels of processing involved in story comprehension; it goes further by explicitly searching for the brain activity encodings for individual stimuli such as the mention of a specific story character, the use of a specific syntactic part-of-speech or the occurrence of a given semantic feature. … It has not been shown previously that one could model in detail the rapidly varying dynamics of brain activity with fMRI while reading at a close to normal speed.

Many of the results of the natural reading while being scanned are not surprising. But there are some very interesting insights. We think of language, especially syntax, as being primarily a left hemisphere function. “The strong right temporal representation of syntax that we found was not expected. Indeed we did not find other papers that report the large right hemisphere representation of sentence structure or syntax that we obtain. One reason might be that our syntax features are unique: whereas most experiments have approximated syntactic information in terms of processing load (length of constituents, hard vs easy phrase structure etc.) we model syntax and structure using a much more detailed set of features. Specifically, our model learns distinct neural encodings for each of 46 detailed syntax features including individual parts of speech, (adjectives, determiners, nouns, etc.) specific substructures in dependency parses (noun modifiers, verb subjects, etc.), and punctuation. Earlier studies considering only increases or decreases in activity due to single contrasts in syntactic properties could not detect detailed neural encodings of this type. We hypothesize that these regions have been previously overlooked.

There have been questions in the past about how connected syntactic and semantic processing are. “The question whether the semantics and syntactic properties are represented in different location has been partially answered by our results. There seems to be a large overlap in the areas in which both syntax and semantics are represented.

 

The characters actions seems to use areas of imagined action. But dialog may make special demands. “Presence of dialog among story characters was found to modulate activity in many regions in the bilateral temporal and inferior frontal cortices; one plausible hypothesis is that dialog requires additional processing in the language regions. More interestingly, it seems like presence of dialog activates the right temporo-parietal junction, a key theory of mind region. This observation raises an exciting hypothesis to pursue: that the presence of dialog increases the demands for perspective interpretation and recruits theory of mind regions.

This is a great step forward in studying language in the context of actual communication.

Abstract:

Story understanding involves many perceptual and cognitive subprocesses, from perceiving individual words, to parsing sentences, to understanding the relationships among the story characters. We present an integrated computational model of reading that incorporates these and additional subprocesses, simultaneously discovering their fMRI signatures. Our model predicts the fMRI activity associated with reading arbitrary text passages, well enough to distinguish which of two story segments is being read with 74% accuracy. This approach is the first to simultaneously track diverse reading subprocesses during complex story processing and predict the detailed neural representation of diverse story features, ranging from visual word properties to the mention of different story characters and different actions they perform. We construct brain representation maps that replicate many results from a wide range of classical studies that focus each on one aspect of language processing and offer new insights on which type of information is processed by different areas involved in language processing. Additionally, this approach is promising for studying individual differences: it can be used to create single subject maps that may potentially be used to measure reading comprehension and diagnose reading disorders.”
ResearchBlogging.org

Wehbe, L., Murphy, B., Talukdar, P., Fyshe, A., Ramdas, A., & Mitchell, T. (2014). Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses PLoS ONE, 9 (11) DOI: 10.1371/journal.pone.0112575

Leave a Reply

Your email address will not be published. Required fields are marked *