Category Archives: cognition

Doing science backwards

Leave a reply

A recent article, (Trettenbrein, P. (2016); The Demise of the Synapse As the Locus of Memory: A Looming Paradigm Shift?; Frontiers in Systems Neuroscience, 10), questions what many consider settled science – plastic changes to synapses are the basis of learning and memory – may not be correct. Thanks to Neurosceptic for noting this paper (here).

“Actually, as of today, large parts of the field have concluded, primarily drawing on work in neuroscience, that neither symbolism nor computationalism are tenable and, as a consequence, have turned elsewhere. In contrast, classical cognitive scientists have always been critical of connectionist or network approaches to cognitive architecture.”Trettenbrein is in the classical cognitive scientist camp.

First Trettenbrein assumes that the brain is a Turing machine. In other words that the coinage of thought is symbols and that they are manipulated by algorithms (programs) that write to a stable memory and read from it. The brain is assumed to deal in representation/symbols as variables, stepwise procedures as programs and random access memory, giving together a Turing machine. “The crucial feature of a Turing machine is its memory component: the (hypothetical) machine must possess a read/write memory in order to be vastly more capable than a machine that remembers the past only by changing the state of the processor, as does, for example, a finite-state machine without read/write memory. Thus, there must be an efficient way of storing symbols in memory (i.e., writing), locating symbols in memory (i.e., addressing), and transporting symbols to the computational machinery (i.e., reading). It is exactly this problem, argue Gallistel and King (2009), that has by and large been overlooked or ignored by neuroscientists. …”

“Synaptic plasticity is widely considered to be the neurobiological basis of learning and memory by neuroscientists and researchers in adjacent fields, though diverging opinions are increasingly being recognized. From the perspective of what we might call “classical cognitive science” it has always been understood that the mind/brain is to be considered a computational-representational system. Proponents of the information-processing approach to cognitive science have long been critical of connectionist or network approaches to (neuro-)cognitive architecture, pointing to the shortcomings of the associative psychology that underlies Hebbian learning as well as to the fact that synapses are practically unfit to implement symbols.” So an assumption that we have a Turing machine dictates that it needs a particular type of memory which is difficult to envisage with plastic synapses.

I like many others believe, science starts with observations and moves on to explanations of those observations, or to state it differently, the theories of science are based on physical evidence. It is not science to start with a theoretical assumption and argue from that assumption what has to be. Science starts with ‘what is’ not ‘what has to be’.

Trettenbrein is not thinking that the brain resembles a computer in many ways (computer metaphor), he is thinking that it IS a computer (actual Turing machine). If the brain is an actual computer than it is a Turing machine, working in a stepwise fashion controlled by an algorithmic program. Then he reasons that the memory must be individual neurons that are - what? Perhaps they are addressable items in the random access memory. Well, it seems that he does not know. “To sum up, it can be said that when it comes to answering the question of how information is carried forward in time in the brain we remain largely clueless… the case against synaptic plasticity is convincing, but it should be emphasized that we are currently also still lacking a coherent alternative.” We are not clueless (although there are lots of unknowns) and the case for synaptic plasticity is convincing (as it has convinced many/most scientists) because there is quite a bit of evidence for it. But if someone starts with an assumption, then looks for evidence and finds it hard to produce – they are doing their science backwards.

Trettenbrein is not doing neuroscience, not even biology, in fact not even science. There are a lot of useful metaphors that we use to help understand the brain but we should never get so attached to them that we believe they can take the place of physical evidence from actual brains.

Just because we use the same words does not mean that they describe the same thing. A neurological memory is not the same as a computer memory. Information in the neurological sense is not the same as the defined information of information theory. Brain simulations are not real brains. Metaphors give resemblances not definitions.

Beta waves

Leave a reply

Judith Copithorne image

Brain waves are measured for many reasons and they have been linked to various brain activities. But very little is known about how they arise. Are they the result or the cause of the activities they are associated with? How exactly are they produced at a cellular or network level? We know little about these waves.

One type of wave, beta waves (18-25 Hz) are associated with consciousness and alertness. In the motor cortex they are found when muscle contractions are isotonic (contractions that do not produce movement) but are absent just prior and during movement. They are increased during sensory feedback to static motor control and when movement is resisted or voluntarily suppressed. In the frontal cortex the beta waves are found during attention to cognitive tasks directed to the outside world. They are found in alert attentive states, problem solving, judgment, decision making, and concentration. The more involved the cognitive activity the faster the beta waves.

ScienceDaily reports a press release from Brown University on the work of Stephanie Jones and team, who are attempting to understand how beta waves arise. (here) Three types of study are used: MEG recordings, computer models, and implanted electrodes in animals.

The MEG recordings from the somatosensory cortex (sense of touch) and the inferior frontal cortex (higher cognition) showed a very distinct form for the beta waves, “they lasted at most a mere 150 milliseconds and had a characteristic wave shape, featuring a large, steep valley in the middle of the wave.” This wave form was recreated in a computer model of the layers of the cortex. “They found that they could closely replicate the shape of the beta waves in the model by delivering two kinds of excitatory synaptic stimulation to distinct layers in the cortical columns of cells: one that was weak and broad in duration to the lower layers, contacting spiny dendrites on the pyramidal neurons close to the cell body; and another that was stronger and briefer, lasting 50 milliseconds (i.e., one beta period), to the upper layers, contacting dendrites farther away from the cell body. The strong distal drive created the valley in the waveform that determined the beta frequency. Meanwhile they tried to model other hypotheses about how beta waves emerge, but found those unsuccessful.” The model was tested in mice and rhesus monkeys with implanted electrodes and was supported.

Where do the signals come from that drive the pyramidal neurons? The thalamus is a reasonable guess at the source. Thalamo-cortex-thalamus feedback loop makes those very contacts of the thalamus axons within the cortex layers. The thalamus is known to have signals with 50 millisecond duration. All of the sensory and motor information that enters the cortex (except smell) comes though the thalamus. It regulates consciousness, alertness and sleep. It is involved in processing sensory input and voluntary motor control. It has a hand in language and some types of memory.

The team is continuing their study. “With a new biophysical theory of how the waves emerge, the researchers hope the field can now investigate beta rhythms affect or merely reflect behavior and disease. Jones’s team in collaboration with Professor of neuroscience Christopher Moore at Brown is now testing predictions from the theory that beta may decrease sensory or motor information processing functions in the brain. New hypotheses are that the inputs that create beta may also stimulate inhibitory neurons in the top layers of the cortex, or that they may may saturate the activity of the pyramidal neurons, thereby reducing their ability to process information; or that the thalamic bursts that give rise to beta occupy the thalamus to the point where it doesn’t pass information along to the cortex.”

It seems very clear that understanding of overall brain function will depend on understanding the events at a cellular/circuit level; and that those processes in the cortex will not be understood without including other regions like the thalamus in the models.

Metaphors and shapes

Leave a reply

Judith Copithorne image

Metaphors (including analogs and similitudes) appear to be very basic to thought. These are very important to language and communication. A large bulk of dictionary meanings of words are actually old metaphors, that have been used so much and for so long that the words has lost its figurative root and become literal in their meaning. We simply do not recognize that it was once a metaphor. Much of our learning is metaphorical. We understand one complex idea by noticing its similarity to another complex idea that we already understand. For example, electricity is not easy to understand at first but we have learned to understand a great deal about how water flows as we have grown up by watching it. Basic electrical theory is often taught by comparing it to water. By and large, when we examine our knowledge of the world, we find it is rife with metaphors. We can trace many ways we think about things and events to ‘grounding’ in experiences of infants. The way babies establish movement and sensory information is the foundation of enormous trees and pyramids of metaphorical understanding.

But what is a metaphor? We can think of it as a number of entities that are related in some way (in space, in time, in cause-effect, or in logic etc.) to form a structure that we can understand and think of/ remember/ name/ use as a predictive model and treat as a single thing. This structure can be reused without being reinvented. The entities can be re-labeled and so can the relations between them. So if we know water flowing through a pipe will be limited by a narrower length of pipe we can envisage an electrical current in a wire being limited by a resistor. Nothing needs to be retained in a metaphor but the abstract structure. This facility of being able to manipulate metaphors is important to thinking, learning, communicating. Is there more? Perhaps.

A recent paper (Rolf Inge Godøy, Minho Song, Kristian Nymoen, Mari Romarheim Haugen, Alexander Refsum Jensenius; Exploring Sound-Motion Similarity in Musical Experience; Journal of New Music Research, 2016; 1) talks about the use of a type of metaphor across the senses and movement. Here is the abstract:

“People tend to perceive many and also salient similarities between musical sound and body motion in musical experience, as can be seen in countless situations of music performance or listening to music, and as has been documented by a number of studies in the past couple of decades. The so-called motor theory of perception has claimed that these similarity relationships are deeply rooted in human cognitive faculties, and that people perceive and make sense of what they hear by mentally simulating the body motion thought to be involved in the making of sound. In this paper, we survey some basic theories of sound-motion similarity in music, and in particular the motor theory perspective. We also present findings regarding sound-motion similarity in musical performance, in dance, in so-called sound-tracing (the spontaneous body motions people produce in tandem with musical sound), and in sonification, all in view of providing a broad basis for understanding sound-motion similarity in music.”

The part of this paper that I found most interesting was a discussion of abstract ‘shapes’ being shared by various senses and motor actions.

“A focus on shapes or objects or gestalts in perception and cognition has particularly concerned so-called morphodynamical theory … morphodynamical theory claims that human perception is a matter of consolidating ephemeral sensory streams (of sound, vision, touch, and so on) into somehow more solid entities in the mind, so that one may recall and virtually re-enact such ephemeral sensations as various kinds of shape images. A focus on shape also facilitates motion similarity judgments and typically encompasses, first of all, motion trajectories (as so-called motion capture data) at various timescales (fast to slow, including quasi-stationary postures) and amplitudes (from large to small, including relative stillness). But shapes can also capture perceptually and affectively highly significant derivatives, such as acceleration and jerk of body motion, in addition.”

The authors think of sound objects as occurring in the time range of half a second to five seconds. Sonic objects have pitch and timbre envelopes, rhythmic, melodic and harmonic patterns. In terms of dynamics, sonic objects can: be impulsive with an envelop showing an abrupt onset and then decay, or be sustained with a gradual onset and longer duration, or be iterative with rapidly repeated sound, tremolo, or drum roll. Sonic objects could have pitch that is stable, variable or just noise. These sonic objects are related to similar motion objects – objects in the same time range that produce music or react to it. For example the sonic objects in playing a piano piece or in dancing. They also have envelopes of velocity and so on. This reminds me of the similar emotions that are triggered by similar envelopes of musical sound and speech. Or, the objects that fit with the nonsense words ‘bouba’ and ‘kiki’ being smooth or sharp. ‘Shape’ is a very good description of the vague but strong and real correspondences between objects from different domains. It is probably the root of being able to use adjectives across domains. For example, we can have soft light, soft velvet, soft rustle, soft steps, soft job, and more or less soft anything. Soft describes different things in different domains but, despite the differences, it is a metaphoric connection between domains so that concrete objects can be made by combining a number of individual sensory/motor objects which share abstract characteristics like soft.

“In several studies of cross-modal features in music, a common element seems to be the association of shape similarity with sound and motion, and we believe shape cognition can be considered a basic amodal element of human cognition, as has been suggested by the aforementioned morphodynamical theory …. But for the implementation of shape cognition, we believe that body motion is necessary, and hence we locate the basis for amodal shape cognition in so-called motor theory. Motor theory is that which can encompass most (or most relevant) modalities by rendering whatever is perceived (features of sound, textures, motion, postures, scenes and so on) as actively traced shape images.”

The word ‘shape’, used to describe corresponding characteristics from different domains, is very like the word ‘structure’ in metaphors and may point to the foundation of our cognition mechanisms, including much more than just the commonplace metaphor.

 

Complexity of conversation

Leave a reply

Language is about communication. It can be studied as written sentences, as production of spoken language, or as comprehension of spoken language, but these do not get to the heart of communicating. Language evolved as conversation, each baby learns it in conversation and most of our use of it each day is in conversations. Exchanges, taking turns, is the essence of language. A recent paper by S. Levinson in Trends in Cognitive Sciences, “Turn-taking in Human Communication – Origins and Implications for Language Processing”, looks at the complications of turn-taking.

The world’s languages vary in almost all levels of organization but there is a striking similarity in exchanges – rapid turns of short phrases or clauses within single sound envelopes. There are few long gaps or much overlapping speech during the changes of speaker. Not only is a standard turn-taking universal in human cultures but it is found in all types of primates and it is learned by babies before any language is acquired. It may be the oldest aspect of our language.

But it is paradoxical – for the gap between speakers is too short to produce a response to what has been said by the last speaker. In fact, the gap tends to be close to the minimum reflex time. A conversational speaking turn averages 2 seconds (2000ms) and the gap between speakers is about 200ms, but it takes 600ms to prepare the first word (1500ms for a short phrase). So it is clear that production and comprehension must go on at the same time in the same areas of the brain and that comprehension must include a good deal of prediction of how a phrase is going to end. Because comprehension and production have been studied separately, it is not clear how this multitasking, if that is what it is, is accomplished. First, the listener has to figure out what sort of utterance the speaker is making – statement, question, command or whatever. Without this the listener does not know what sort of reply is appropriate. The listener then must predict (guess) the rest of the utterance, decide what the response should be and formulate it. Finally the listener must recognize the signal/s of when the end of the utterance will be. The listener can immediately begin to talk as soon as the utterance ends. There is more to learn about how the brain does this and what the effect of turn-taking has on the nature of language.

There are cultural conventions that override turn-taking so that speakers can talk for some time without interruption, and even if they pause from time to time, no one jumps in. Of course, if someone speaks for too long without implicit permission, they will be forcibly interrupted fairly soon, people will drift away or some will start new conversations in sub-groups. That’s communication.

Here is the abstract of - Stephen C. Levinson. Turn-taking in Human Communication – Origins and Implications for Language Processing. Trends in Cognitive Sciences, 2015:

“Most language usage is interactive, involving rapid turn-taking. The turn-taking system has a number of striking properties: turns are short and responses are remarkably rapid, but turns are of varying length and often of very complex construction such that the underlying cognitive processing is highly compressed. Although neglected in cognitive science, the system has deep implications for language processing and acquisition that are only now becoming clear. Appearing earlier in ontogeny than linguistic competence, it is also found across all the major primate clades. This suggests a possible phylogenetic continuity, which may provide key insights into language evolution.

Trends

The bulk of language usage is conversational, involving rapid exchange of turns. New information about the turn-taking system shows that this transition between speakers is generally more than threefold faster than language encoding. To maintain this pace of switching, participants must predict the content and timing of the incoming turn and begin language encoding as soon as possible, even while still processing the incoming turn. This intensive cognitive processing has been largely ignored by the language sciences because psycholinguistics has studied language production and comprehension separately from dialog.

This fast pace holds across languages, and across modalities as in sign language. It is also evident in early infancy in ‘proto-conversation’ before infants control language. Turn-taking or ‘duetting’ has been observed in many other species and is found across all the major clades of the primate order.”

 

Liking the easy stuff

1 Reply

It is not just true that if something is not understood, it is assumed to be easily done. It is also true that if it is easier to grasp then it is more likeable. A recent study looked at this connection between fluency and appreciation. (Forster M, Gerger G, Leder H (2015) Everything’s Relative? Relative Differences in Processing Fluency and the Effects on Liking. PloS ONE 10(8): e0135944. doi:10.1371/journal. pone.0135944)

The question Forster asks is whether the judgement of fluency is absolute or relative. If we have internal reference standards for liking that depend on the ease of perceiving then the level of liking is an absolute judgement. Internal standards seem to be the case for perfect pitch and the feeling of familiarity when something is recalled from memory. But in the case of the effort of perception, our feeling of liking is a relative judgement – a comparison with other amounts of effort for other images.

Abstract: “Explanations of aesthetic pleasure based on processing fluency have shown that ease-of-processing fosters liking. What is less clear, however, is how processing fluency arises. Does it arise from a relative comparison among the stimuli presented in the experiment? Or does it arise from a comparison to an internal reference or standard? To address these questions, we conducted two experiments in which two ease-of-processing manipulations were applied: either (1) within-participants, where relative comparisons among stimuli varying in processing ease were possible, or (2) between-participants, where no relative comparisons were possible. In total, 97 participants viewed simple line drawings with high or low visual clarity, presented at four different presentation durations, and rated for felt fluency, liking, and certainty. Our results show that the manipulation of visual clarity led to differences in felt fluency and certainty regardless of being manipulated within- or between-participants. However, liking ratings were only affected when ease-of-processing was manipulated within-participants. Thus, feelings of fluency do not depend on the nature of the reference. On the other hand, participants liked fluent stimuli more only when there were other stimuli varying in ease-of-processing. Thus, relative differences in fluency seem to be crucial for liking judgements.”

Counting crows

1 Reply

It is getting to be common knowledge that some birds can count. Recent research (citation below) has shown some of the details of how crows handle numbers. They have a different brain architecture from mammals but in some ways show similar functions to our neo-cortex in their endbrain association area. This points to possible convergent evolution.

Ditz and Nieder planted electrodes in the endbrain of crows and recorded activity of NLC (nidopallium caudolaterale) neurons. The birds were shown groups of items and the NCL neurons shown activity to specific numbers of items. The activity of a particular neuron peaked at a particular number.

Here is the abstract: “It is unknown whether anatomical specializations in the endbrains of different vertebrates determine the neuronal code to represent numerical quantity. Therefore, we recorded single-neuron activity from the endbrain of crows trained to judge the number of items in displays. Many neurons were tuned for numerosities irrespective of the physical appearance of the items, and their activity correlated with performance outcome. Comparison of both behavioral and neuronal representations of numerosity revealed that the data are best described by a logarithmically compressed scaling of numerical information, as postulated by the Weber–Fechner law. The behavioral and neuronal numerosity representations in the crow reflect surprisingly well those found in the primate association cortex. This finding suggests that distantly related vertebrates with independently developed endbrains adopted similar neuronal solutions to process quantity.”

It is interesting, and confirms other bird studies, that:

  • they can put items in categories in order to count them,
  • they can make a set of the items in a particular category,
  • they can assess the quantity on a logarithmic scale (like 1, 2, 3, 4, 6ish, 9ish, 15ish etc),
  • this is an abstract quantity and does not depend on the arrangement, size etc. of the items.

Citation: Helen M. Ditz and Andreas Nieder. Neurons selective to the number of visual items in the corvid songbird endbrain. PNAS, June 2015 DOI: 10.1073/pnas.1504245112

 

Music affects on the brain

2 Replies

A recent paper identified genes that changed their expression as a result of music performance in trained musicians. (see citation below). There were a surprising number of affected genes, 51 genes had increased and 22 had decreased expression, compared to controls who were also trained musicians but were not involved in making or listening to music for the same time period. It is also impressive that this set of 73 genes has a very broad range of presumed functions and effects in the brain.

Another interesting aspect is the overlap of a number of these genes with some that have been identified in song birds. This implies that the music/sophisticated sound perception and production has been conserved from a common ancestor of birds and mammals.

It has been known for some time that musical training has a positive effect on intelligence and outlook – that it assists learning. Musical training changes the structure of the brain. Now scientists are starting to trace the biology of music’s effects. Isn’t it about time that education stopped treating music (and other arts for that matter) as unimportant frills? It should not be the first thing to go when money or teaching time is short.

Here is the Abstract:

“Music performance by professional musicians involves a wide-spectrum of cognitive and multi-sensory motor skills, whose biological basis is unknown. Several neuroscientific studies have demonstrated that the brains of professional musicians and non-musicians differ structurally and functionally and that musical training enhances cognition. However, the molecules and molecular mechanisms involved in music performance remain largely unexplored. Here, we investigated the effect of music performance on the genome-wide peripheral blood transcriptome of professional musicians by analyzing the transcriptional responses after a 2-hr concert performance and after a ‘music-free’ control session. The up-regulated genes were found to affect dopaminergic neurotransmission, motor behavior, neuronal plasticity, and neurocognitive functions including learning and memory. Particularly, candidate genes such as SNCA, FOS and DUSP1 that are involved in song perception and production in songbirds, were identified, suggesting an evolutionary conservation in biological processes related to sound perception/production. Additionally, modulation of genes related to calcium ion homeostasis, iron ion homeostasis, glutathione metabolism, and several neuropsychiatric and neurodegenerative diseases implied that music performance may affect the biological pathways that are otherwise essential for the proper maintenance of neuronal function and survival. For the first time, this study provides evidence for the candidate genes and molecular mechanisms underlying music performance.”

ResearchBlogging.org

Kanduri, C., Kuusi, T., Ahvenainen, M., Philips, A., Lähdesmäki, H., & Järvelä, I. (2015). The effect of music performance on the transcriptome of professional musicians Scientific Reports, 5 DOI: 10.1038/srep09506

A radical suggestion

3 Replies

How much and what type of our thinking is consciously done? The naïve feeling is that all our thinking is conscious. But we know better and tend to believe that a good deal of our thoughts are created unconsciously. I want to put forward the notion that none of our thoughts are the product of consciousness. Please set aside your disbelief for a short while in order to understand this idea and then you can resume your critical faculties and judge it.

Consciousness is about memory not thought. We cannot remember experiences unless we consciously experienced them. We can only know that we have been unconscious be noticing a discontinuity in our memory. We are probably only forced to have conscious experience of items that have been held in working memory – this has been called type 2 cognition, which always forms a conscious experience and uses working memory. That does not necessarily mean that the type 2 cognition is a product of the mechanism of consciousness.

Memory of experiences has functions. Why would we remember an event? We might find such information useful in future is about the only answer. For example, if we know there is a nasty dog in a particular yard, we may want to notice whether the gate is closed before we pass by. The various places we have experienced and mapped in memory have a lot of information associated with them. That is useful to have ‘on tap’ when we find ourselves in a particular place. ‘Where’ is an important element of the memory of an event. Also ‘when’, ‘who’ and ‘what’ are elements of most events. This information is available from the mechanisms of perception, recognition, navigation etc. We know that the processes that create these elements are not conscious, but the end product is. We also want other pieces of information to form an event to remember and use in recall. We want to know the event’s place in chains of cause and effect, whether it was an important event, what our emotional involvement was, whether it was a surprise or predicted. A very important element has to do with agency. We want to know whether we had any part in causing the event, and if we did was it deliberate or accidental, and whether the outcome was favourable or not. We assume that much of this volition information is created by conscious rather than unconscious mechanisms but experiments put that in doubt. And quite honestly there is no way that we could tell the difference.

Consciousness only needs to contain what is worth remembering but not all may be remembered. We can think of consciousness as the leading edge of memory containing all the information needed for the stable memory. However, we really do need to tell the difference between the ‘now’ and the stored memory of the past. And, although a fairly full description of ‘now’ may be delivered to short-term memory, much of it may be discarded before it reaches a more stable form. Memories are sketchy compared to conscious experience. The conscious stage of memory also has access to the current state of much of the brain. Low-level vision, hearing, feeling etc. can be used by the conscious model of ‘now’ to give it vivid realism – this would not be as easy for older memories.

Of course, these episodic memories are not our only memories and there are memories that are not produced from consciousness. Consciousness may have other functions than memory. All that I am trying to show here is that it is possible that consciousness is not involved in cognition. It may record some aspects if they will be important to remember for the future, but consciousness is not a cognition or thought engine in the brain. It is the engine to assemble experiences to be remembered as experiences.

Resume critical faculties…

Hand gestures

Leave a reply

About 20 years ago I took an interest in the non-verbal part of speech communication: gesture, facial expression, posture, tone of voice. During this time I watched the hands of speakers carefully and noted how they gestured. I saw four types of movement that seemed distinct:

One.. Word gestures took the place of words, quite literally. They were made at the point were the word would be used and there was a gap in speaking for it to fit into so to speak. Also treated like words were the (sometimes impolite) gestures for which Italians are famous.

Two.. Illustrating gestures do not interrupt speech but are separately ‘saying’ the same as the words, like miming while talking.

Three.. There are emotional gestures that are very ancient and even understood across species. They are often completely unconscious. Palms towards the body communicate submission or at least non-aggression. Palms away from the body communicate rejection or defense.

Four.. The fourth type is also usually unconscious. I called it baton gestures. They set a rhythm to the speech and quite often the listener moved in keeping with the baton. It also seemed to emphasize important phrases. The baton beat seemed to mark out groups of words that should be processed together, a great help to a listener if they used it to wrap up one meaning and start on the analysis of the next words.

It is this last type that has been the subject of a recent paper. Unfortunately I have no access to the paper and must be content with the abstract. (grr) Here are the abstracts of this paper and an earlier one by the same authors.

Abstract of (Biau, Torralba, Fuentemilla, Balaguer, Soto-Faraco; Speaker’s hand gestures modulate speech perception through phase resetting of ongoing neural oscillations; Cortex Dec 2014) “Speakers often accompany speech with spontaneous beat gestures in natural spoken communication. These gestures are usually aligned with lexical stress and can modulate the saliency of their affiliate words. Here we addressed the consequences of beat gestures on the neural correlates of speech perception. Previous studies have highlighted the role of theta oscillations in temporal prediction of speech. We hypothesized that the sight of beat gestures may influence ongoing low-frequency neural oscillations around the onset of the corresponding words. Electroencephalographic (EEG) recordings were acquired while participants watched a continuous, naturally recorded discourse. The phase-locking value (PLV) at word onset was calculated from the EEG from pairs of identical words that had been pronounced with and without a concurrent beat gesture in the discourse. We observed an increase in PLV in the 5-6 Hz theta range as well as a desynchronization in the 8-10 Hz alpha band around the onset of words preceded by a beat gesture. These findings suggest that beats tune low-frequency oscillatory activity at relevant segments during natural speech perception, providing a new insight of how speech and paralinguistic information are integrated.”

Abstract of (Biau, Soto-Faraco; Beat gestures modulate auditory integration in speech perception; Brain and Language 124, 2, Feb 1013) “Spontaneous beat gestures are an integral part of the paralinguistic context during face-to-face conversations. Here we investigated the time course of beat-speech integration in speech perception by measuring ERPs evoked by words pronounced with or without an accompanying beat gesture, while participants watched a spoken discourse. Words accompanied by beats elicited a positive shift in ERPs at an early sensory stage (before 100 ms) and at a later time window coinciding with the auditory component P2. The same word tokens produced no ERP differences when participants listened to the discourse without view of the speaker. We conclude that beat gestures are integrated with speech early on in time and modulate sensory/phonological levels of processing. The present results support the possible role of beats as a highlighter, helping the listener to direct the focus of attention to important information and modulate the parsing of the speech stream.”

Also from a summary of an oral presentation by the same group: “We observed an increase in phase-locking at the delta–theta frequency range (2–6 Hz) from around 200 ms before word-onset to 200 ms post word-onset, when words were accompanied with a beat gesture compared to audio alone. Furthermore, this increase in phase-locking, most noticeable at fronto-central electrodes, was not accompanied by an increase in power in the same frequency range, confirming the oscillatory-based nature of this effect. These results suggest that beat gestures are used as robust predictive information capable to tune neural oscillations to the optimal phase for auditory integration of relevant parts of the discourse during natural speech processing.”

This research points to a synchronization between speaker and listener where a visual clue is used to divide the speech stream into chunks that can be processed (at least to a large degree) in isolation from the words before and after the chunk. The warning at the beginning of a new chunk, given automatically by the speakers hands, is used automatically by the listener to ‘clear the decks’ and begin a new chunk. This takes some of the strain out of listening. Of course, this information probably is also carried by the voice as well. Redundancy in oral language is common. Conversation is a wonderful dance of voice, face, hands and body that transfers an idea from one brain to another. It only seems easy because of how complicated and automatic it is.

 

Ways to navigate

Leave a reply

When I was a little girl, my father stood me on the door step and pointed across the yard and said, “that’s north”. He went on that the house behind me was south, the village was west and the grove of trees was east. To this day when I think of north I see the barn and so on; my sense of direction is based, even after 70 odd years, on the vision of the farm yard I grew up in. I have a small problem with left and right, but if I just think of facing the barn then left is in the west towards the village. Until I traveled away from the flat prairies, that was all I needed and the only skill required was to keep track of where north was. I found later that landmarks were useful and so was a map.

My husband has his own way of finding his way and never seems to worry about the cardinal directions. He does not seem to keep an continuous, unconscious tally of which way he is facing. His only way of dealing with cardinal directions is to know that the sun is going to be to the south and going from east to west during the day. (This was a problem when he was first in the tropics where the sun is not always to the south – he could get lost within half a city block.) I had never paid any attention to the sun to know which direction I was going – it had never occurred to me. It is clear to me that there is more than one way to navigate.

A recent paper (citation below) examines types of navigation. Head-direction cells in the entorhinal/subicular area have been known for some time. It appears to be why Alzheimer’s sufferers tend to lose their sense of direction early in the disease; one of the first areas affected is the entorhinal cortex. But heading cells alone cannot give navigation accuracy. What is needed is a goal-direction cell to work with heading to keep movement in the direction of the goal. And this directional information has to be framed in either a world view (north, south, east, west) or a self view (left, right, forward, back). The geocentric information appears to be processed in the entorhinal/subicular area, egocentric information in the precuneus region. Navigation could also be done by following a sequence of visible landmarks using the place cells of the hippocampus. All of these methods could and would be used depending on the circumstances.

The researchers looked for goal-direction cells using multivoxel pattern analysis. (The method used to try and guess which video someone was watching that caused the interest in ‘mind reading’ last year; or, as reported in a previous posting, the difference between physical and social pain.) They found that the direction of the goal is stored by the same cells as the direction the body is facing. These cells were in the entorhinal/subicular area and geocentric. The same cells could be used for both heading and goal direction. The exact way this is done was not clear in this study. “Due to the relatively poor temporal resolution of fMRI, we are not able to determine what the temporal dynamics of head-direction simulation may be. Our assumption is that head-direction populations are initially involved in representing current facing direction and then switch to simulation during navigational planning. However, other temporal dynamics, such as constant oscillation between facing and goal direction, would explain our results equally well. Thus, we remain agnostic regarding the precise temporal dynamics involved in head-direction simulation, which will have to be resolved with alternative methodological approaches. ”

Their findings are relevant to actual navigation. “We found a significant positive correlation between entorhinal/subicular facing direction information and overall task accuracy….These results therefore show that participants with a stronger representation of current heading direction are both more accurate and faster at making goal direction judgments in this task ”

Here is the abstract: “Navigating to a safe place, such as a home or nest, is a fundamental behavior for all complex animals. Determining the direction to such goals is a crucial first step in navigation. Surprisingly, little is known about how or where in the brain this ‘‘goal direction signal’’ is represented. In mammals, ‘‘head-direction cells’’ are thought to support this process, but despite 30 years of research, no evidence for a goal direction representation has been reported. Here, we used fMRI to record neural activity while participants made goal direction judgments based on a previously learned virtual environment. We applied multivoxel pattern analysis to these data and found that the human entorhinal/subicular region contains a neural representation of intended goal direction. Furthermore, the neural pattern expressed for a given goal direction matched the pattern expressed when simply facing that same direction. This suggests the existence of a shared neural representation of both goal and facing direction. We argue that this reflects a mechanism based on head-direction populations that simulate future goal directions during route planning. Our data further revealed that the strength of direction information predicts performance. Finally, we found a dissociation between this geocentric information in the entorhinal/subicular region and egocentric direction information in the precuneus.”
ResearchBlogging.org

Chadwick, M., Jolly, A., Amos, D., Hassabis, D., & Spiers, H. (2014). A Goal Direction Signal in the Human Entorhinal/Subicular Region Current Biology DOI: 10.1016/j.cub.2014.11.001