local or not

A recent press release describes a paper ( T. A. Engel, N. A. Steinmetz, M. A. Gieselmann, A. Thiele, T. Moore, K. Boahen. Selective modulation of cortical state during spatial attention. Science, 2016; 354 (6316): 1140 DOI: 10.1126/science.aag1420 ) on the neural activity during awake attention. Here is the abstract:

Neocortical activity is permeated with endogenously generated fluctuations, but how these dynamics affect goal-directed behavior remains a mystery. We found that ensemble neural activity in primate visual cortex spontaneously fluctuated between phases of vigorous (On) and faint (Off) spiking synchronously across cortical layers. These On-Off dynamics, reflecting global changes in cortical state, were also modulated at a local scale during selective attention. Moreover, the momentary phase of local ensemble activity predicted behavioral performance. Our results show that cortical state is controlled locally within a cortical map according to cognitive demands and reveal the impact of these local changes in cortical state on goal-directed behavior.

I find the techniques and the results very interesting. However, I have trouble with the idea that attention has a purely cortical mechanism. Why are the fluctuations in activity said to be endogenously generate? Why is the cortical state controlled locally within a cortical map according to cognitive demands and reveal the impact of these local changes in cortical state on goal-directed behavior? The cortex is not isolated from the rest of the brain. To say some effect is locally generated in the cortex would required showing that the activity level was not affected by the thalamus and associated parts of the brain. The back and forth between cortical columns and the thalamus is the key to cortical function and a requirement for attention, consciousness and wakefulness. This is not a new idea but has been around for a long time. Why does this study not just ignore it, but deny it?

The conclusion to a paper (Sallmann and Kastner, Cognitive and Perceptual Functions of the Visual Thalamus Neuron. 2011 Jul 28; 71(2): 209–223) outlines some signaling between various parts of the thalamus and the cortex.

The overall evidence that has emerged during recent years suggests that the visual thalamus serves a fundamental function in regulating information transmission to the cortex and between cortical areas according to behavioral context. Selective attention and visual awareness have been shown to modulate LGN (thalamus lateral geniculate nucleus) activity, thus indicating that the LGN filters visual information before it reaches the cortex. Behavioral context appears to even more strongly modulate pulvinar activity and, due to its connectivity, the pulvinar (a part of the thalamus) is well-positioned to influence feedforward and feedback information transmission between cortical areas. Because the TRN provides strong inhibitory input to both the LGN and pulvinar, the TRN (thalamic reticular nucleus) may control and coordinate the information transmitted along both retino-cortical and cortico-cortical pathways.

Parasuraman and Davis in Varieties of Attention, page 236, described the networks involved in attention as long ago as 1984.

Three interacting networks mediating different aspects of attention: (1) a posterior attention system comprising parietal cortex, superior colliculus (a midbrain area), and pulvinar(thalamus area) that is concerned was spatial attention; (2) anterior system centered on the anterior cingulate in the medial frontal lobe that mediates target detection and executive control; (3) a vigilance system consisting of the right frontal lobe and brainstem nuclei, principally the noradrenergic locus coerulus (LC).

The brain is a functioning whole not a group of completely independent parts. As the Engel group do not seem to even address the question of involvement of regions of the brain other then the cortex – how can they state that the activity level of a column is locally produced?



i have just lost my husband and will not have time or inclination to post for a while. I will be back in a a few months.

Beta waves

Judith Copithorne image

Judith Copithorne image

Brain waves are measured for many reasons and they have been linked to various brain activities. But very little is known about how they arise. Are they the result or the cause of the activities they are associated with? How exactly are they produced at a cellular or network level? We know little about these waves.

One type of wave, beta waves (18-25 Hz) are associated with consciousness and alertness. In the motor cortex they are found when muscle contractions are isotonic (contractions that do not produce movement) but are absent just prior and during movement. They are increased during sensory feedback to static motor control and when movement is resisted or voluntarily suppressed. In the frontal cortex the beta waves are found during attention to cognitive tasks directed to the outside world. They are found in alert attentive states, problem solving, judgment, decision making, and concentration. The more involved the cognitive activity the faster the beta waves.

ScienceDaily reports a press release from Brown University on the work of Stephanie Jones and team, who are attempting to understand how beta waves arise. (here) Three types of study are used: MEG recordings, computer models, and implanted electrodes in animals.

The MEG recordings from the somatosensory cortex (sense of touch) and the inferior frontal cortex (higher cognition) showed a very distinct form for the beta waves, “they lasted at most a mere 150 milliseconds and had a characteristic wave shape, featuring a large, steep valley in the middle of the wave.” This wave form was recreated in a computer model of the layers of the cortex. “They found that they could closely replicate the shape of the beta waves in the model by delivering two kinds of excitatory synaptic stimulation to distinct layers in the cortical columns of cells: one that was weak and broad in duration to the lower layers, contacting spiny dendrites on the pyramidal neurons close to the cell body; and another that was stronger and briefer, lasting 50 milliseconds (i.e., one beta period), to the upper layers, contacting dendrites farther away from the cell body. The strong distal drive created the valley in the waveform that determined the beta frequency. Meanwhile they tried to model other hypotheses about how beta waves emerge, but found those unsuccessful.” The model was tested in mice and rhesus monkeys with implanted electrodes and was supported.

Where do the signals come from that drive the pyramidal neurons? The thalamus is a reasonable guess at the source. Thalamo-cortex-thalamus feedback loop makes those very contacts of the thalamus axons within the cortex layers. The thalamus is known to have signals with 50 millisecond duration. All of the sensory and motor information that enters the cortex (except smell) comes though the thalamus. It regulates consciousness, alertness and sleep. It is involved in processing sensory input and voluntary motor control. It has a hand in language and some types of memory.

The team is continuing their study. “With a new biophysical theory of how the waves emerge, the researchers hope the field can now investigate beta rhythms affect or merely reflect behavior and disease. Jones’s team in collaboration with Professor of neuroscience Christopher Moore at Brown is now testing predictions from the theory that beta may decrease sensory or motor information processing functions in the brain. New hypotheses are that the inputs that create beta may also stimulate inhibitory neurons in the top layers of the cortex, or that they may may saturate the activity of the pyramidal neurons, thereby reducing their ability to process information; or that the thalamic bursts that give rise to beta occupy the thalamus to the point where it doesn’t pass information along to the cortex.

It seems very clear that understanding of overall brain function will depend on understanding the events at a cellular/circuit level; and that those processes in the cortex will not be understood without including other regions like the thalamus in the models.

Fighting Libet’s experiment

A post in Science of Us in Feb, by Christian Jarrett, reviews the Libet experiment and recent attempts to overturn the implications of it. (http://nymag/scienceofus/2016/02/a-neuroscience-finding-on-free-will.html ) I find the struggle to reverse Libet’s finding to be the result of a mistaken way of viewing thought. An enormous amount of effort has gone into failed attempts to show this experiment was flawed over the last 30 years. Why are the implications so hard for people to accept?

Here is the first bit of Jarrett’s article (underlining is mine).

Back in the 1980s, the American scientist Benjamin Libet made a surprising discovery that appeared to rock the foundations of what it means to be human. He recorded people’s brain waves as they made spontaneous finger movements while looking at a clock, with the participants telling researchers the time at which they decided to waggle their fingers. Libet’s revolutionary finding was that the timing of these conscious decisions was consistently preceded by several hundred milliseconds of background preparatory brain activity (known technically as “the readiness potential”).

The implication was that the decision to move was made nonconsciously, and that the subjective feeling of having made this decision is tagged on afterward. In other words, the results implied that free will as we know it is an illusion — after all, how can our conscious decisions be truly free if they come after the brain has already started preparing for them?

For years, various research teams have tried to pick holes in Libet’s original research. It’s been pointed out, for example, that it’s pretty tricky for people to accurately report the time that they made their conscious decision. But, until recently, the broad implications of the finding have weathered these criticisms, at least in the eyes of many hard-nosed neuroscientists, and over the last decade or so his basic result has been replicated and built upon with ever more advanced methods such as fMRI and the direct recording of neuronal activity using implanted electrodes.

These studies all point in the same, troubling direction: We don’t really have free will. In fact, until recently, many neuroscientists would have said any decision you made was not truly free but actually determined by neural processes outside of your conscious control.

That is the stumbling block: ‘neural processes outside of conscious control’. That is what some scientists are fighting so hard not to lose. The whole notion of what free will is rests on how we view who we are, what our consciousness is, and how control works.

When we think of who we are, we cannot separate self from non-self within our bodies. We are not really divided at the neck, or between the upper and lower parts of the brain, or between different ‘minds’ co-existing in one skull. This idea of two separate minds, that was inherited from Freud and others, has not been demonstrated to be true. It has not been shown that we have two distinct thinking minds that are somehow separate. Thinking appears to be a complex, widespread but interconnected and unified affair. Whether a particular thought process becomes conscious or remains non-conscious does not depend on the basic process of thought.

There is every reason to reject the notion of a separate conscious mind that thinks in a ‘conscious’ manner to produce conscious thoughts. We are aware of thoughts (some thoughts) but we are not aware of the mechanisms that produced the thoughts. We do not metaphorically hear the gears of thought production grinding. We are simply not aware of how thought happens. Consciousness is a form of awareness and probably not much more. There is awareness of some things that go on in the brain but not of all things or even the bulk of things.

So why are some thoughts made conscious while others aren’t? A good guess is that consciousness gives a remembered experience, an episodic memory, or at least the material for such memories. With memories of our actions, it would be important information to remember whether the action was our doing or just happened to us, whether it was accidental or intended, whether it was a choice or coerced, carefully planned or an automatic habit and so on. These pieces of information are important to save and so would be incorporated into conscious events. We need that information to learn from experience. Just because the feeling of having an intent, an urge and then an execution of an action is there in our conscious awareness does not mean that they were a form of conscious control. They are there as important parts of the event that consciousness is recording.

We can still control our actions, and we still can be aware of controlling our actions, but that does not mean that our awareness is producing the control that we are aware of. Consciousness does not produce the tree that I am aware of – it just produces the awareness. And you are just you, and not your awareness of you. There is reality and there are models of reality; there is territory and there are maps of the territory; there is an original and there are copies of the original. There is you and there is your awareness of you. You make decisions (with neural activity) but your awareness of a decisions is not the same as making it.

I personally find it a little difficult to understand why this idea of a conscious mind as opposed to a conscious awareness is so strong and indestructible an idea to most people. I cannot remember exactly how or when (it was a gradual process) but some time in my late teens, over 50 years ago, my consciousness became a flickering imperfect movie screen and not a thinking mind. So “determined by neural processes outside of conscious control” is obvious because there is no such thing as conscious control and what is more, it is a comforting rather than alarming viewpoint.

I am assuming that the current experiments with showing ‘free won’t’ will not turn out to be any more robust than the attempts to show free will. We shall see.

Why sleep

It would be surprising if there were a single function for sleep, but there are often articles implying that the mystery is solved and THE reason for sleep has been found. Recently there was one in the New Scientist which prompted my post (https://www.newscientist.com/article/2096921-mystery-of-what-sleep-does-to-our-brains-may-finally-be-solved/).

We can look logically at reasons why we sleep. For any biological behaviour or process, there can be a spectrum of causes. At the one end of the spectrum are the ultimate causes – the evolutionary reason, the function being carried out, and the ‘why’. At the other end are the proximate causes – the individual immediate cause, the trigger, and the ‘how’.

The thing that seems the most distance and universal ultimate cause is probably that it is evolutionarily difficult to be well fitted to two dissimilar niches at the same time. There are animals that are active in the day and adapted to that; they hid and conserve their energy at night because they are not adapted to night. Or an animal can be adapted to night and hid in the day. This idea applies to animals that hibernate through a cold season every year to which they are not well adapted. There are animals that go dormant in dry seasons and are active when it is wet. It would be a good bet that all other functions of sleep are built on this mechanism of being inactive during recurring periods. Sleep is widespread amongst animals.

If there is an inactive time, that is the time to do all the things that cannot be done easily when active. Growth and repair would be immensely easier during rest. Imagine growing new muscle cells in a walking leg – much easier to wait until the leg is not moving. Growth and repair is best done on all the organs when they are just ticking over at most. This would include any maintenance needed in the brain.

These ultimate causes produce, during evolution, a proximate mechanism: an oscillating function that produces drowsiness and than sleep and followed by awakening and activity. Hormones and signals can work this rhythm without triggers, but do use light and darkness to get the timing right. The mechanism also affects the working of the body to make it suitable for growth and repair – the temperature, heart and breathing rates and many hidden levels have different sleep and wake settings. This sort of mechanism causes sleep but it is the ‘how’ of sleep not the ‘why’.

The brain is a complex organ with many functions. It is not a simple concept to say that the brain is inactive, resting and recuperating. There may be many processes in the brain taking advantage of sleep; there are a number of different types of sleep which obviously have different functions.

One function that has been investigated is waste removal. During parts of sleep some of the brain cells become physically smaller and this allow the movement of liquid through the brain, clearing out waste. It probably would interfere with the working of the brain to have cells temporarily lose volume – better to do this during sleep.

Another function that sleep houses is REM phase/dreaming. Dreaming is not completely understood but it is clearly needed for the successful integration of new memories into the web of established memories in the cortex. In this process the memories seem to be partially activated and re-experienced in combination with previous memories. For this process to be safe (without sleep walking or worse) the brain disconnects the possibility of skeletal muscle movement. This paralysis could only be acceptable in the safe inactive state of sleeping.

Another function that is known but not completely understood is the resetting of the brain’s level of activity. This is the one outlined in the NewScientist article. Consider that during the day, there has been continuous firing of neurons as a result of continuous signaling through the synapses of the brain. The synapses that are not used do not lose any strength while the ones that are used increase in strength. We end the day with a very excitable brain. During sleep this is brought down to a lower level to start the next day. All the synapses lose strength so that the new difference between strong ones and weaker ones remains but the overall strength is lower. As a photographic metaphor, it is like lowering the ‘brightness’ while maintaining the ‘contrast’ on a washed out over exposed photo. If this is not done, due to sleep deprivation, then the brain finds it more and more difficult to function. It seems that sleep alternates between strengthening particular synapses and weakening all synapses.

Of course, this is probably only the tip of the iceberg for finding sleep functions in the brain. I would not be surprised at a number of others being identified. There may even be processes that apply to insects but not to humans, as insects can sleep too. That is why an article that implies that there is a single function, THE reason for sleep, and that single reason has finally been found, is annoying. All the causes of, reasons for, functions of sleep are important and unlikely to be all found or understood.

Click bait PR in science

ScienceDaily reports on a recent paper (Leon Gmeindl, Yu-Chin Chiu, Michael S. Esterman, Adam S. Greenberg, Susan M. Courtney, Steven Yantis. Tracking the will to attend: Cortical activity indexes self-generated, voluntary shifts of attention. Attention, Perception, & Psychophysics, 2016) which looks at the areas in the brain involved in volition. Here is the abstract:

The neural substrates of volition have long tantalized philosophers and scientists. Over the past few decades, researchers have employed increasingly sophisticated technology to investigate this issue, but many studies have been limited considerably by their reliance on intrusive experimental procedures (e.g., abrupt instructional cues), measures of brain activity contaminated by overt behavior, or introspective self-report techniques of questionable validity. Here, we used multivoxel pattern time-course analysis of functional magnetic resonance imaging data to index voluntary, covert perceptual acts—shifts of visuospatial attention—in the absence of instructional cues, overt behavioral indices, and self-report. We found that these self-generated, voluntary attention shifts were time-locked to activity in the medial superior parietal lobule, supporting the hypothesis that this brain region is engaged in voluntary attentional reconfiguration. Self-generated attention shifts were also time-locked to activity in the basal ganglia, a novel finding that motivates further research into the role of the basal ganglia in acts of volition. Remarkably, prior to self-generated shifts of attention, we observed early and selective increases in the activation of medial frontal (dorsal anterior cingulate) and lateral prefrontal (right middle frontal gyrus) cortex—activity that likely reflects processing related to the intention or preparation to reorient attention. These findings, which extend recent evidence on freely chosen motor movements, suggest that dorsal anterior cingulate and lateral prefrontal cortices play key roles in both overt and covert acts of volition, and may constitute core components of a brain network underlying the will to attend.

I have not been able to read the original paper but I assume that it is a careful and useful study of how intentions and decisions happen when there is no compulsion involved. It has further evidence of the dorsal anterior cingulate and lateral prefrontal areas being involved in preparation of voluntary action. I assume that the authors do not stoop to ‘click bait’ in the original paper; I assume they use the sort of language that they use in the abstract. The press release put out by Johns Hopkins University is the problem. There are repeated uses of the phrase ‘free will’ and even the phrase “volition, or free will” implying that these words are interchangeable. And ‘free will’ is even used in the title of the press release, which seems like clear click bait to me. There is still debate on whether free will exists and if it does what its mechanism is. Because of this many people would be interested in a scientific paper that deals with free will. Mentioning free will in the PR for the paper is click bait unless the paper actually deals with the subject. Instead the paper seems to be about how decisions prepared and executed. The problem is that the study did not involve any measure of if-when-how the intention or the act was felt in the subject’s consciousness. We do not know what the subjects thought.

There are a number of definitions of free will: in religion it is lack of predestination; in philosophy it is lack of material determination (classic dualism); in jurisprudence it is owning the responsibility for an action (not coerced, accidentally or unconsciously done but in involving conscious intent); in neuroscience it has come to mean a decision taken under conscious control (an action that is started or can be stopped by conscious intent) – very similar to the legal meaning. What the last three have in common is control of intent/execution by conscious thought. Volition is a word without any necessary connection to consciousness. Unless an experiment tracks conscious events as well as other events, it has nothing to say about free will. It can have a great deal to say about volition, decision, intention, motor control, action plans etc. etc. but without involving consciousness, it has absolutely nothing to say about free will. As I said above, I have not been able to read the original paper, but if as I suspect it does not measure or time conscious feelings of intent or execution then its PR is misleading.

Synergy and Modular control

When we learned the simple overview of the nervous system in grade school, we were taught that the brain sent signals to muscles to contract and that is how we moved. And by brain, we assumed the thinking part up high in the head. But it cannot be so.

A little deer is born and in a very short time is standing and in a little longer is taking its first wobbly step. Within a couple of days it is running and frolicking. Deer are not that special; other animals ‘learn’ to get around very quickly too. Even humans babies, if they are held upright with their feet touching a surface will walk along that surface. In a sense, the spinal cord knows how to walk by lifting and moving forward alternate legs. It does not know how to walk well, but the basics are there. Human babies are slower at managing to get around because they are born at a less developed stage and walking on two legs rather than four is trickier. In all sorts of observations and experiments there is evidence that the ability to walk is innate in the spinal cord and does not require the brain.

The spinal cord has some primitive control modules or muscle synergies. Muscle synergies are present in a number of natural behaviors; they are low-level control networks found in the brain stem and spinal cord that coordinate a group of muscles. They make common movements easier to order up. We have the ‘intent to go over there’ and without any more conscious thought we do it in an automatic way. Now if we had to trigger individual muscles in the right time sequence, it would likely take many hours to get not very far with a number of falls along the way. One could say that we would ‘get the hang of it’ as we did it. But that is saying we would make parts of it automatic (create modules and synergies).

This modularization of motor control is layered. The simplest control is in the spinal cord, but it is modified and adapted to conditions by the brain stem and especially the cerebellum. The cerebellum gets instructions from other parts of the brain and finally these modules within modules are able to execute the simple ‘intention to go over there’.

The synergies in a baby’s spinal cord are an ancient set that is similar of all mammals (probably all land vertebrates). The muscles work in a rhythm where each event triggers the next in a circle. There are two primitives that are involved in human walking that we are born with. One is to bend the leg so that the foot leaves the ground and moves forward then goes back down and straightens. Two is a forward push against the ground by the straight leg. These two complexes of muscle contractions and relaxations are wired so that their action in one leg inhibits their action in the other. When the left leg does one, the right leg cannot do one but can do two. And when the left leg does two, the right cannot do two but can do one. They are also wired so that in each leg it is the end of one that triggers the start of two and the end of two triggers the start of one. It is the same in four legged animals except there is another set of inhibitions between the front and hind legs. At this level it is not very adaptive and can only react to sensory information that comes through the spinal cord from the muscles, joints and skin. Babies cannot use this facility to get around because they do not have the strength to maintain the posture needed with such a large heavy head on such a little body, and more importantly, the spinal cord has no information from the ears about balance. Balance is very important for bipedal walking. The baby must create two other synergies: to react to balance information and to use the hips, back and arms to keep the center of gravity over the legs. In the meantime, when they don’t have the strength, they can crawl using the 4 legged modules.

The cerebellum and brain stem add the control of balance and of pace (there are relative changes to the timing of events when the whole process is sped up). They can correct for uneven ground. They can keep the direction of motion toward a target. But the coordination control of the lower brain is not just direct signals to muscles but uses the synergies built into the spinal cord. And it is much more complex than the action in the spinal cord. In fact, the cerebellum has more neurons that the whole rest of the brain. It manages the modules, timing, adjustments to modules, effects from sensory input and feedback and commands from higher levels of the brain, then packages it all for execution. Another great trick of the cerebellum is to do two things at the same time, say walk and throw a ball. Both may be deep seated modules but there are adjustment to be made where they interfere with one another.

The point I am making here is that although movement seems so easy for us to execute, that is because it is not arranged consciously, or even largely in the cerebral hemispheres. It is modularized so that a simple request in the cerebral cortex goes through layers of calculation and fine-tuning to become individual signals to individual muscles. It is synergy/modularization that gives us this flexible but easy to use system. We are surprised that it is easier to create a program to play chess in the abstract (and win) than it is to program a robot to physically move the pieces and operate the time clock in a game. When we do not understand how something is done, it appears easy. It is a common trap.


Metaphors and shapes

Judith Copithorne image

Judith Copithorne image

Metaphors (including analogs and similitudes) appear to be very basic to thought. These are very important to language and communication. A large bulk of dictionary meanings of words are actually old metaphors, that have been used so much and for so long that the words has lost its figurative root and become literal in their meaning. We simply do not recognize that it was once a metaphor. Much of our learning is metaphorical. We understand one complex idea by noticing its similarity to another complex idea that we already understand. For example, electricity is not easy to understand at first but we have learned to understand a great deal about how water flows as we have grown up by watching it. Basic electrical theory is often taught by comparing it to water. By and large, when we examine our knowledge of the world, we find it is rife with metaphors. We can trace many ways we think about things and events to ‘grounding’ in experiences of infants. The way babies establish movement and sensory information is the foundation of enormous trees and pyramids of metaphorical understanding.

But what is a metaphor? We can think of it as a number of entities that are related in some way (in space, in time, in cause-effect, or in logic etc.) to form a structure that we can understand and think of/ remember/ name/ use as a predictive model and treat as a single thing. This structure can be reused without being reinvented. The entities can be re-labeled and so can the relations between them. So if we know water flowing through a pipe will be limited by a narrower length of pipe we can envisage an electrical current in a wire being limited by a resistor. Nothing needs to be retained in a metaphor but the abstract structure. This facility of being able to manipulate metaphors is important to thinking, learning, communicating. Is there more? Perhaps.

A recent paper (Rolf Inge Godøy, Minho Song, Kristian Nymoen, Mari Romarheim Haugen, Alexander Refsum Jensenius; Exploring Sound-Motion Similarity in Musical Experience; Journal of New Music Research, 2016; 1) talks about the use of a type of metaphor across the senses and movement. Here is the abstract:

People tend to perceive many and also salient similarities between musical sound and body motion in musical experience, as can be seen in countless situations of music performance or listening to music, and as has been documented by a number of studies in the past couple of decades. The so-called motor theory of perception has claimed that these similarity relationships are deeply rooted in human cognitive faculties, and that people perceive and make sense of what they hear by mentally simulating the body motion thought to be involved in the making of sound. In this paper, we survey some basic theories of sound-motion similarity in music, and in particular the motor theory perspective. We also present findings regarding sound-motion similarity in musical performance, in dance, in so-called sound-tracing (the spontaneous body motions people produce in tandem with musical sound), and in sonification, all in view of providing a broad basis for understanding sound-motion similarity in music.”

The part of this paper that I found most interesting was a discussion of abstract ‘shapes’ being shared by various senses and motor actions.

A focus on shapes or objects or gestalts in perception and cognition has particularly concerned so-called morphodynamical theory … morphodynamical theory claims that human perception is a matter of consolidating ephemeral sensory streams (of sound, vision, touch, and so on) into somehow more solid entities in the mind, so that one may recall and virtually re-enact such ephemeral sensations as various kinds of shape images. A focus on shape also facilitates motion similarity judgments and typically encompasses, first of all, motion trajectories (as so-called motion capture data) at various timescales (fast to slow, including quasi-stationary postures) and amplitudes (from large to small, including relative stillness). But shapes can also capture perceptually and affectively highly significant derivatives, such as acceleration and jerk of body motion, in addition.

The authors think of sound objects as occurring in the time range of half a second to five seconds. Sonic objects have pitch and timbre envelopes, rhythmic, melodic and harmonic patterns. In terms of dynamics, sonic objects can: be impulsive with an envelop showing an abrupt onset and then decay, or be sustained with a gradual onset and longer duration, or be iterative with rapidly repeated sound, tremolo, or drum roll. Sonic objects could have pitch that is stable, variable or just noise. These sonic objects are related to similar motion objects – objects in the same time range that produce music or react to it. For example the sonic objects in playing a piano piece or in dancing. They also have envelopes of velocity and so on. This reminds me of the similar emotions that are triggered by similar envelopes of musical sound and speech. Or, the objects that fit with the nonsense words ‘bouba’ and ‘kiki’ being smooth or sharp. ‘Shape’ is a very good description of the vague but strong and real correspondences between objects from different domains. It is probably the root of being able to use adjectives across domains. For example, we can have soft light, soft velvet, soft rustle, soft steps, soft job, and more or less soft anything. Soft describes different things in different domains but, despite the differences, it is a metaphoric connection between domains so that concrete objects can be made by combining a number of individual sensory/motor objects which share abstract characteristics like soft.

In several studies of cross-modal features in music, a common element seems to be the association of shape similarity with sound and motion, and we believe shape cognition can be considered a basic amodal element of human cognition, as has been suggested by the aforementioned morphodynamical theory …. But for the implementation of shape cognition, we believe that body motion is necessary, and hence we locate the basis for amodal shape cognition in so-called motor theory. Motor theory is that which can encompass most (or most relevant) modalities by rendering whatever is perceived (features of sound, textures, motion, postures, scenes and so on) as actively traced shape images.

The word ‘shape’, used to describe corresponding characteristics from different domains, is very like the word ‘structure’ in metaphors and may point to the foundation of our cognition mechanisms, including much more than just the commonplace metaphor.


Fish integrate their senses

Judith Copithorne image

Judith Copithorne image

Consciousness seems to have at its foundation the melding of information from all the senses into a integrated model of the world (and ourselves in it). It would be impossible to meld a sound with a sight, for example, without having a common framework of space and of time. And without the different senses informing one another, they would lose much of their usefulness. Therefore when we see melding of sensory information into a model, we can guess that there is a good probability that some level of consciousness exists. Two recent papers on fish show this sort of hint.

The first paper (Thompson, Vanwalleghem, Heap, Scott; Functional Profiles of Visual-, Auditory-, and Water Flow-Responsive Neurons in the Zebrafish Tectum; Current Biology 2016) shows that the tectum integrates sense information in a similar way to the human superior colliculus. “In order to function efficiently, fish and humans need a unified sensory view of the external world contributed to by multiple senses”, says Ethan Scott.

Using calcium imaging in transparent zebrafish, the dynamics of visual processing were shown to replicate previous studies. When sound or waterflow stimuli were used, a small number of cells in the tectum responded, similarly to the visual response but not showing the same cells. The visual response was somewhat less when other signals were present at the same time. This was similar to processes in the mammalian superior colliculus – information from various senses is integrated there.

The second paper (Schumacher, de Perera, Thenert, von der Emde; Cross-modal object recognition and dynamic weighting of sensory inputs in a fish; Proceedings of the National Academy of Sciences 2016) showed that fish can switch between senses as do monkeys, dophins, rats and humans.

The elephantnose fish explores objects in its surroundings by using its eyes or its electrical sense – sometimes both together. The skin contains numerous sensor organs that perceive objects in the water by means of the changed electrical field. “This is a case of active electrolocation, in principle the same as the active echolocation of bats, which use ultrasound to perceive a three- dimensional image of their environment.” Electrolocation is more useful at close range and vision is better at longer distances. The fish can, in effect, turn off one of the senses if the information from the other sense is more reliable.

Using darkness to force electrolocation and electrically transparent objects to force vision, the researchers could study the switching of the senses. They found that the fish could remember, find and recognize shapes experienced with one sense when using the other sense. They form a model of the space which could be used by either or both senses.

It seems that fish form a model of the their environment that all their senses can contribute to in an integrated way.

To see others as we see ourselves

In psychology there is a theory about the ‘fundamental attribution error’, the error in how we attribute causes to actions. When we look at our own actions, they are caused by our cognition in the circumstances in which we are deciding what to do. When we look at the actions of others, they are caused by their personality or character traits. So we do not really take into consideration the circumstances of others when we judge their actions. Nor do we consider the fixed patterns of our own behavior that do not enter into our conscious thoughts when we judge our own actions. We just do what is reasonable at the time and they just do what they always do. I can be too busy to help while they can be too thoughtless. This is a problem for us but at least we can understand the problem and occasionally overcome it. (My way to deal with it is to just assume that people are intelligent and well-meaning most of the time. If they do something that seems dumb or nasty, I look at the circumstances to see if there is a reasonable explanation. There very often is. I realize that this view of my own behaviour is somewhat ironic in its internal attribution – well nothing is perfect.)

But this problem with attribution is much greater than human social interaction. We do the same thing with animals. Elephants were tested for self recognition with the mirror test. If they recognize a black spot appearing on their forehead then it is clear that they know it is their forehead. Elephants failed the test and so they were said to not have a sense of self. It turned out that the mirrors used were too small. The elephants could not make out that it was an elephant in the mirror let alone themselves. If we start out underestimating an animals intelligence, and either not test that assumption or test it in a way that is inappropriate for the animal – then we are making a big attribution error.

There is an assumption on the part of many that vertebrate brains are quite different in the various sorts of vertebrates. This is not true! All animals with a spine have the same brain pattern with the same regions. All vertebrates have seven parts and no more or less: accessory olfactory bulb; cerebellum; cerebral hemispheres; medulla oblongata; olfactory bulb; optic tectum; and pituitary gland. There are differences in size, details and subdivisions, but there are no missing parts. (R.G. Northcutt; Understanding Vertebrate Brain Evolution; Integr. Comp. Biol. 2002 42(4) 743-756). There is every reason to believe that the brain works in fundamentally the same way in mammals, birds, reptiles, amphibians and fish. And by and large, this same pattern of brain has the same functions – to move, find/eat food, escape enemies and so on. It is obvious that animals have motor control and sensory perception.

What evidence is there that other animals have emotions, memory, or consciousness? Can they be automatons with no mental life? The reports trickle in year after year that add to the evidence that animals have a mental life similar to ours.

Reptiles probably dream. Most animal species sleep, from invertebrates to primates. However, neuroscientists have until now only actively recorded the sleeping brains of birds and mammals. Shein-Idelson et al. now describe the electrophysiological hallmarks of sleep in reptiles. Recordings from the brains of Australian dragons revealed the typical features of slow-wave sleep and rapid eye movement (REM) sleep. These findings indicate that the brainstem circuits responsible for slow-wave and REM sleep are not only very ancient but were already involved in sleep dynamics in reptiles.(Shein-Idelson, Ondracek, Liaw, Reiter, Laurent; Slow waves, sharp waves, ripples, and REM in sleeping dragons; Science 2016 Vol 352 (6285) 590-596) These wave types in sleep also are evidence for a memory system similar to ours.

Fish don’t make noise or wave their fins to show emotion but that does not mean they don’t have emotions. “Whether fishes are sentient beings remains an unresolved and controversial question. Among characteristics thought to reflect a low level of sentience in fishes is an inability to show stress-induced hyperthermia (SIH), a transient rise in body temperature shown in response to a variety of stressors. This is a real fever response, so is often referred to as ‘emotional fever’. It has been suggested that the capacity for emotional fever evolved only in amniotes (mammals, birds and reptiles), in association with the evolution of consciousness in these groups. According to this view, lack of emotional fever in fishes reflects a lack of consciousness. We report here on a study in which six zebrafish groups with access to a temperature gradient were either left as undisturbed controls or subjected to a short period of confinement. The results were striking: compared to controls, stressed zebrafish spent significantly more time at higher temperatures, achieving an estimated rise in body temperature of about 2–48C. Thus, zebrafish clearly have the capacity to show emotional fever. While the link between emotion and consciousness is still debated, this finding removes a key argument for lack of consciousness in fishes.” (Rey, Huntingford, Boltana, Vargas, Knowles, Mackenzie; Fish can show emotional fever: stress-induced hyperthermia in zebrafish; 2015 Proc. R. Soc. B 282: 20152266)

One of the problems with comparing the brains of different vertebrates is that they have been named differently. When development is followed through the embryos, many differently named regions should really have a single name. Parts of the tectum are the same as our superior colliculus and they have been found to act in the same way. They integrate sensory stimuli from various senses. They can register whether events are simultaneous. For example in tadpoles the tectum can tell if a sight and vibration stimulus are simultaneous. That is the same function with the same development in the same part of the brain in an amphibian and a mammal. (Felch, Khakhalin, Aizenmen; Multisensory integration in the developing tectum is constrained by the balance of excitation and inhibition. 2016 eLife 5)

We should be assuming that other vertebrates think like we do to a large extent – just as we should assume that other people do – and try to understand their actions without an attribution error.