Cooperation of sight and sound

As a child you were probably taught to tell how far away lightening was. When there is a flash, you count with a particular rhythm until you hear the thunder and that is how many miles the lightening is away from you. Parents are not going to stop teaching this because it is something for a nervous child to do in a thunder storm and it convinces them that they are usually a safe distance from danger. But it only works for distant events.

Events that are close by are synchronized by the brain and consciously we collapse the vision and hearing clues both for time and space to make a single event. We are not conscious of a difference in the timing or of any slight difference in the placing of the event. A particular region of the brain does this aligning - “the superior colliculus, a midbrain region that functions imperatively for integrating auditory and visual signals for attending to and localizing audiovisual stimuli”. But if the difference is too large between the vision and hearing, the collapse into a single event does not happen.

However, we know that, even though it is not consciously experienced, the information about small differences in sound arrival can be used by blind humans to echo-locate by making continuous little clicking noises. Could it be that the discrepancy between sound and sight could be used in other ways? A recent paper (Jaekl P, Seidlitz J, Harris LR, Tadin D (2015) Audiovisual Delay as a Novel Cue to Visual Distance. PLoS ONE 10(10): e0141125. doi:10.1371/journal.pone.0141125) studies the effect of sound delays on the perception of distance. Like the lightening calculation, but it is done unconsciously.

Here is the abstract:

For audiovisual sensory events, sound arrives with a delay relative to light that increases with event distance. It is unknown, however, whether humans can use these ubiquitous sound delays as an information source for distance computation. Here, we tested the hypothesis that audiovisual delays can both bias and improve human perceptual distance discrimination, such that visual stimuli paired with auditory delays are perceived as more distant and are thereby an ordinal distance cue. In two experiments, participants judged the relative distance of two repetitively displayed three-dimensional dot clusters, both presented with sounds of varying delays. In the first experiment, dot clusters presented with a sound delay were judged to be more distant than dot clusters paired with equivalent sound leads. In the second experiment, we confirmed that the presence of a sound delay was sufficient to cause stimuli to appear as more distant. Additionally, we found that ecologically congruent pairing of more distant events with a sound delay resulted in an increase in the precision of distance judgments. A control experiment determined that the sound delay duration influencing these distance judgments was not detectable, thereby eliminating decision-level influence. In sum, we present evidence that audiovisual delays can be an ordinal cue to visual distance.

Leave a Reply

Your email address will not be published. Required fields are marked *