Number 99
Before your very eyes
15 Mar 97
Human vision is a very much more complex process than we
might guess from our everyday experience. Researchers are
now beginning to understand how parallel processes in the brain
recreate the visual world
FOR most of us, the ability to see and interpret our surroundings is
such an automatic part of everyday experience that we seldom pause
to consider what an astonishing feat of analysis it is. We perceive
colour, form and motion without even thinking about it. Yet computer
scientists have struggled to develop robot vision that can deal with
anything like the level of detail available to us in any visual field-the
window on the world in front of our eyes. We instantly recognise
familiar objects and quickly categorise unfamiliar ones. We recognise a
chair as a chair whatever angle we see it from, even if it is upside
down or partly hidden by a table. The subtle differences between
human faces are enough for us to identify them instantly as those of
particular individuals-even if we cannot always put a name to them.
More astonishing still, we are able to appreciate the many facets of
an intricate visual image simultaneously.
How do we manage to see the world around us in all its complexity?
This question has kept philosophers busy for literally thousands of
years. Yet only in the past century or so have the techniques of
investigative neurobiology begun to reveal how the brain deals with all
the information entering it from the eye. Our understanding has
advanced to a stage that illuminates the workings of visual
pathways, and even the very mechanisms of consciousness itself.
At the beginning of the present century the Spanish anatomist
Santiago Ramon y Cajal and the British physiologist Charles
Sherrington between them laid the foundations for the modern science
of neurobiology. They showed that the secret of the brain's amazing
abilities lies in its connectivity, the millions of interconnections
between different groups of interacting nerve cells. So the best way
to understand vision is to follow the pathways that visual information
takes from the eye to the brain.
All visual information reaches us in the form of light at wavelengths
from the visible part of the spectrum (about 300 to 700 nanometres),
which is reflected from objects in the world around us. It enters the
eye through the transparent window of the cornea and is focused by
the lens, forming an image on the retina. This image is upside down,
like the image in a pinhole camera: the top half of the retina receives
light from the bottom half of the visual field, and vice versa. Similarly,
the left side of each retina receives light from the right visual field,
while the right side receives light from the left field.
This has an interesting consequence for the route the visual
pathways take to the main visual centres in the brain. The cerebral
cortex-the outer layer of brain tissue where most of the nerve cells
are found and most of the information processing takes place-has two
halves or hemispheres, each dealing with information from the
opposite side of the body. For example, sensory information and motor
instructions relating to the right side of your body are dealt with by
the cortex on the left side of your brain. This is also true with vision.
The left visual cortex, situated at the back of the brain, processes
information from the right visual field. In the left eye, this information
falls on the left side of the retina. Long fibres called axons, which
come from nerve cells in this part of the retina, enter the optic nerve
and pass on to waystations on the same side of the brain. But in the
right eye, the axons of nerve cells on the left side of the retina must
cross over in a structure called the optic chiasm, so that they also
reach the left side of the brain. In this way information from both
eyes relating to the same part of the visual field reaches the same
part of the brain.
On the way to the cortex, the new grouping of axons transmitting
information from the visual field of the opposite side passes in the
optic tract to the lateral geniculate nucleus. This highly organised
structure in the midbrain has six layers. The axons from each eye
terminate in separate layers, three for each eye. The separate inputs
from the two eyes are not combined until they reach the cortex.
Axons from the nerve cells of the lateral geniculate nucleus leave in a
bundle called the optic radiation and terminate in connections with
the visual cortex at the back of the brain.
Image analysis
Feature by feature
The analysis of the massive amount of
information in a visual image begins in the eye
itself. The retina of each eye contains 126 million
photoreceptor cells, but only one million axons
leave the retina in the optic nerve. These axons
carry the output of the retinal ganglion cells,
each of which integrates the responses of
photoreceptors on a small patch of the retina
known as its receptive field. The way these
receptive fields are organised is critical to the
retina's preliminary analysis of form, colour and
movement.
Each receptive field is a circular patch of retina, and the ganglion cell
responds differently depending on whether light falls on the centre of
the circle or on the surrounding area. "On-centre, off-surround" cells,
as their name suggests, increase their activity if light falls on the
centre of their receptive field, but decrease it if the light is in the
outer ring. Other ganglion cells have the opposite response. The
combined efforts of the ganglion cells send a map of the visual field
to the brain that highlights areas where there are changes in the
levels of illumination, such as the edges of objects. The map also
includes information about colour: a proportion of the ganglion cells
integrate inputs from the three types of cones in the retina,
photoreceptors that are sensitive to blue, green or red light.
Once the retina has recorded where everything is in the visual world,
the rest of the visual system uses that map as it conducts an even
more detailed analysis. Cells dealing with adjacent parts of the visual
field also tend to be physically close to one another in the brain. The
lateral geniculate nucleus, for example, more or less faithfully repeats
the map created by the ganglion cells.
The cortical area where the optic radiation terminates is called the
primary visual cortex, or V1. It has a distinctive line running
through it when viewed under a microscope, so it is also known as the
striate cortex. The surrounding cortical areas are called the
prestriate cortex or "association cortex". Until very recently,
scientists assumed that the primary visual cortex carried out most of
the analysis of visual information, and then passed the result on to
the association areas. Here visual images would be "associated" with
previous visual memories, as well as with input from other senses,
eventually giving rise to conscious perceptions.
But this rather simple scheme turns out to be seriously flawed. Often
in science the most dangerous and misleading assumptions are those
which are not even recognised as such. In the case of vision, we all
have a very strong subjective feeling of what it is like to see. When
we look at a scene we instantly see all its visual attributes-colour,
form, texture, motion and so on. It seems perfectly natural to
suppose that all these facets are analysed together in one area of the
brain. And the obvious candidate for this area is the striate cortex,
the first cortical area to receive all the information coming from the
eyes. The very precise point-to-point mapping of the visual fields in
this area seems to lend weight to this idea.
Yet it turns out that our unitary visual experience is not an
appropriate model of how our brains actually work. In recent years
neurobiologists have provided impressive evidence that the different
attributes of a visual image are in fact analysed in different areas of
the brain. Much of what was vaguely thought to be association cortex
plays a much more fundamental role in the analysis of form, motion
and colour. In a few short years, some carefully gathered
experimental data have rendered obsolete centuries of philosophising.
And it has neatly explained some of the strange effects experienced
by patients who have suffered strokes affecting the visual cortex
(see Box 1).
Early attempts to define the areas of the cortex that were specialised
for different functions were focused largely on the arrangement of the
layers of cells visible under the microscope. Because much of the
visual cortex had a fairly uniform structure, neuroscientists assumed
that its function was also uniform. But modern methods that enable
neuroscientists to record the responses of living cells and trace their
connections have revealed subdivisions within the visual cortex.
Some of the most exciting research in this area is carried out on
humans using positron emission tomography (PET), a scanning
technique that reveals changes in local blood flow in the living brain.
An advantage of these experiments is that humans can report their
subjective experiences at the same time as researchers collect the
experimental data.
Visual cortex
Making pictures
Of all the visual areas, V1 contains the most
detailed point-to-point map of the retina. Its
cells are organised into a stunningly complex
system of distinct modules. Alternating columns
of cells show a preference for responding to
stimuli coming from one eye or the other. These
ocular dominance columns are further
subdivided in a regular manner into columns of
orientation-selective cells, which respond to
an edge or bar in their receptive fields only when
it is held at a particular orientation. All the cells
in one column respond to one orientation, cells in
the adjacent column respond to an orientation a
few degrees off from the first, and so on until all
possibilities are covered. There are other
groupings of cells within the ocular dominance
columns which are not orientation selective, but
instead show a tendency to respond to light at particular
wavelengths. In this way V1 preserves the segregation of form and
colour that begins in the retina.
But V1 carries out a more elaborate analysis on these data. It
contains further groupings of cells that respond only to a stimulus
that is not just of the correct orientation, but also moving in a
particular direction. And its orientation-selective cells are sensitive
not only to real boundaries, but to illusory ones (Box 2), created when
the cortex begins to reconstruct a mental world of objects from the
patterns of light and dark transmitted from the retina. The intricate
detail of V1 helps to explain why it is the largest visual area, since
it
scans the field of view for all the features of the visual scene, which
are represented within it in a multiple series of overlapping maps.
Research on laboratory monkeys over the past couple of decades has
revealed several further distinct visual areas, labelled V2, V3, V4 and
so on. Techniques such as PET scanning in humans indicate that we
have separate specialised visual areas connected to V1, which are
similar, but not always identical, to those in monkey brain.
Most of V1's output goes to an area immediately surrounding it called
V2, where there is a similar series of overlapping maps representing all
the visual features. As well as cells sensitive to colour, motion and
orientation, V2 contains cells that are sensitive to disparity-the
slightly different view from each of the two eyes that is the basis of
stereoscopic vision. V1 and V2 have intricate connections with each
other and with other more specialised visual areas.
V4 specialises in the perception of colour. This is not nearly as
straightforward a task as it might seem. The cones in the retina
respond to light at different wavelengths-but there is no
straightforward relationship between wavelength and the colour we
perceive. The wavelengths of light reflected from an object vary
enormously according to lighting conditions. Yet the leaves on trees
still appear green whether at dawn or dusk, in the midday sun or in
the darkening sky of an approaching storm. Achieving this colour
constancy is one of the main jobs of V4, and it does it by comparing
the wavelengths reflected by groups of adjacent objects with their
overall brightness.
V5's function is to analyse motion, while V3 is concerned with the
analysis of form and depth-how far away an object is. Even more
complex, the recently-discovered area V6 appears to be responsible
for analysing the absolute position of an object in space. This is what
makes you aware that a magazine in front of you stays in the same
place even when you turn to look at someone coming into the room.
As information passes from one visual area to the next, cells become
less concerned with where an object is than with what it is. V1 cells
will respond only to objects in a small section of the visual field. But
cells in the more specialist areas tend to have much larger receptive
fields. Some respond to certain categories of object regardless of
where their images appear on the retina. The old idea that visual
information passed up a rigid hierarchy of cells until it reached a single
cell that would respond only to a specific image, such as that of your
grandmother, has long been discredited. But what does seem possible
is that there are visual areas that encode information about complex
objects, including faces, in relatively small networks of perhaps 100 or
so cells.
Our perception results from selection and synthesis of available
information-we do not record things simply like a video camera does.
What we see depends largely on our past experience of the way the
visual world is organised, a fact which forms the basis of many
visual illusions.
Seeing and knowing
Conscious experience
This new understanding of the way in which the
brain handles visual information has profound
implications for our understanding of
consciousness, that most mysterious and elusive
property of our minds. Older ideas suggested that
the primary analysis of visual information
happened in the striate cortex, which then fed
this information forward to be associated with
information from other senses. The implication
appeared to be that the associated information
would then be fed forward somewhere else until
eventually a place was reached where
perception and consciousness would be
generated.
Instead we now know that different parts of our
awareness-from colour to the expression on a
person's face-are generated simultaneously in different specialised
cortical areas. And if one of the specialised areas is damaged we lose
the relevant perception, causing strange alterations of
consciousness such as the awareness of colour without form, or the
ability to see form but not motion (Box 1). This indicates that all parts
of our cortex contribute directly to consciousness, which is the result
of ongoing activity in many intimately connected, specialised cortical
areas. Indeed, the interconnections are so complex that their
description and analysis will provide plenty of work for several more
generations of neuroscientists.
* * * * *
1: Missing parts of the picture
OF all the organs in the body, the brain is the most critically
dependent on its blood supply. Interruption of the flow of blood for
even a few minutes causes irreversible damage to the region of
brain affected-a stroke. Since different regions of the brain are
specialised for performing specific functions, strokes vary in their
effects depending on which part of the brain is involved. For
example, damage to the motor area of the cortex causes paralysis,
while damage to the primary visual cortex causes blindness.
Strokes often affect quite large areas of the brain, with widespread
and tragic consequences for the patient. But the problems caused
by smaller areas of damage have provided neurologists with some
remarkable insights into the way the human brain handles
information and controls behaviour. Particularly strange things can
happen when strokes affect the visual areas. Damage to the
primary visual cortex produces a complete blind spot in the
opposite visual field. But localised damage to the more specialised
areas can disturb some aspects of vision while leaving others
intact.
Louis Verrey, a Swiss ophthalmologist, described in 1888 the case
of a 60-year-old woman who suffered a stroke affecting the visual
cortex of the left cerebral hemisphere. As a result, she could no
longer see the world in colour in the right half of her field of view.
Instead, everything she saw in that half appeared in shades of
grey. Although strokes with this effect are quite rare, they have
been described many times, providing strong evidence that colour
is analysed separately from the other elements of a visual scene.
This cortical colour blindness or achromatopsia is quite different
from the common type of colour blindness which affects the whole
visual field, and is due to an abnormality of the
wavelength-sensitive light receptors in the retina.
Even stranger cases include a 43-year-old woman who suffered a
stroke, and found that she could no longer see objects which were
in motion, though stationary objects presented no problem. This
caused considerable difficulty. For example, she found it hard to
pour a cup of tea because the moving liquid appeared frozen like a
glacier, and she could not stop pouring at the right time because
she could not see the cup filling up. She also had problems crossing
roads: a car would seem to be far away, then suddenly would be
very near as she went to cross. This cortical motion blindness is
strong evidence that motion is also analysed in its own special
area.
The type of visual disturbance made famous by the neurologist
Oliver Sacks in his book The Man Who Mistook His Wife for a Hat
goes by the splendid, if rather tongue-twisting name of
prosopagnosia. These patients suffer an inability to recognise
familiar faces, including their own. They understand what a face is
and can see various features, such as the eyes, nose and mouth,
but they just cannot recognise it as a particular face. Even more
bizarre is the fact that some patients with this condition, unable to
identify anyone from their face, nevertheless retain the ability to
recognise the expression on a face, indicating that there is another
cortical area that specialises in the analysis of facial expressions.
More extraordinary still is the phenomenon of blindsight. Some
people with damage to the primary visual cortex, who deny being
able to see anything in the part of the visual field affected, can
still make correct judgments about the position, wavelength, or
direction of movement of objects in the blind spot if forced to do
so . One patient, for example, could follow a moving striped object
with his eyes, even though he said he could not see it. The most
likely explanation is that the information is coming from surviving
visual pathways beneath the cortex, such as the lateral
geniculate nucleus but, because it does not reach the cortex, it is
not available to conscious awareness.
* * * * *
2: There's more to vision than meets the eye
FROM early in this century psychologists have been fascinated by
the phenomenon of visual illusions, which give a powerful sense of
a reality that simply is not there. Some of these, such as the
illusion of movement you experience when you look through the
window of a train that has stopped at a station, are simply the
result of adaptation to prolonged stimulation in part of the
system. But others occur because the brain is trying, on the basis
of its past experience of the visual world, to come up with a "best
guess" about what is really there.
Most of the time this does not cause any problems, because the
most likely interpretation is probably the right one. Ambiguous
figures ( Figure a) are an exception. The brain can decide that the
figure is a vase, or it can decide that it is two faces; but it cannot
see both interpretations at once, and tends to alternate between
one and the other.
Other illusions arise because certain features
in the visual environment provide such strong
clues about the position of objects in a scene
that it becomes impossible to ignore them.
Painters make use of the fact that light falling
on objects causes shadows to create the
illusion of depth in their pictures. Converging
lines also suggest distance, and many simple
optical illusions use this device to fool you into
making false judments about the size or shape
of objects ( Figure b).
A third example is that of illusory contours,
where the brain extends partial outlines to
create shapes that do not exist (Figure c).The
striking feature here is the powerful sense
that the illusory triangle glows with more
brightness than the surrounding white space.
Making a reasonable hypothesis about the
elements of the scene, the brain decides that
there is a bright triangle slightly in front of the
other objects.
Further reading:
A Vision of the Brain by Semir Zeki (Blackwell Scientific,
1993), an excellent account of the neurological basis of
vision.
An Introduction to the Visual System by Martin TovÉe
(Cambridge University Press, 1996), an up-to-date textbook.
John Lee is senior lecturer in pathology at the University of
Sheffield.
From New Scientist magazine, vol 153 issue 2073, 15/03/1997, page
© Copyright New Scientist, RBI Limited 2001