Experience is not something we feel but something we do: a principled way of explaining sensory phenomenology, with Change Blindness and other empirical consequences.
J. Kevin O'Regan
Laboratoire de Psychologie Experimentale,
Centre National de Recherche Scientifique, Paris, France
and
Alva Noe
Department of Philosophy
University of California, Santa Cruz
Talk given at the ASSC Conference:
THE UNITY OF CONSCIOUSNESS: BINDING, INTEGRATION, AND DISSOCIATION
Brussels, June 29-July 2, 2000
ABSTRACT
Any theory of experience which postulates that brain mechanisms generate "raw feel" encounters the impassable "explanatory gap" separating physics from phenomenology.
A way round the problem is to postulate that experience is not something we feel, but something we do: a kind of give-and-take with the environment, analogous to the "feel" of driving a car. One consequence of such a "sensorimotor" theory of experience is that it provides a way of explaining the differences between seeing, hearing, touch, etc., which is more principled and has more explanatory power than Muller's notion of "specific nerve energy" or its modern counterpart, the notion of sensory pathways or cortical areas. The feasibility of sensory substitution is an empirically verifiable implication of this approach.
As applied to visual perception, a consequence of the sensorimotor approach is the idea that seeing does not consist in the creation of a "re-"presentation of the world inside the brain, but rather in knowledge that the outside world is immediately accessible through a flick of the eye or of attention, like an "outside memory". The world-as-an-outside-memory idea has empirically verifiable consequences in the phenomenon of Change Blindness, among others.
CONTACT:
J. Kevin O'Regan
Laboratoire de Psychologie Experimentale,
Institut de Psychologie, Centre Universitaire de Boulogne
71, avenue Edouard Vaillant
92774 Boulogne-Billancourt Cedex
oregan@ext.jussieu.fr
http://nivea.psycho.univ-paris5.fr
Tel: (33 1) 55 20 59 26
KEYWORDS: Qualia, visual sensation, visual perception, change blindness
If I show you a picture and make a change in it, like this, you immediately see the change. there's no problem, it pops out.
But if I make the change at the same time as I put a slight flicker on the screen, like this, then in many cases you miss the change.
Flicker example: woman in kayak
Here's another example.
It's quite interesting that sometimes you can be looking directly at the change and still not see it. So here if I tell you to look at the man's nose, you'll be within a few pixels of the change, and yet often you wont see it: it's the bar in the background going up and down.
Here's another example
Some changes are easier to see than others. But even a change which occupies a very large portion of the visual field may not be noticed if it's not part of what the picture would be said to be about. Here for example,
you would not normally say that the picture is about ..... the reflection in the lake. But as soon as I tell you that, you see the change. Whereas here,
you probably consider the picture to be about a glass of milk, so its easy for you to see that that's what's changing, even though it's much smaller than the reflection in the lake was.
[Diaposit...+]
This phenomenon has been called change blindness, and has attracted quite a lot of attention over the last few years. There are a number of variations of the phenomenon. I've shown you the flicker paradigm, but the phenomenon has also been obtained with eye saccades, blinks, film cuts, and even in real-life situations.
A particularly interesting variant of the CB paradigm is the mudsplash paradigm.
The reason this is interesting is that the mudsplashes are positioned in such a way so as to not cover the change location. That way it can't be argued that the reason you miss the change is that it is somehow masked or wiped out by any kind of superposition with flicker.
It's clear that there is something very shocking about all these experiments: they seem to suggest that our internal representations of the outside world, instead of being very detailed and rich...
[Diaposit...+]
are actually rather sparse...So what's going on here? How can we have the impression of richness in the world if there is no richness in the head?
[Diaposit...+]
I suggest that a possible solution is the idea of what I call the World as Outside Memory. The idea is that to get the impression of richness, there's actually no need for the richness to be in the head. What has to be in the head is merely algorithms or recipes for getting at the information in the world.
[Diaposit...+]
Such algorithms we have, in the form of movements of the eyes or shifts of attention. If we're interested in some detail of the visual scence, we simply need to move our eyes or our attention to that detail, and it is immediately available.
Instead of storing all the information about the outside world in the brain, we use the outside world as an external memory storage. Thus, we get the impression that we're seeing everything there is to see in the visual field, because if we so much as faintly wonder whether we're actually seeing something, we turn our eye (and our attention) to that thing, and it becomes available for processing.
Perhaps you've played the children's game in which someone puts a household object like a cork, or a potato, or a harmonica into a bag, and you put your hand into the bag and try to figure out what the object is.
[Diaposit...+]
At first you feel this or that texture on the tips of your fingers. You have no idea what the object is. But suddenly you have a kind of "Aha" experience. Suddenly you feel you're no longer touching bits of texture on the ends of your fingers, but you're holding a whole object: it's a harmonica. And it's ALL there at once, even though you're in fact only touching a few parts of it. It's not just that you know it's a whole harmonica, you actually feel it's a whole harmonica
The reason you have the feeling of touching the whole harmonica is that you KNOW that IF you were to move your fingers this way, then you would get THIS feeling, and if you moved them that way, you would get THAT feeling. You feel familiar with all the things you can do with your fingers at this moment. It is the knowledge of feeling at home with the possible things you can do, and with the resulting changes in the feelings on your fingertips, that gives you the impression of having the whole harmonica in your hand.
Extended to the domain of vision, this analogy suggests how it could be possible to have the distinct feeling of seeing whole objects and whole scenes, even though only a minute part of the scene is actually available for visual processing at any moment. The impression of seeing everything there is to see in the visual field is therefore a sort of illusion...
It's a bit like the light in the refrigerator:
[Diaposit...+]
The light always seems to be on: you open the door, the light is on. You close it. You quickly open the door again to check: yup, the light is still on. You have the illusion that the light is on all the time, but you have to keep checking to convince yourself.
Let me go on to show you some of the consequences of this way of thinking for scene perception.
[Diaposit...+]
This picture shows the path that the eye of one observer took while searching for large changes which occurred every time he blinked.
Here is an example of a scan path for another picture.
[Diaposit...+]
This scan path corresponds to a few seconds of exploration, and you would think that since the observer was actively looking for a change, he would look around in the picture in a rather systematic way, covering all the picture elements. But if you look at what the observer did over the next few tens of seconds, this is what you find.
[Diaposit...+]
It looks like the observer is just going round and round in circles. In fact this kind of behavior is typical of what people do when they look at pictures. Only a fairly limited number of positions are directly fixated by the eyes, and they are fixated repetitively. Why is this?
Under the WOM point of view I'm sketching out here, this can be explained. It could be that seeing a picture is not: accumulating information into an internal representation, but rather: checking that you have access to the things that the picture is about. If you think the picture is about a couple having dinner, then seeing the picture involves making sure that those things that you think the picture is about, are really there. The eye will therefore go round and round checking.
Ongoingness/Continuousness
[Diaposit...+]
In summary up to now, I've suggested the possibility that the feeling we get of seeing everything in the visual field doesn't require us having an internal representation of everything. It suffices to have immediate access to the information in the external world, which acts somewhat like an external memory store.
But you might object that there's still a problem. Consider again the comparison with the refrigerator light. When I look at the world, I have quite a different impression than I do with respect to the refrigerator light:
I have to keep opening the refrigerator door surrepticiously, and still there remains a bit of doubt about whether the light really stays on all the time. I think it's on continuously, but I dont see it on continously. This differs from the real world, which I have the impression of seeing in an "ongoing" or continuous fashion. Why is that?
I think the answer is to do with two things. I call one "bodiliness", and the other "grabbiness".
[Diaposit...+]
Bodiliness is the fact that in vision, the things you do in order to get information are very closely linked to minute and even unconscious bodily actions: The slightest twitch of an eye muscle allows you to change from one point of the scene to another. A small head movement or body movement modifies what you see.
By virtue of this bodiliness, the outside world is intimately linked to you, almost as though it was part of your own body. I suggest that this makes seeing more real, more felt and ongoing than the refrigerator light, which doesn't budge when you move around.
Now let me talk about grabbiness.
We all know that if there's a sudden flicker in the visual field, we can't help but immediately look at it. This is because there exist mechanisms in the first stages of the visual system designed to detect fast transitions in local luminance, and which incontrovertibly grabs your attention. Movement detectors are examples of such transient detectors.
It could be that this grabbiness of sudden events constitutes a second factor which contributes to the feeling of continual presence, and ongoingness of visual stimulation. Grabbiness makes it seem like we have continual "tabs" on everything that's going on in the visual field, and gives us the illusion of seeing things continuously, because if anything should change we're immediately informed.
Developments
Let me now stand back and make some remarks about the way of thinking that I've sketched here.
[Diaposit...+]
Under the standard view, seeing consists in making an internal representation of the outside world. Under the new view, seeing consists in knowing different things you can do, and knowing the changes that these things will produce in your sensory input.
Though unnatural at first sight, the new view has an interesting advantage:
In neuroscience today, one of the problems people are grappling with is to try to understand how a physical entity like a brain can give rise to something like the feeling of seeing, which is patently not physical.
[Diaposit...+]
Some as yet unknown mysterious, possibly even nonphysical mechanism has to be postulated to instill experience into the brain. But under the new view, the problem disappears, because experience is not in the brain at all.
[Diaposit...+]
It's in the doing of the exploration, and in the knowledge of the things that will change as you explore. Instead of the role of the brain being to generate the experience of seeing, the role of the brain simply becomes that of generating the exploratory activity which underlies the seeing, and that of holding the knowledge of current possibilities for action that underlies seeing.
Thus, the problem of finding a mechanism to generate experience in the brain disappears.
Still, there is an objection that may be disturbing you. You could say, ok, seeing is a thing we do... I see the red cup when I'm going about checking I have access to it by the flicks of my eye movements. But what about when I finally am actually looking directly at the red cup. I now have red stimulation on my retina. Now surely there must be something that takes that takes red stimulation on my retina and causes me to experience the redness. We seem to be back to the situation of having to explain how brain activation can engender experience.
But it seems to me that it's possible to escape from the difficulty even for the raw sensation of redness.
[Diaposit...+]
Consider looking at a piece of red paper. Depending on whether you turn the paper so that it's yellowish sunlight or bluish skylight, or reddish incandescent light that is reflected off the paper, the spectrum of light being sensed by the eye is quite different. I suggest you see the paper as red when the laws that are obeyed by the changes in incoming spectrum are typical of redness. Thus, red is not a pattern of excitation caused by incoming light, but knowledge about the laws that the excitation obeys when you move the paper around.
[Diaposit...+]
Another fact about red has to do with the way the eye samples color. At the center of the retina, color information is readily available, being sampled by retinal cones sensitive to long, medium and short-wavelength light, symbolized by colored dots in the slide. But the density of the cones falls off quite rapidly, so that the nature of the neural stimulation that arises from looking straight at a red surface is quite different from that obtained by looking at the surface in peripheral vision. here, there are many more rod photoreceptors, symbolized by black dots, not sensitive to different colors. I suggest that the quality of red is NOT just the particular combination of long, medium and short wavelength stimulation, but also the way in which the stimulation changes as you move your eye on and off the red object.
[Diaposit...+]
Like the experience of seeing everything, the experience of seeing red then is also a kind of knowledge: knowledge that the appropriate contingencies between sensory input and motor actions are currently applicable.
[Diaposit...+]
The notion of sensorimotor contingency can be generalized to cover not just the sensation of red, but, I suspe
