Qualia
--The Bridge between Mind and Brain--
Ken Mogi
This essay was written while I was at
Physiological Laboratory
University of Cambridge
Downing Street
CB2 3EG
U.K.
km10009@cus.cam.ac.uk
I am now at
Sony Computer Science Laboratories
Takanawa Muse Bldg.
3-14-13, Higashigotanda
Shinagawa-ku, Tokyo, 141-0022 Japan
Tel +81-3-5448-4380
Fax +81-3-5448-4273
kenmogi@csl.sony.co.jp
http://www.csl.sony.co.jp/person/kenmogi.html (Office)
http://www.qualia-manifesto.com (The Qualia Movement)
(March 1997)
The concept of qualia describes the unique properties that accompany our senses. It is an essential concept when we try to understand the principle that bridges the neural firings in our brain and our perception. The idea of qualia is also of crucial importance when we try to study the functions of the brain from an objective point of view. Qualia must be part of the mathematical formulation of information we use to understand the function of the brain.
Qualia
We perceive the world through our senses. If you go out in the garden on a summer morning and look up into the sky, the blackish blue color comes into your mind. When you see morning dew on the leaves, you become aware of their round shape and the sheen on their surface. Putting your finger on the dew, you feel a sensation of coolness on the skin. Sniff the leaves, and you smell the distinctive scent of the plant. Then you realize that all the while, you have been bombarded with the auditory sensation of the sparrow singing on the roof...The characteristic qualities that accompany these sensations have been called "qualia". It is a patent fact that we perceive the world through a collection of qualia. Any attempt to understand the relation between the mind and the brain would be impossible without incorporating qualia in its framework.
On the other hand, we find the brain to be a tremendously complex and well-made molecular machine. The action potential consists of the inward flow of Na+ ion through the membrane. At the synapses, neurotransmitters such as glutamate and GABA are released into the synaptic cleft. The substances produced near the nucleus are transported to the synapses on the track of microtubules that extend through the axon. The gene regulation in the nucleus plays an essential role in the functional specification of the neurons, and is believed to play a role in the formation of long term memory. An orchestrated system of biomolecules of unimaginable complexity underlies the neural activity in the brain.
The various sensations that occur in our mind are accompanied by various qualia. In contrast, the brain is a complex molecular machine. How are these two worlds related to each other? What is the neural correlate of qualia? This is the central question of the so-called mind-body problem.
The question of qualia is not a solely philosophical one. The origin of qualia, ultimately, should be regarded as part of the natural law, just as the laws of physics or chemistry. Even from the functionalist's point of view, qualia would prove essential in understanding the principle of information processing in the brain, as I argue below. For example, one cannot hope to construct a computer that thinks like a human unless we implement qualia. Qualia should be an integral part of the concept of information when we try to describe the function of the brain.
Despite the crucial importance in understanding the brain, the enigma of qualia is a deep one. How profoundly difficult the problem of qualia is testified by the words of Francis Crick, in his book The Astonishing Hypothesis.
The reader might accept all this but could well complain that I have talked all around the topic of consciousness, with more speculation than hard facts, and have avoided what, in the long run, is the most puzzling problem of all. I have said almost nothing about qualia--the redness of red--except to brush it to one side and hope for the best.
The quest for a solution of the enigma of qualia, the most difficult and important scientific puzzle that confronts humanity, is just beginning.
The neuron doctrine in perception
One of the central questions that concerns us is to understand how our perception is related to the neural firings in the brain. In 1972, Horace Barlow of Cambridge University applied the neuron doctrine to the problem of perception. Barlow suggested that the characteristics of our perception is specified by the nature of neural firings in our brain only. That is to say, no matter what mechanisms are involved, the properties of our mind is to be determined by the neural firing state only. This seems to be a reasonably sound starting point, judging from the experimental clues available today.
"Response selectivity" is a concept of central importance in neurophysiology. For example, in the primary visual cortex (V1), we find neurons that selectively responds to a bar with a certain orientation. In areas MT, V4, IT, we find neurons that respond to motion, color (in the context of "color constancy"), and form. As we go to the higher visual area, we find neurons with more complex response selectivities, and larger receptive fields. The objective of the single unit recording study can be defined as the determination of the response selectivity of a particular neuron.
One idea that emerges is to assume that when a neuron with a response selectivity to a particular visual feature fires, the perception of that feature occurs. For example, when a neuron selectively responsive to a bar slanted by 45 degrees to the right fires, the perception of the slanted bar would be incurred. When a neuron selectively responsive to "face" fires in area IT, the perception of a "face" is invoked, and so on.
Unfortunately, there is a fundamental flaw in this line of argument, which becomes apparent when one tries to answer the following question. "When a neuron selectively responsive to a feature A fires, how does the brain (or the subject) know that it is selectively responsive to feature A?" The visual feature space is vast and complex. The fact that a neuron fires vigorously to a particular feature A does not necessarily mean that the neuron has a response selectivity to feature A only. In fact, in order to confirm the response selectivity of a neuron, every possible visual feature should, in principle, be presented to the neuron. Of course this is impossible. Moreover, our perception is constructed based on the neural firings at a particular psychological moment. (This is what the neuron doctrine dictates). It is impossible, just based on the firing of the neurons at a particular time, to establish the response selectivity of a a particular neuron. The reason for this being that response selectivity is based on the idea of an "ensemble", a set of stimulus-response relations which is at the heart of an statistical approach.
In general, as we go higher up the visual system, the more difficult it becomes to define the response selectivity of a particular neuron in an operational manner. This contradicts the idea that the neurons in the higher visual areas play a crucial role in our perception through their response selectivity. Even in the case where our perception is evoked by the successive neural firings from the lower visual areas to the higher visual areas, the significance of the neurons in higher visual area becomes obscure, if indeed the response selectivity plays an essential role in perception.
From these considerations, I conclude that response selectivity cannot be the foundation for the relation between neural firing and perception. Some other bridging principle(s) should come into the picture.
Mach's principle in Perception
Ernst Mach (1838-1916) was a physicist, philosopher, psychologist who had a great influence on Albert Einstein in his development of the theory of relativity. "Mach's principle" states that the mass of a particle is determined by its relation to all the other particles in the universe. If there was only one particle in the universe, it is meaningless to question how large its mass is. In general, the properties of an individual is determined by its relation to other individuals in the system.
A similar line of thought is relevant when we consider how our perception is formed through the neural firings in our brain. Namely, a neural firing plays a particular role in our perception, not because it is selectively responsive to a visual feature, but because the neural firing is related to other neural firings in the brain in such a way that the particular role in perception is endowed on the neural firing in question. We shall call this idea "Mach's principle in perception".
For example, suppose a neuron in area IT fired, and a perception of "rose" is invoked in our mind. In this case, perception of the "rose" is invoked not because the neuron selectively responds to the presentation of a rose, but because the neural firing in question is endowed with the property of "rose" through its relation to other neural firings in the brain. Specifically, the cluster of neural firings connected by interaction through action potentials that is initiated in area V1 and leads up to the neural firing in area IT codes the perception of "rose". It is meaningless to consider a single neural firing in isolation and assess its significance in perception, even if the response selectivity of that neuron could be established unequivocally.
In this picture, a percept (element of perception) is coded not by a single neural firing, but by a cluster of interaction-connected neural firings. This is the bridging principle that is consistent with the neuron doctrine in perception, and Mach's principle in perception. A percept is coded non-locally. Neural firings in spatially distant areas of the brain are integrated into a cluster through their mutual interactions, and form a percept. The distinction between the excitatory and inhibitory connections now becomes important. Specifically, it appears that only excitatory connections are included explicitly in the cluster of neural firings that forms a percept. Inhibitory connections influence the formation of percepts indirectly. For example, in the color constancy mechanism, inhibitory inputs from surrounds will lead to a non-formation of a percept of a color in the center. Inhibitory connections have significance in that they can "veto" the formation of a percept. However, inhibitory connections are not included explicitly in the cluster of neural firings that forms a percept. In order to see the intuitive meaning of this arrangement, consider a white bar in a black surround. In order that the white bar is a bar, it is necessary that the area surrounding the bar is black, rather than white. If the surround was white, then the bar would not be a bar! So the surround contributes to the formation of a white bar by not being white. However, the black surround does not constitute an explicit part of the percept "white bar".
The above picture gives some hints to the solution of some long-standing problems in visual perception. One of them is the so-called "binding problem". The various visual features, such as form, motion, color, and texture are represented in spatially separate cortical areas. Yet we are able to perceive these features as integrated into one visual object. For example, even though the features "rose", "moving rightward", "red", "velvety" are coded in areas IT, MT, V4, and V2 respectively, we have the integrated perception of "red velvety rose moving rightward". Several solutions, including the ideas "convergence zone" and "synchronous firing" are being proposed. However, none of them are immune from some fundamental difficulties. In order to arrive at a rudimentary solution of the binding problem, we need to go back to the very foundations of the neural correlates of perception. Namely, we need to adopt the idea that a percept is to be represented as a cluster of interaction-connected neural firings, which are spatially distributed among the various cortical areas.
Now suppose the percept "red" is invoked by a cluster of neural firings that extends from area V1 up to area V4. Under such circumstances, the qualia of "redness of red", whatever we may name it, should be specified by the spatio-temporal pattern of the neural firings which belong to the cluster, and by nothing else. This is a direct consequence of our assumption that the nature of our perception is determined by the characteristics of the neural firings that sustain it (the neuron doctrine in perception). Namely, the picture emerges that the correspondence between a particular pattern of cluster of neural firings, and the qualia invoked by it, is given in an a priori manner. Namely, we assume that the correspondence between the neural firing patterns and qualia is one to one, and there is no arbitrariness involved in it. Of course, one could argue that there is no guarantee that the correspondence is one to one. But then there is also no plausible reason to assume otherwise.
This above idea is of a fundamental importance. A natural law in the conventional sense is complete if it succeeds in predicting the spatio-temporal pattern of neural firings in the brain with a full precision. It is yet another type of natural law, which determines what kind of qualia is invoked in our perception once the spatio-temporal pattern of neural firings in the brain is given. As the correspondence between the firing pattern and qualia is one to one, it should be regarded as a kind of natural law, although of a rather different character from the conventional natural laws.
Interaction Simultaneity
Our perception is organized under a certain spatio-temporal order. For example, the space-time structure in which our visual perception occurs is different from that in which our auditory perception occurs. In audition, we don't have the massive parallel processing organized in an spatial order as in the case of vision. The "modality" of senses are marked by differences in the spatio-temporal structures that characterize them. Such differences between the modalities are usually taken for granted, and neural mechanisms of perception is discussed on top of them. That may be effective as a phenomenological and intermediate method of description. However, ultimately, the difference between the sensory modalities should be derived, from the first principles, based on the nature of the spatio-temporal patterns of the neural firings that sustain vision, audition, taste, smell, and touch.
How is our psychological time constructed? Albert Einstein, in his first paper of relativity theory published in 1905, stated
One thing should be remarked here. Such a mathematical description is physically meaningless unless the way we construct time is made clear. All our judgments about time is one about events that occur simultaneously.
What kind of mechanism determines the nature of simultaneity in our mind, namely the nature of psychological "now"?
Let us start from the neuron doctrine in perception. Namely, we assume that knowledge about the firing neurons is necessary and sufficient to determine the content of perception. The concept of "interaction simultaneity" dictates how to determine the nature of psychological time in a way consistent with the neuron doctrine. Under the principle of interaction simultaneity, when a neural firing and another neural firing are connected by interaction (i.e., action potential propagation and subsequent synaptic interaction), these are considered to be simultaneous events. It takes a finite length of time (say 5 milliseconds) for the effect of a neural firing to propagate to a postsynaptic neuron. However, under the principle of interaction simultaneity, these firing events should be regarded as "simultaneous". We call the time parameter thus constructed "proper time" and write it as t. When the presynaptic neuron fires at time t, and the postsynaptic neuron fires at time t+Dt (as a result in part of the EPSP caused by the presynaptic firing), we assign the same proper time t to the pre- and postsynaptic events.
Interaction simultaneity is derived from a more fundamental principle, that of "causality". Here, "causality" is taken to mean that given the state of the system at proper time t, we are able to derive the state of the system at a slightly later time t + Dt. We need to use the proper time t in order to describe the dynamical evolution of the neural network in a causal way.
Interaction simultaneity is based on the idea that if we are to derive the properties of our perception from neural firings, we should not adopt the position where we "observe" the neural firings from "outside the brain". If we observe the brain from outside, we can describe the dynamical evolution of the neural network with any desired temporal accuracy. We may, for example, describe the release and diffusion of neurotransmitters at synapses with submillisecond temporal resolution. However, under the neuron doctrine in perception, only the neural firing enter explicitly in our perception. Therefore, properties of our perception should be obtained without resorting to the idea of an outside observer. Thus, it becomes necessary to adopt the principle of interaction simultaneity.
We can obtain some interesting conclusions about the nature of psychological time. Firstly, the psychological "present" has a finite duration, when measured by the physical time t. The duration corresponds to the transmission delay present when the cluster of interaction-connected neural firings is formed. This could be of the order of ~ 50 ms. In other words, there is a "unit" of the psychological time, with a duration of ~50ms. Despite the existence of such a finite duration of the "moment", the flow of psychological time is shown