From: © Encyclopedia of Neuroscience 3d ed, 2004 

Richard E. Cytowic, MD

[Enable Pop Windows for animations]

Sharing a root with anesthesia (meaning “no sensation”), synesthesia means “joined sensation” (Greek syn = together + aisthesis = perception), wherein two or more senses are coupled such that a voice, for example, is not only heard, but also felt, seen, or tasted. Individuals who experience these “joined sensations” are called synesthetes.

 As children, synesthetes are surprised to discover that others do not share their sensory experiences. Often ridiculed and disbelieved, they learn to keep their atypical perceptions private. Nonetheless, the phenomenon remains involuntary and consistent throughout their lives. Some type of synesthetic experience occurs in 1 in 23 individuals.

 Popular at the turn of the previous century, synesthesia became taboo with the rise of behaviorism. It now enjoys a revival as neuroscientists relax their reflexive hostility to subjective experience. Twenty years ago, no one in my academic circle had ever heard the term and all were dismissive of my index case. I only knew the word thanks to Luria’s The Mind of a Mnemonist, whose memory expert retained decades of trivia because of synesthesia in every sense.

 I heard the bell ringing . . . A small round object rolled right before my eyes . . . My fingers sensed something rough like a rope . . . then a taste of saltwater . . . and something white.

 Thereafter came a chance encounter with MW, who delayed our seating at dinner by announcing, “There aren’t enough points on the chicken.” He meant that it failed to evoke the prickly sensation he had intended. To him, taste and smell were also a physical touch in his face and hands (Figure 1). How do the brains of such people differ from those of the majority?

1. What is the experience like?

 Lexical synesthesia, pertaining to letters and words, is the most common form. Sensing color upon hearing, reading, or thinking of letters and integers accounts for two-thirds of synesthetic instances (Figure 2). Whole words are often sensed as also having depth and movement.

Most often it is the written element, or grapheme, that evokes color, so homonyms look different to synesthetes (Figure 3, 3a).

Graphe-color synesthesia is far more common than phoneme-color  synesthesia. In only 10% of cases, the individual sounds of language, or phonemes, evoke the colored shapes that arise, move, alter, and dissipate––somewhat like fireworks.

Download Cytowic & Ward Encyclopedia of Language article, Synesthesia and Language

In the broader category of colored hearing, music and environmental sounds evoke these photisms (Figure 4).

There are, obviously, many types of synesthesia depending on the combination of sensations.

Spatial location and shape are bound to concepts involving serial order. A century ago, Sir Francis Galton called these number forms. Usually colored, they typically allot more Euclidean space to lower members (Figure 5). Time and calendar forms are common (Figure 6).

Spatial location is crucial to those who have number forms. Individuals speak of “going to” or “looking at” a specific location projected outside of their body (Figures 6a, 6b). Number forms commonly loop around or behind one’s body.

Synesthetes often report “odd” or “ugly” colors that they would not choose deliberately. One subject with S–cone deficiency, which makes it difficult to distinguish blues and purples, spone of seeing “Martian colors.” Synesthetes see “unnatural” hues because their visual brains are stimulated nonoptically. Although simple geometry is characteristic of perceptions, subjects struggle to convey an ineffable experience. Even animations (Play Animation 1) fail to adequately represent “what it is like” for synesthetes.

A ranking of sense combinations (Table 1) shows vision to be evoked most frequently. Over 40% of individuals combine multiple senses (polymodal synesthetes). Whereas we typically speak of synesthesia as “joined sensations,” movement, letters, numbers, or months are not senses at all but categories of knowledge, a feature expounded below.

2. Diagnostic criteria

 The term synesthesia has been used to describe such disparate things as metaphor or deliberate contrivances like son et lumière and odorama. Therefore, clear inclusion criteria avoid a muddle. Idiopathic (or developmental) synesthesia arises naturally, absent any external agent or brain abnormality. Medically, there is nothing to treat in such cases. In fact, synesthesia may be a normal brain process that simply fails to reach consciousness in most individuals.

Until an operational definition becomes feasible, five clinical features define idiopathic synesthesia: it is

(1) involuntary and automatic,

(2) spatially extended,

(3) durable and generic,

(4) memorable, and

(5) affect-laden.

2.1. Involuntary and automatic

 Synesthesia happens to a person. Perceptual grouping and pop out betray its automatic nature (Figure 7). Such tests must be tailored to the individual, given that synesthetic associations are idiosyncratic. No agreement exists among subjects. Measuring reaction times for tasks that are either congruent or incongruent with a synesthete’s photisms (Stroop paradigm) also demonstrates automaticity.

Color is bound to visual forms early enough to make searches more efficient compared with controls (Figure 8).

Color thus appears to be bound to a form as it is recognized but after binocular fusion (Figure 9), setting the lower brain limit for lexical synesthesia above V1. Structure-from-motion brain areas (V5) further interact automatically with areas binding color to form, suggesting that back-projections to V1 may be operative.

Although synesthesia is automatic, it does sway to top-down attentional influences (Figure 10).

2.2. Spatially extended

 Synesthesia differs from ordinary vision and imagery by its quality of spatial locus. Percepts extend in an odd Euclidean space as seen in number forms. Synesthetes speak of “going to” or “looking at” a locus to examine a sensation. MW speaks of reaching out to feel a tasted texture, whereas DS describes a screen in front of her face on which music activates falling balls and metallic waves similar to oscilloscope tracings.

 2.3. Durable and generic

 Once established in childhood, synesthetic associations remain constant for life, as demonstrated by test-retest situations spanning many years. This stability compared with controls is suggested as a “test of genuineness,” although this approach is obviously practicable in only some synesthetic permutations.

In the context of synesthesia, generic means that percepts are not pictorial, but elementary in quality—blobs, lattices, cold, rough, sour, and so forth. Given that color, movement, shape, and location normally activate multiple brain areas separated spatially and temporally, synesthesia could be regarded as an anomalous binding of generic attributes, and thus be relevant to the binding problem.

 2.4. Memorable

When asked what good it does, synesthetes immediately answer, “It helps you remember.” Synesthetes do have measurably high memories, sometimes eidetic. The stark relation between synesthesia and elevated memory in Luria’s mnemonist originally suggested the limbic system as a possible linkage site, given that it engages both sensory and memory systems, an idea that has since been modified (see section 4).

 2.5. Affect-laden

 Synesthesia carries a sense of certitude, sometimes described as a type of epiphany or “Eureka!” feeling. Most find synesthesia highly pleasurable and say that to lose it would be odious. For synesthetes, normally trivial tasks are laden with affect: mental calculations are “very pleasurable”; recalling a phone number is “delightful.” On the other hand, mismatched perceptions are “like fingernails on a blackboard.” In fact, a minority of synesthesiae are so wretched (vile-tasting words, nausea on playing a musical instrument) that the condition interferes with daily life. This strong affect is central to understanding synesthesia.

3. Is it real?

Skeptics who question synesthesia’s reality often want a third-person, technological validation of a first-person experience. The criticism that subjective experience is inherently unreliable and thus scientifically impermissible has a long history. Increasingly, it is precisely synesthetes’ subjective reports that are leading scientists to forge unassailable experiments that make disprovable predictions regarding synesthesia’s biological reality..

 3.1. Genetics

 Synesthesia is robustly heritable. Linkage analyses confirm X-linked dominance, probably “with lethality” given that the ratio of affected females to males exceeds 3:1. This means that if hemizygous males carry no normal allele, the gene will be lethal to 50% of synesthetic mothers’ male concepti. A further prediction is increased miscarriage among synesthetic women. Prospective research can clarify these possibilities. Download article:  Discordant Monozygotic Twins

Molecular expression is undoubtedly important and may underlie such observations as phenotypic discordance in monozygotic twins.

 3.2. Cerebral lateralization

Lateralization is somewhat a zero sum game, meaning that extraordinary talent in one area implies a deficit elsewhere. Idiot savants illustrate the extreme principle. Synesthetes have not only extraordinary perception, but also high IQs and memory quotients. Anecdotal claims for mild deficits in arithmetic, right-left confusion, and navigation in synesthetes as a group have not been followed up systematically. Cumulative evidence thus far suggesting a left hemispheric localization is of great theoretical importance, given synesthesia’s relationship to metaphor and the acquisition of language.

4. What is the brain mechanism?

Early work in 1980 proposed that synesthesia depended on a relative deactivation of the analytical neocortex with a corresponding enhancement of limbic elements, partly because all sensation converges in the hippocampus along with memory, salience, and affect, thereby accounting for all features of synesthesia. Later technology did not show the predicted hippocampal activation but did confirm that synesthetic brains are dramatically perturbed by trivial stimuli far beyond that witnessed in nonsynesthetes.

It therefore remains to explain biologically why synesthesia is such an affect-laden experience. An approach more fruitful than strict localization may be hyperbinding.

4.1. The distributed system and cognitive modules

 Synesthesia as hyperbinding relies on a type of neural organization called the distributed system, which depends on topologic relations and patterns of connection rather than strict mapping of function to anatomy. This approach does not localize synesthesia in the sense of classical neurology but sees it existing as the dominant process at a given time within a distributed system.

This approach is not widely known (Mesulam, 1998, 2002). Whereas Brodmann areas are defined by structure, topology classifies cortical tissue into just five types based on the neural transformations they perform, conceiving of tissue in terms of what it does rather than what it is (Figure 11 and Figure 12). Of relevance to synesthesia, only primary sensory and unimodal association areas are sense-specific. Sensory fidelity does not apply to downstream synaptic levels occupied by heteromodal, paralimbic and limbic entities—collectively described as transmodal. Transmodal areas bind modules into multisensory representations.

Classic reticular, nuclear, and laminar structures are referred to as entities. It is the pattern of connection among entities, any one of which can belong to several distributed systems and onto which several functions can be mapped, that constitutes the distributed system. Multiple synaptic levels are simultaneously active, reminding us that localization is probabilistic. Scans mislead us by emphasizing probabilistic peaks.

Both clinical syndromes and imaging show that sensory modules (nodes) operate independently and asynchronously. For example, color (V4) is perceived before form (V2, V3), which is perceived before motion (V5). Clinical states such as achromatopsia, akinetopsia, or the agnosias further show that consciousness can be impaired in severely restricted domains alone. If attributes such as color, movement, direction, and shape can exist independently, the synesthetic binding of color to sound, for example, should not seem so implausible.

4.2. Transmodal binding in a distributed system

 To go from any unimodal sense area to the hypothalamus requires just six synapses (Figure 13). Sensory fidelity is always preserved through the first four synaptic levels. If early connections did exist between senses as they do within a given sense, we would be confused, seeing what we heard and hearing what we saw.

Whereas cross-wiring is forbidden at the first four levels, the deeper transmodal modules are richly cross-linked as indicated in the Figure 13. Transmodal nodes act as bridges binding diverse sensory qualities. Some are well known: Wernicke’s area binds auditory word recognition with visual word recognition so that what we read and what we hear are understood as the same.

4.3. Is binding local, or must it go downtown?

 Figure 13 suggested two possible routes for synesthetic binding. The first violates convention by proposing de novo connections between upstream unimodal modules. For example, color letter synesthesia might arise from cross-wiring between the grapheme and color areas, which do lie near each other. However, this would require wholly new connections in brains of persons who are otherwise normal and does nothing to explain synesthetic affect, spatial location, memorability, and other attributes of the experience. The second possibility hyperbinds categories via existing transmodal bridges—meaning the adding on of aspects not usually associated with a given sense, such as binding color to hearing, or integers to spatial locations.

A Metro map (Metro mass transit system, Washington, DC)analogizes this distinction between horizontal versus vertical connections (Figure 14). Figure 13 showed how local (horizontal) connections between unimodal areas produce sensory confusion. Yet synesthetes are never confused. For them, perceptual elements combine without losing their individual qualities, which results if cognitive qualities are bridgedvertically by transmodal entities, which are further linked to nodes of memory and emotion, thus accounting for synesthesia’s cardinal features.

Vertical organization is a most fundamental principle of brain organization. The cortical column was established as the basic structural unit long ago by the likes of Lorente de Nó, Mountcastle, and Szentágothai. Natural laws follow general principles much more often than they violate them. Vertical organization and the fact that intercolumnar interactions are overwhelmingly local are not reconcilable with speculation that synesthesia results from retained juvenile connections (neoteny) between sensory modules. Agreement exists that synesthetes do retain juvenile connections; the uncertainty is over where they lie.

4.4. The neonatal synesthesia hypothesis

 Cross-modal matching experiments suggest that all neonates are synesthetic in early life, but most lose the ability as their brains mature. Supporting this behavioral data, transient and functioning intersensory connections do exist perinatally in various mammalian species.

The argument for neonatal synesthesia (Maurer, 2005) is as follows: (1) All neonates appear to be synesthetic. (2) Most of their cortical modules function poorly if at all; yet transmodal entities are operative, especially transmodal nodes that later in life will underlie cardinal features of synesthetic hypermnesis and affect. (3) Physiologic necrosis is a normal process whereby transient connections among developing modules form in great excess only to be pruned later. (4) Inheriting a genetic mutation results in failure to prune juvenile connections, and their persistence in mature brains leads to synesthesia. (5) The observation that synesthesia is more common in children suggests that in most individuals neonatal connections are pruned sufficiently so that the effects of hyperbinding never reach consciousness. This last possibility is supported by the observation that synesthesia is 10 times more frequent during Zen meditative states compared with baseline prevalence.

5. Synesthesia and perceptual constancy

Synesthesia’s binding of perceptual qualia to categorical mental concepts points to the process of categorization. Lesions in transmodal nodes produce the clinical agnosias that demonstrate how we do think in categories.

 Perceptual constancy requires the discarding of much flux. Retinex experiments, for example, show that to assign constant colors to surfaces, we must ignore wavelength composition of reflected light, whereas with size perception we must discount the viewing distance and with shape the viewing angle. Out of an infinitely changing energy flux, the brain––whose resources are finite––must assign constant features to objects and determine canonical attributes that constitute a category. This is an enduring puzzle in neuroscience. More than a curiosity, synesthesia may be a window onto a wide swath of mental life.

 6. Further readings

Adler H, Zeuch U (2002): Synästhesie: Interferenz, Transfer, Synthese der Sinne. Würzburg: Königshausen & Neumann

Baron-Cohen, J Harrison, eds (1997): Synaesthesia: Classic and Contemporary Readings. Oxford: Blackwell

Cytowic RE (2002): Synesthesia: A Union of the Senses, 2nd ed. Cambridge: MIT Press (http://Cytowic.net/)

Dann KT (1998): Bright Colors Falsely Seen: Synesthesia and the Search for Transcendental Knowledge. New Haven: Yale University Press

PSYCHE symposium on synesthesia (1995): http://psyche.cs.monash.edu.au/psyche-index-v2.html#syn

Walsh R (2005): Can synesthesia be cultivated? Indications from a survey of meditators. Journal of Consciousness Studies (in press, rwalsh@uci.edu)

Additional bibliography compiled by Sean Day

Bibliographic references

Mesulam MM (2002) Principles of Behavioral and Cognitive Neurology, 2nd ed. New York: Oxford University Press

Maurer D, Mondlach CJ (2005) “Neonatal synesthesia: A Reevaluation,” pp 193-213 in Synesthesia: Perspectives from Cognitive Neuroscience, LC Robertson, S Sagiv (eds). New York: Oxford University Press

Palmeri TJ, et al. (2002): The perceptual reality of synesthetic colors. PNAS 99(6): 4127-4131 http://www.pnas.org/cgi/pmidlookup?view=full&pmid=11904456

Ramachandran VS, Hubbard EM (2001): Psychophysical investigations into the neural basis of synaesthesia. Proc Royal Soc London 268:979-983

Smilek D, Dixon MJ (2002): Towards a synergistic understanding of synaesthesia:
combining current experimental findings with synaesthetes’ subjective descriptions. http://psyche.cs.monash.edu.au/v8/psyche-8-01-smilek.html

Figure legends

 Figure 1. Synesthete MW. “With an intense flavor, the feeling sweeps down my arm, and I feel weight, texture, its shape, and whether it’s warm or cold, like I’m actually grasping something.” Return to Text

 Figure 2. Lexical synesthesia. Note how this synesthete perceives “6” as spatially greater than other integers, thus constituting a number form (cf. Figure 5). Return to Text

 Figure 3. Grapheme-color synesthesia is much more common (90%) than phoneme-color synesthesia (10%). Return to Text

 Figure 4. (Top) Hearing her doorbell makes this synesthete see brown and gray triangles drifting off to the right as they fade; (middle) her dog’s bark produces shimmering circles moving out from the center; and (bottom) the whoosh of the furnace ignition produces a stack of colored lines. Such two-dimensional drawings are said to be only about 60% representative of the actual experience. Simple geometry is characteristic of synesthetic perceptions. Return to Text

 Figure 5. Number forms pertain to the Euclidean extension of concepts concerning serial order. Return to Text

 Figure 6. In calendar forms, the topmost month is by no means always January. Note unequal allotment of psychophysical space. For this individual, brown November contains an internested form of brown days in a serpentine shape. Return to Text

Figure 6a, b   Courtesy Dr. David Eagleman, who is recruiting subjects with number forms

 Animation 1. by Dr. Rolf Schmidt, Medizinische Hochschule Hannover, schmidt.rolf@mh-hannover.de

 Figure 7. Automatic perceptual grouping in synesthetes leads to pop out. (From Ramachandran & Hubbard, 2001) Return to Text

 Figure 8. (Top) In the search display, WO stated that a unique orange patch drew his attention to the target before he explicitly recognized what it was. This perception may make his searches more efficient compared with controls. (Bottom) Because sixes and eights both appear bluish to WO, search time increases linearly with set size. (From Palmeri et al., 2002) Return to Text

 Figure 9. Random dot stereogram. The left eye looks at one pattern of dots, and the right eye looks at another pattern of dots, so that a three-dimensional object pops out when the two images fuse in the brain. Synesthetic binding of color to form occurs after binocular fusion. (From Palmeri et al., 2002)  Return to Text

 Figure 10. Synesthetic perception of an achromatic cyclopean figure. When the subject attends to the global 5, it looks greenish; when attending to the constituent 2s, the 5 becomes orange (Palmeri et al., 2002). Return to Text

 Figure 11. (Top) Schematic hierarchy of the five topologic tissue types as sensation flows inward to become incorporated into the texture of cognition. Compared to the 53 Brodmann areas defined by cyto-architecture, Topology divides cortex into just 5 types based on functional topology—i.e. what a tissue does rather than what it is. (Bottom) Polarization of reciprocal monosynaptic sensory-fugal connections in vision and hearing. Thick arrows represent more massive connections than thin arrows. Dashed arrows illustrate motor output pathways. Somatosensory pathways are similarly organized while also displaying unique properties such as monosynaptic connections between idiotypic sensory and motor areas. Smell and taste, being chemical senses, are organized differently. (From Mesulam, 2002)  Return to Text

 Figure 12. The multiplex concept merges hierarchical, linear processing with parallel distribution in topologic zones of unique neural tissues. This complex and recursive pathway from the outside world starts with the orange ribbon at the upper left and ends in an internal sense of self indicated by the darkening gradient at the figure’s rear plane. Early stages are largely linear, whereas parallel mapping starts roughly at the purple horizontal line, principally via vertical intercortical connections with other zones and horizontal intracortical connections within the same zone. No interconnections exist among primary sensory or unimodal association cortices belonging to different senses. Yet in the heteromodal, paralimbic, and limbic zones, strong horizontal connections exist with other components in the same zone. Rich linking combines the neural transformations of many structures, and limbic entities give salience to any event. The effects of volume transmission cannot be illustrated here. The spiral curving inward toward the dark internal milieu indicates that downstream entities contribute both divergent and convergent processes to the construction of consciousness and self. The sphere labeled “sense of self” appears self-contained and set apart from the aforementioned entities; yet it too is part of recursive multiplex systems. The most distal (interior) transformations feed back to the most proximal members, the sense organs themselves. (From Cytowic, 2002) Return to Text

     *The curved arrow indicates the one-to-many multiplex projections from idiotypic cortices characterized by the distributed system.

     **Heteromodal and paralimbic tissues perform two kinds of transformation: (1) further parallel elaboration of multiple sensory maps and (2) integration of the result with drive, emotion, and mental content. Earlier cortical types are fairly homogeneous transformers of a single sense; later types have heterogeneous input-output relations, and no uniform type of behavior can be ascribed. Sense specificity yields to intermodal binding. Even the distinction between what is motor and what is sensory is now lost.

 Figure 13. A, Synaptic levels in topographic relations for vision (green) and hearing (blue). Each concentric ring represents a different synaptic level. Primary sensory cortex occupies level 1. Small circles represent macroscopic cortical nodes measuring 1 to several centimeters in diameter. Nodes at a given level are connected reciprocally as illustrated by the black arcs of the concentric rings; gaps in the first four levels indicate the lack of monosynaptic connections between modality-specific nodes of hearing and vision systems. Broken lines represent reciprocal monosynaptic connections from one synaptic level to another. Dorsal and ventral refer to the divergence of visuo-fugal pathways, especially at the fourth synaptic level, into the dorsal “where” and ventral “what” streams. B, Transmodal nodes are indicated (red) at the fourth through sixth levels. Transmodal cortices have few monosynaptic projections to primary sense areas, and they send fewer projections to upstream than to downstream components of unimodal areas. Sensory fidelity is actively maintained through the first four synaptic levels, and the first two levels are relatively protected from values-based modulations. Convergent-divergent organization intensifies from level 3 onward. The first synaptic level is finely tuned to a primary sense. The second level extracts attributes such as color and motion. The third and fourth levels are critical to categorical identification and contain coarsely tuned neurons. Note how some vision and hearing modules perform similar functions at deeper levels, such as visual and auditory recognition of word forms or assigning spatial locations to things seen or things heard.   Return to text

A1, primary auditory cortex; V1, primary visual cortex; V2, V4, V5, upstream visual areas; f, area specialized for face encoding; wr, area specialized for word–form encoding; s, area specialized for spatial-location encoding; v, area specialized for identifying individual voice patterns; W, Wernicke’s area; T, heteromodal lateral temporal cortex; L, hippocampal-entorhinal or amygdaloid components of the limbic system; P, heteromodal posterior parietal cortex; Pf, lateral prefrontal cortex. (Modified from Mesulam, 2002)

 Figure 14. Metro map (Metro mass transit system, Washington, DC) analogy to synesthetic hyperbinding via either de novo horizontal or existing vertical transmodal pathways. Although two stations on the Red Line lie at the same level, there is no local connection. To get from one to the other, you must go vertically by way of downtown nodes, which are further connected with other distributed systems.  Return to text

Table 1. Types of synesthesia (n = 365)

Colored graphemes	66.8%
Colored time units	19.2%
Colored musical sounds	14.5%
Colored general sounds	12.1%
Colored phonemes	9.6%
Colored musical notes	10.4%
Colored personalities	4.4%
Colored tastes	6.3%
Colored pain	4.4%
Colored odors	5.8%
Colored temperature	2.2%
Colored touch	1.9%
Sound ® touch	2.7%
Sound ® taste	2.7%
Sound ® smell	1.1%
Sound ® temperature	0.5%
Taste ® hearing	0.3%
Taste ® touch	1.1%
Touch ® taste	0.5%
Touch ® smell	0.3%
Touch ® hearing	0.5%
Vision ® taste	1.9%
Vision ® hearing	1.1%
Vision ® smell	1.1%
Vision ® touch	0.8%
Smell ® sound	0.3%
Smell ® touch	1.1%

   Over 40% of individuals have multiple synesthesias. (From Cytowic, 2002)  Return to Text