Evolving Perception: Tele-Synesthesia and Touch Technology

How are human beings adapting to increasingly virtual environments? Belgian artist and founder of the Belgian Synesthesia Association, Dr Hugo Heyrman, has devoted 40 years to studying and experimenting with perception. His terms 'tele-senses' and 'tele-synesthesia' describe how our senses are merging with and enhanced by interactive technology.

In synesthesia, neurological cross-wiring causes senses to mingle, often with exotic effect. However, we also know that one sense links to other senses by association. In cybermedia, and touch-technology in particular, the boundaries between the internal and external, between the what is here and what is there, become confused. Our senses become 'tele-senses' - enhanced and extended in cyberspace ('tele' means 'far' in Greek). As interactive multimedia and electronic networks create uncharted possibilities of interconnection, we are enabled to expand the reach of our sensorial perception.

'Tele-synesthesia' is the synesthetic principle that is expanded and extended by means of new media: the traveling senses. It is defined as virtual interactions between the tele-senses, developed by means of new technological means in order to overcome the constraints of the human senses.

This is fast becoming reality, and in some cases, aesthetically and conceptually beautiful. Jeff Han of Perceptive Pixel has developed a highly sophisticated, interactive multi-touch technology with multiple user capacity. Each pixel is a touch sensor, providing an infinite number of points that can be captured and manipulated by the user.




The user (or users) simultaneously employs fingers, hands and arms on large-scale screens, collaborating parts of the brain used for interaction with physical objects.

"By synchronising images, sound, movement and haptic experiences, electronic media are able to bring about the intermingling and fusion of one medium into another, resulting in making colours audible, visualising sound and making words palpable... Future libraries will become brain banks instead of book banks, synesthetic archives: an interactive multimedia integration of the visual, the kinetic, the haptic, the sonic, and the telematic."

In terms of adapting to virtual environments & multi-sensory learning, Heyrman concludes that synesthetes, 'Homo Futuris', have a head start on the rest of us: "because with the futuristic, telematic extension of the human senses, everything will become more and more synesthetic."


SEE ALSO:
Customizing Sensory Reality
Hearing Colors, Tasting Shapes
Fusiform Gyrus Sounds good

Do You See What I See?

Try out the following online tests & experiments for synesthesia:

• David Eagleman's 'Synesthesia Battery' is the most comprehensive synesthesia test I've found so far & excellent for synesthetes who want explore a possibly extended range of their gifts, especially as research is showing that 40% of synesthetes have more than one form of synesthesia. The site lists a wide range of possible synesthetic combinations for testing, including musical-instruments>color, sound>smell, Chinese grapheme>color and orgasm>color. Disappointingly, when I checked the box that applied to me, sound>taste, I was immediately directed to a Results page which thanked me for my participation. For me, took about 1 minute to complete.
  
  
  
• The BBC's charmingly named Do you see what I see?, from which I appropriated the title of this post, hosts a single test for grapheme-color synesthesia and spatial-sequence (number form) synesthesia. About 10 minutes.
  
  

• BU's Synesthesia Project has a great test for non-graphemic shapes>color synesthesia. The Project's research on grapheme-color synesthesia found that many synesthetes report having colored basic shapes, including triangles and squares, in addition to their colored letters and/or numbers. Are you one of them? Take the 'Non-Graphemic Shape Battery' here. About 20 minutes.
  

Fusiform Gyrus Sounds Good

While I’m not a synesthete myself, 'fusiform gyrus' has got to be one of the yummiest phrases around. It takes the mouth so many places, & sounds like an onomatopoeic compression of Goldfrapp’s Strict Machine.



To neuroscientist Vilayanur Ramachandran, the fusiform gyrus section of the brain may be the key to the ‘linked senses’ phenomenon of synesthesia. The fusiform gyrus is part of the temporal lobe and houses the areas responsible for color and number perception, which happen to be next to one another. Noting the hereditary nature of the condition, Ramachandran posits that in synesthetes there is cross-wiring between these two areas resulting in simultaneous perception of sensory input and very often, enhanced creativity.

Ramachandran is a dynamic and entertaining speaker. In the TED Talk below he expands on this theory. The section on synesthesia starts at 17:53.

Hearing Colors, Tasting Shapes

Synesthesia is a perceptual phenomenon whereby otherwise normal people have 'tangled' senses. The real information gathered by one sense (e.g. sight) is accompanied by a perception in another sense (e.g. touch or sound).

Imagine that every time you saw the number 4, it would be eggshell blue; or every time you heard an F tone you saw a crisp purple arc half a foot in front of you.

Synesthetic perception can occur between any 2 senses. Probably the most common type of synesthesia is grapheme—color (chromatographemic) synesthesia. For the life of the synesthete, letters and numbers are tinged with a particular shade or color. Attending a cocktail party a few years ago, I was chatting about the subject when a handsome stranger suddenly blurted: ”4 is Green! Lettuce is 4!” In his 40s, this eavesdropping attorney had never known there was a name for this peculiarity in his perception. Like most synesthetes, he assumed everyone saw the way he did until an experience at school had him realize he was ‘different’, & he had never mentioned it to anyone until the day we spoke. He later discovered that his father also saw certain letters and numbers shaded. Research does indicate that synesthesia is to some extent hereditary.

Synesthesia usually doesn’t interfere with day to day life. There are 8 times as many synesthetes working in the creative professions – artists, poets, writers, musicians – than in the general population.

Many synesthetes find that having linked senses assists them in tasks of memorization. Musician Noriko Nagata who sees colors in sounds (chromestesia – colored-hearing synesthesia) reports, “As I was receiving professional education in music (I also have a sense of perfect pitch), the resonance of a sound and the image of a color have always been deeply connected. When it came to composing music, I would think, "I will make blue colored music", or think, "What were these codes I remembered in pink and beige?" during a test on guessing the right code names of tension codes. It has been my habit since I was small to feel colors and memorize things in colors using this way.”

Examples of some more elaborate forms of synesthesia follow:

Some synesthetes taste shapes: ergo the statement, ‘There aren’t enough points in the chicken.’ The taste of roast chicken made this synesthete feel a round shape in his hands, as if he were rubbing a bowling ball instead of feeling the prickly shape he expected. (Cytowic, “The Man who Tasted Shapes”, MIT Press p.11)

Others upon hearing a sound see light, color and identifiable images. ‘Presented with a tone pitched @ 250Hz amplitude 64db, S saw a velvet cord with fibres jutting out on all sides. The cord was tinged with a delicate, pleasant pink-orange hue… Presented with a tone pitched @ 3000Hz amplitude 113db, he saw a whisk broom that was of a fiery color, while the rod attached to the whisks seemed to be scattering off into fiery points. The experiments were repeated during several days and invariably the same stimuli produced identical experiences.” (Baron-Cohen & Harrison, "Synaesthesia: Classic and Contemporary Readings", Blackwell Publishers, Cambridge, MA, p.102)

MIT's 'The Synesthetic Experience' has a couple of on-line demos simulating synesthetic experience, and accounts of first-hand experiences of synesthesia.