Beyond Sense

In Manhattan’s fashion-forward Garment District, Catalan artist Neil Harbisson’s vividly multi-hued shirt, accessorized with a bowtie made out of Legos, is far from radical. But even by the neighborhood’s advanced standards of self-expression, there’s something remarkable about the antenna springing out from his pageboy bowl cut.

The antenna is no mere sartorial affectation. It’s actually implanted into Harbisson’s skull, a legally recognized part of his body according to the UK Passport Authority, which has made an exception to its normal rules to allow him to show it in his passport photo.

Harbisson was born in 1982 with a rare and severe form of color blindness called achromatopsia, which left him with a grayscale view of the world. The antenna that attached to his occipital bone in 2004, which he calls his “eyeborg,” began as Harbisson’s way of hacking his color perception problem. A sensor at its tip feeds information about surrounding light to a chip implanted in his head, translating color frequencies into sonic tones that are transmitted to his inner ear by bone conduction.

“I say I hear color,” Harbisson explains over coffee at the studio he shares with his longtime collaborator Moon Ribas, “but it’s really a new sense.” The phrase “hearing color” implies something like synesthesia, the conflation of information across senses that allows some people to perceive, say, a visual sensation from auditory input. “Synesthesia is the union of two senses,” Harbisson says. “In my case it’s not, because I never see color. It’s a new sense, and it goes beyond sight. I call it sonochromatism.”

What began as simply a way of overcoming a sensory handicap soon evolved into something more profound. As Harbisson’s brain began adapting to the new input from his eyeborg, letting him sense color subconsciously – including dreaming in sonochromatic color – he began to see himself as a fundamentally different type of being than he was before. “After five months of hearing color,” he says, “I suddenly felt no difference between the software and the brain, and that’s when I felt cyborg.”

Over the years Harbisson has upgraded the eyeborg several times, adding new features. It’s now Bluetooth compatible and able to connect to the Internet. In deciding how to translate color to sound, an idea physicists have been toying around with since Newton, Harbisson rejected traditional musical scales in favor of a microtonal range that fits the entire normal visible spectrum of light within a one-octave range. He’s expanded its sensory range to include infrared and ultraviolet frequencies that the human eye can’t pick up unassisted. He can even use it to connect to sensors on satellites and the International Space Station, allowing him to sense colors from space.

Along with his expanding senses, the eyeborg’s also given Harbisson a unique perspective into the evolving, increasingly intimate relationship between humans and technology, and the transformative power of combination. “I no longer feel comfortable identifying myself as a human, actually,” he says. “I feel more comfortable describing myself as an organism, because an organism means that you’re in a wider group. I’m in the same group as an insect or a plant. As a cyborg I feel much closer to these other species because I share senses. Like seeing ultraviolet and infrared light, or hearing via bone conduction, the way dolphins do. Also I share body parts with insects that I didn’t share before.”

A growing number of tech hackers have started experimenting with body modification, from sub-dermal RFID chips to magnetic implants that allow them to feel electromagnetic fields, but few people on the planet have taken the idea of expanding human senses by incorporating technology into our bodies as far as Harbisson. Ribas is one of them.

Without a clear sensory problem to solve like Harbisson’s color blindness, Ribas has had more freedom in her choice of sensory improvements. Over the past decade she’s used technology – first as peripherals, then committing to implants – to try out a number of different types of perception, like the relative speed of objects around her and a type of 360-degree spatial sense. The goals with those experiments, she says, were “to perceive movement in a different way, because I’m a choreographer and a dancer. To have a greater feeling, a different connection with it.”

Since 2013, Ribas has had a small motor implanted in her left elbow, connected to the Internet, that vibrates whenever seismic activity occurs anywhere on Earth. She feels earthquakes, with their power reflected in the intensity of the implant’s vibrations. This tactile connection to the entire planet is a sense that no other creature on Earth has ever experienced, and it’s given her a connection to the planet that’s both profound and exceedingly simple in its intimacy. “I feel like earthquakes are a natural dance themselves,” she says. A corresponding implant in her right elbow will let her sense moonquakes, too, once some technical glitches are ironed out.

As artists, Harbisson and Ribas face the unique challenge of interpreting their senses for an audience that doesn’t have the capability to experience them. Harbisson has experimented with geometric paintings using color palettes drawn from the eyeborg’s translations of tones from famous musical compositions, as well as pieces that do essentially the opposite, composing audio pieces based on the eyeborg’s interpretations of people’s faces – often celebrities like Prince Charles or Leonardo DiCaprio.

At his performance for Moogfest 2016, he’ll paint the toes of Spanish artist Pau Riba while Riba improvises guitar over the eyeborg tones of each color of paint. Ribas will play a “seismic percussion” piece based around input from her implant – an improvised duet with the entire Earth’s crust as her partner, invisible to everyone in the audience.

These interpretations of their newfound senses have given Harbisson and Ribas ins to the global art circuit, but they insist that their senses are ends unto themselves. “Becoming a cyborg is an art itself,” Ribas explains. “We treat our brain like it’s a sculpture.”

It seems inevitable that in the future cybernetic implants will become commonplace. Harbisson points at people’s increasing reliance on their smartphones as evidence that we’re already partway there. “We’re already psychologically connected to technology,” he says. “It’s already become normal to have it permanently in our pockets. It’s just a matter of time to see how we merge physically.”

Body hacking has become big enough in the tech world to start drawing attention from the mainstream press, but the ideas for its applications – like conducting retail transactions with a literal wave of the hand, or new smartphone interfaces – add up to a dreary, mundane vision of the future where we’re all walking around checking work emails in our heads.

The picture Harbisson and Ribas paint of the future is considerably more interesting. They think that new sensory experiences, and not commerce or productivity, will push the evolution of cyborgism. What kind of art can be made when more of us could see in ultraviolet, or hear infrasound frequencies like whales or elephants do? “If you have new senses it opens up a whole new range of art,” Harbisson says. “We think we’ll start seeing more cyborg artists that will change the way we perceive reality through their perception, their new senses.”

Harbisson’s experience transitioning into cyborg life hints at the potential for technologically-expanded senses to positively change the way we experience the world. After a lifetime of seeing only in grays, he initially thought that red was his favorite color. But after upgrading his eyeborg, he figured out that he preferred one that lay outside the normal spectrum of human vision. His new favorite color is infrared.