Visual Indicators to Make Music More Enjoyable for the Hearing Impaired

A combination of new visual and touch techniques could enhance the experience of deaf musicians, researchers say.

"Traditional waveform representations are unable to truly describe what music actually sounds like. There are much more subtle characteristics of sound that may be better represented visually, for example, by using simple shapes and symbols to describe harmonic content," said Richard Burn, doctoral student of music technology at Birmingham City University in Britain.

Burn is exploring the usefulness of these techniques in his new research with a view to providing deaf people a greater opportunity to express themselves and help enable them to create new works for both deaf and hearing audiences to enjoy.

Deaf musicians tend to favour acoustic instruments - quite often percussion - which produce a distinct physical feedback from vibration generated by the instrument, alongside more subtle visual clues.

However, using electronic instruments, they often find it more difficult to resolve some of the characteristics of sound, such as pitch and harmonics.

Burn proposes the creation of a new musical interface that will combine haptic and visual forms of feedback to create a more inclusive experience for deaf people.

Alongside vibrations, visual indicators will appear on a digital display that collectively form a 'sonic fingerprint' when an instrument is played, highlighting different components that make up the sound.

"My proposed system will hopefully give deaf players of electronic instruments the same sort of experience as that enjoyed by deaf players of acoustic instruments," Burn said.

"I will be conducting a range of interviews and surveys in order to understand what type of feedback would be useful to deaf musicians. The second stage will be developing systems that deliver those feedback loops and then the third phase will be testing the systems that I have developed to see whether they fit the project's initial aims," Burn said in an official statement.