Researchers develop device that translates sign language into text

Researchers have developed a novel computer system that enables deaf people to translate gestures used in sign language into text.

Using the new technology, people who are deaf or hard of hearing can simply type words and sentences using a keyboard and read those typed to them, researchers said.

Researchers Microsoft Asia and the Institute of Computing Technology at the Chinese Academy of Sciences in this latest effort turned to Microsoft's Kinect device.

They showcased the software that has been developed for the Kinect that successfully translates American Sign Language (ASL) into text, Phys.org reported.

The system operates in two modes: The first, called simply Translation Mode, translates physical hand or body movements into text or speech.

The second, called Communication Mode, allows a person speaking in ASL to communicate with someone else who is communicating in typed English.

The system uses an Avatar to translate text coming from someone typing text on a keyboard, then converts their response to text and sends it back to the other person.

The demonstration by researchers showed that the system is capable of translating sentences, not just words, a significant step forward.

The researchers stressed that their system is still a work in progress but hope to eventually create a system that is fully functional and reasonably inexpensive.

Researchers said that would mean a Kinect based communication system that operates entirely with hand gestures and spoken words - all in real time. Also, it would allow for conversion to other sign language dialects as well.