Brain Computer Interface Turns Mental Handwriting Into Text on Screen, Could Help People With Paralysis

A team of researchers at Stanford University have, for the first time, decoded brain activity related to writing by hand. To further explore this, the researchers collaborated with a participant with paralysis who had sensors implanted in his brain. The team then used an algorithm that identified the letters the participant imagined and attempted to write. The system then displayed the same text on-screen in real time, meaning as and when the participant imagined them. Krishna Shenoy, the study co-author and a Howard Hughes Medical Institute Investigator at the university, said the innovation, with more development, could well help people with paralysis rapidly type without using their hands.

In the experiment, the participant could type 90 characters per minute, which was more than double the typing record set by the previous “brain-computer interface,” the researchers said. Shenoy jointly supervised the study with Jaimie Henderson, a Stanford neurosurgeon.

Jose Carmena, a neural engineer at the University of California, Berkeley, who was not involved in the research says such technology had the potential to help people with all kinds of disabilities. "It's a big advancement in the field," he says.

An injury or a disease may take away a person's ability to move but the brain's neural activity connected with walking, holding a cup of coffee or tea, or even speaking a sentence remains. And this activity is exactly what the scientific community can tap into to enable people with paralysis or amputations to express their thoughts or even carry out an activity.

In recent years, Shenoy's team has deciphered the neural activity related to speech, hoping to reproduce it. Not just that, his team had also developed a technology to help participants move a cursor on the screen with help of implanted sensors.

This helped them type at least 40 characters on the screen by pointing and clicking on letters. Frank Willett, a neuroscientist in Shenoy's group, said that they wanted to find "new ways of letting people communicate faster," adding that he could try "something different" also motivated him.

Krishna Shenoy's team now envisages a more comprehensive system that — besides handwriting for text entry — includes point-and-click navigation and even attempted speech decoding. "Having those two or three modes and switching between them is something we naturally do," he said.

For this, his team next plans to collaborate with someone who cannot speak. For example, someone with amyotrophic lateral sclerosis, a degenerative neurological disorder causing the loss of movement and speech.

Henderson says the new system, when operation, could help people suffering from paralysis caused by several conditions, including brain stem stroke.

Jean-Dominique Bauby, the author of the book The Diving Bell and the Butterfly, suffered from the same, but he was able to write a beautiful book by "selecting characters painstakingly, one at a time, using eye movement," Henderson said.

"Imagine what he could have done with Frank's handwriting interface!"