Posts Tagged ‘Edd Gent’

By Edd Gent

The idea of a machine that can decode your thoughts might sound creepy, but for thousands of people who have lost the ability to speak due to disease or disability it could be game-changing. Even for the able-bodied, being able to type out an email by just thinking or sending commands to your digital assistant telepathically could be hugely useful.

That vision may have come a step closer after researchers at the University of California, San Francisco demonstrated that they could translate brain signals into complete sentences with error rates as low as three percent, which is below the threshold for professional speech transcription.

While we’ve been able to decode parts of speech from brain signals for around a decade, so far most of the solutions have been a long way from consistently translating intelligible sentences. Last year, researchers used a novel approach that achieved some of the best results so far by using brain signals to animate a simulated vocal tract, but only 70 percent of the words were intelligible.

The key to the improved performance achieved by the authors of the new paper in Nature Neuroscience was their realization that there were strong parallels between translating brain signals to text and machine translation between languages using neural networks, which is now highly accurate for many languages.

While most efforts to decode brain signals have focused on identifying neural activity that corresponds to particular phonemes—the distinct chunks of sound that make up words—the researchers decided to mimic machine translation, where the entire sentence is translated at once. This has proven a powerful approach; as certain words are always more likely to appear close together, the system can rely on context to fill in any gaps.

The team used the same encoder-decoder approach commonly used for machine translation, in which one neural network analyzes the input signal—normally text, but in this case brain signals—to create a representation of the data, and then a second neural network translates this into the target language.

They trained their system using brain activity recorded from 4 women with electrodes implanted in their brains to monitor seizures as they read out a set of 50 sentences, including 250 unique words. This allowed the first network to work out what neural activity correlated with which parts of speech.

In testing, it relied only on the neural signals and was able to achieve error rates of below eight percent on two out of the four subjects, which matches the kinds of accuracy achieved by professional transcribers.

Inevitably, there are caveats. Firstly, the system was only able to decode 30-50 specific sentences using a limited vocabulary of 250 words. It also requires people to have electrodes implanted in their brains, which is currently only permitted for a limited number of highly specific medical reasons. However, there are a number of signs that this direction holds considerable promise.

One concern was that because the system was being tested on sentences that were included in its training data, it might simply be learning to match specific sentences to specific neural signatures. That would suggest it wasn’t really learning the constituent parts of speech, which would make it harder to generalize to unfamiliar sentences.

But when the researchers added another set of recordings to the training data that were not included in testing, it reduced error rates significantly, suggesting that the system is learning sub-sentence information like words.

They also found that pre-training the system on data from the volunteer that achieved the highest accuracy before training on data from one of the worst performers significantly reduced error rates. This suggests that in practical applications, much of the training could be done before the system is given to the end user, and they would only have to fine-tune it to the quirks of their brain signals.

The vocabulary of such a system is likely to improve considerably as people build upon this approach—but even a limited palette of 250 words could be incredibly useful to a paraplegic, and could likely be tailored to a specific set of commands for telepathic control of other devices.

Now the ball is back in the court of the scrum of companies racing to develop the first practical neural interfaces.

How a New AI Translated Brain Activity to Speech With 97 Percent Accuracy


by Edd Gent

Wiring our brains up to computers could have a host of exciting applications – from controlling robotic prosthetics with our minds to restoring sight by feeding camera feeds directly into the vision center of our brains.

Most brain-computer interface research to date has been conducted using electroencephalography (EEG) where electrodes are placed on the scalp to monitor the brain’s electrical activity. Achieving very high quality signals, however, requires a more invasive approach.

Integrating electronics with living tissue is complicated, though. Probes that are directly inserted into the gray matter have been around for decades, but while they are capable of highly accurate recording, the signals tend to degrade rapidly due to the buildup of scar tissue. Electrocorticography (ECoG), which uses electrodes placed beneath the skull but on top of the gray matter, has emerged as a popular compromise, as it achieves higher-accuracy recordings with a lower risk of scar formation.

But now researchers from the University of Texas have created new probes that are so thin and flexible, they don’t elicit scar tissue buildup. Unlike conventional probes, which are much larger and stiffer, they don’t cause significant damage to the brain tissue when implanted, and they are also able to comply with the natural movements of the brain.

In recent research published in the journal Science Advances, the team demonstrated that the probes were able to reliably record the electrical activity of individual neurons in mice for up to four months. This stability suggests these probes could be used for long-term monitoring of the brain for research or medical diagnostics as well as controlling prostheses, said Chong Xie, an assistant professor in the university’s department of biomedical engineering who led the research.

“Besides neuroprosthetics, they can possibly be used for neuromodulation as well, in which electrodes generate neural stimulation,” he told Singularity Hub in an email. “We are also using them to study the progression of neurovascular and neurodegenerative diseases such as stroke, Parkinson’s and Alzheimer’s.”

The group actually created two probe designs, one 50 microns long and the other 10 microns long. The smaller probe has a cross-section only a fraction of that of a neuron, which the researchers say is the smallest among all reported neural probes to the best of their knowledge.

Because the probes are so flexible, they can’t be pushed into the brain tissue by themselves, and so they needed to be guided in using a stiff rod called a “shuttle device.” Previous designs of these shuttle devices were much larger than the new probes and often led to serious damage to the brain tissue, so the group created a new carbon fiber design just seven microns in diameter.

At present, though, only 25 percent of the recordings can be tracked down to individual neurons – thanks to the fact that neurons each have characteristic waveforms – with the rest too unclear to distinguish from each other.

“The only solution, in my opinion, is to have many electrodes placed in the brain in an array or lattice so that any neuron can be within a reasonable distance from an electrode,” said Chong. “As a result, all enclosed neurons can be recorded and well-sorted.”

This a challenging problem, according to Chong, but one benefit of the new probes is that their small dimensions make it possible to implant probes just tens of microns apart rather than the few hundred micron distances necessary with conventional probes. This opens up the possibility of overlapping detection ranges between probes, though the group can still only consistently implant probes with an accuracy of 50 microns.

Takashi Kozai, an assistant professor in the University of Pittsburgh’s bioengineering department who has worked on ultra-small neural probes, said that further experiments would need to be done to show that the recordings, gleaned from anaesthetized rats, actually contained useful neural code. This could include visually stimulating the animals and trying to record activity in the visual cortex.

He also added that a lot of computational neuroscience relies on knowing the exact spacing between recording sites. The fact that flexible probes are able to migrate due to natural tissue movements could pose challenges.

But he said the study “does show some important advances forward in technology development, and most importantly, proof-of-concept feasibility,” adding that “there is clearly much more work necessary before this technology becomes widely used or practical.”

Chong actually worked on another promising approach to neural recording in his previous role under Charles M. Lieber at Harvard University. Last June, the group demonstrated a mesh of soft, conductive polymer threads studded with electrodes that could be injected into the skulls of mice with a syringe where it would then unfurl to both record and stimulate neurons.

As 95 percent of the mesh is free, space cells are able to arrange themselves around it, and the study reported no signs of an elevated immune response after five weeks. But the implantation required a syringe 100 microns in diameter, which causes considerably more damage than the new ultra-small probes developed in Chong’s lab.

It could be some time before the probes are tested on humans. “The major barrier is that this is still an invasive surgical procedure, including cranial surgery and implantation of devices into brain tissue,” said Chong. But, he said, the group is considering testing the probes on epilepsy patients, as it is common practice to implant electrodes inside the skulls of those who don’t respond to medication to locate the area of their brains responsible for their seizures.