The Guardian – August 23, 2023
Paralysed woman able to ‘speak’ through digital avatar in world first
By Hannah Devlin
Latest technology uses tiny electrodes on brain surface and is faster than synthesisers which rely on eye tracking
A severely paralysed woman has been able to speak through an avatar using technology that translated her brain signals into speech and facial expressions.
The advance raises hopes that brain-computer-interfaces (BCIs) could be on the brink of transforming the lives of people who have lost the ability to speak due to conditions such as strokes and amyotrophic lateral sclerosis (ALS).
Until now, patients have had to rely on frustratingly slow speech synthesisers that involve spelling out words using eye tracking or small facial movements, making natural conversation impossible.
The latest technology uses tiny electrodes implanted on the surface of the brain to detect electrical activity in the part of the brain that controls speech and face movements. These signals are translated directly into a digital avatar’s speech and facial expressions including smiling, frowning or surprise.
“Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others,” said Prof Edward Chang, who led the work at the University of California, San Francisco (UCSF). “These advancements bring us much closer to making this a real solution for patients.”
The patient, a 47-year-old woman, Ann, has been severely paralysed since suffering a brainstem stroke more than 18 years ago. She cannot speak or type and normally communicates using movement-tracking technology that allows her to slowly select letters at up to 14 words a minute. She hopes the avatar technology could enable her to work as a counsellor in future.
The team implanted a paper-thin rectangle of 253 electrodes on to the surface of Ann’s brain over a region critical for speech. The electrodes intercepted the brain signals that, if not for the stroke, would have controlled muscles in her tongue, jaw, larynx and face.
After implantation, Ann worked with the team to train the system’s AI algorithm to detect her unique brain signals for various speech sounds by repeating different phrases repeatedly.
The computer learned 39 distinctive sounds and a Chat GPT-style language model was used to translate the signals into intelligible sentences. This was then used to control an avatar with a voice personalised to sound like Ann’s voice before the injury, based on a recording of her speaking at her wedding.
The technology was not perfect, decoding words incorrectly 28% of the time in a test run involving more than 500 phrases, and it generated brain-to-text at a rate of 78 words a minute, compared with the 110-150 words typically spoken in natural conversation.
However, scientists said the latest advances in accuracy, speed and sophistication suggest the technology is now at a point of being practically useful for patients.
Prof Nick Ramsey, a neuroscientist at the University of Utrecht in the Netherlands, who was not involved in the research, said: “This is quite a jump from previous results. We’re at a tipping point.”
A crucial next step is to create a wireless version of the BCI that could be implanted beneath the skull.
“Giving people the ability to freely control their own computers and phones with this technology would have profound effects on their independence and social interactions,” said Dr David Moses, an assistant professor in neurological surgery at UCSF and co-author of the research.
The findings are published in Nature (paywall).
Garrison Keillor – August 23 2023: One good reason to travel around America is to meet American people, all the more so if you’re one of them yourself. I went out West for ten days and rediscovered what I always knew, that our people don’t mind talking about themselves. You call a cab at 5 a.m. in Flagstaff and a cheerful guy pulls up at your hotel and you ask him how his day is going — “Fine,” he says, “I’m on the midnight shift and I love to see the sun come up.”
“You from here?” No, he’s from Boston, he came out here to help his son who owns the cab company, and he loves Arizona, the climate relieves his arthritis. “So what did you do back in Boston?”
In Europe, this question might raise hackles, like asking, “What’s your annual income?” but he doesn’t mind. He’s a retired Baptist minister. And now the door opens wide — you’ve got yourself a good conversation all the way to the airport.
 |
He asked if I had come out to see the Grand Canyon. Yes, indeed. The day before, I got up at 5 a.m. and hiked over to the South Rim and stood in the dark in the midst of strangers as the eastern sky lightened and it was like church except no singing and no handshakes.
The light dawns and Creation appears, six million years of rock, the mighty Colorado below, the underlying beauty of our little planet home, the silent fellow tourists around us, whispers of German and Swedish and Japanese, the canyon walls turning reddish orange, and you stand gazing down for an hour or two and I resolved to give up regret, which is merely self-pity, and to embrace what is true, namely love and kindness, the vocation of cheerfulness, the dedication to the day, this day, each hour. And then I went to the café in the Bright Angel Lodge and had oatmeal and coffee.
 |
A man stopped at my table who recognized me from my radio days. “Have a seat,” I said. He’s from Ohio, retired high school English teacher. Like everyone my age, he’s worried about young people. “They’re so busy with sports and activities and social media and video games and whatnot, it got so I couldn’t assign reading, they just didn’t have time for it.”
“We were lucky to be born when we were,” I say. We had the advantage of boredom, which led us to become readers. And we launched into memories of our long-ago youth.
I came back to New York Saturday and Sunday morning in church, we sang, “When from bondage we are summoned out of darkness into light, we must go on hope and patience, walk by faith and not by sight,” and the next phrase, “Let us throw off all that hinders” struck home because we’ve cleaned out boxes of stuff recently, including a disaster of a novel I was glad to see disappear, and the next verse about Jesus leading us by the hand through a barren desert, reminded me of standing at sunrise looking down into the Canyon.
 |
I didn’t go there to study geology. What I know about rock formations you could put in a grasshopper’s left nostril. Like the people around me, I was there to be with the Lord. I felt in bondage to the past, a common phenomenon when you have so much of it, bondage to Midwestern irony, a dryness, a refusal of joy. Or maybe it comes from my evangelical upbringing, a separatist tendency, an aloofness, the elitism of the redeemed and enlightened, which leads naturally to a sort of brittle satire. You look at the morning paper and all of your biases come to attention.
A person needs to break through this armor. I see the refugee kids from Ecuador selling candy on the subway platforms in midtown Manhattan and the heart absolutely breaks. Families have fled to America for simple survival and it’s more than New York can handle but New York still opens its heart to them. I see the videos on Facebook of my Vietnamese great-nieces in Ho Chi Minh City and the armor cracks. I stand in the dawn of Creation and am overwhelmed. Life is good, praise the Lord, but a criminal billionaire has exploited the resentments of millions and is out to destroy the republic. This must not happen, people. It simply must not.