Mind-Reading Technology

Scientists from the University of California, San Francisco have developed a computer program to translate the brain waves of a 36-year-old paralyzed man into text:

Scientists Translate Brain Waves

They implanted an array of electrodes into the sensorimotor cortex of the subject's brain and "used 'deep-learning algorithms' to train computer models to recognize and classify words from patterns in the participant’s brain activity." The training process consisted of showing words on a screen and having the man think about saying them, going through the mental activity of trying to say the words, which he'd lost the physical ability to do. Once the algorithm had learned to match brain patterns to particular words, the subject could produce text by thinking of sentences that included words from the program's vocabulary. Using this technology, he could generate language at a rate of about fifteen words per minute (although not error-free) as opposed to only five words per minute while operating a computer typing program with movements of his head.

Training the program to this point wasn't easy, apparently. The course took 48 sessions over a period of 81 weeks. Still, it's the closest thing to "mind-reading" we have so far, a significant advance over techniques that let a patient control a prosthetic limb by thought alone. According to Dr. Lee H. Schwamm, an officer of the American Stroke Association, “This study represents a transformational breakthrough in the field of brain-computer interfaces."

Here's an article about an earlier experiment in which a paralyzed man learned to produce sentences with "a computer system that turns imagined handwriting into words" at a rate of 18 words per minute.

Mindwriting Brain Computer

The hardware consists of "small, implantable computer chips that read electrical activity straight from the brain." The subject imagined writing letters in longhand, mentally going through the motions. At the same time, the scientists "recorded activity from the brain region that would have controlled his movements." The collected recordings were used to train the AI to translate the man's "mindwriting" into words on a screen. Eventually the algorithm achieved a level of 94.1% accuracy—with the aid of autocorrect, 99%.

While those programs are far from literal telepathy, the ability to read any thoughts that rise to the surface of a subject's mind, they still constitute an amazing advance. As long as such technology requires hardware implanted in an individual's brain, however, we won't have to worry about our computer overlords randomly reading our minds.

Margaret L. Carter

Carter's Crypt

Brain-to-Computer Communication

A research project at Stanford University enables paralyzed people to type on computers by moving a cursor with their thoughts:

Brain-Computer Interface

This technology, according to the article, produces the desired output up to four times as fast as previously existing methods. It's supposed to be superior to the eye-tracking method of operating a computer, which sounds to me as if it would be tiring as well as difficult to master.

Imagine combining a perfected brain-computer interface with the Second Life environment discussed a few weeks ago. Individuals locked into their bodies without even the ability to speak might be able to live a fulfilling life in a virtual environment that feels as multidimensional as the "real world."

Or consider the shell people in Anne McCaffrey's THE SHIP WHO SANG series. Such artificial bodies for people with no control over their physical bodies might become more feasible in actuality once the robotic form could be completely operated by thought alone.

Does the interface described in the SCIENTIFIC AMERICAN article allow speechless people to communicate (through a computer) with something like telepathy? Well, not exactly. The user doesn't beam thoughts through the ether. Wired connections between brain and machine have to be installed. Still, it's an exciting beginning.

Margaret L. Carter

Carter's Crypt

Living a Fuller Life in "Second Life"

Tom Boellstorff, an anthropology professor at the University of California at Irvine (one of my graduate schools), has written a book, COMING OF AGE IN SECOND LIFE, about his experiences doing fieldwork in the virtual world of Second Life. Here's a summary on Goodreads:

Goodreads

He has discovered that people with disabilities such as Parkinson's disease can enjoy full mobility in Second Life and do things impossible for them in their physical bodies:

Parkinson's in Virtual Reality

He resists the conventional assumption that what happens in a VR environment isn't "real." In an article in the UCI alumni magazine, he's quoted as saying, "Even in our physical world, not everything we do is real. And not everything we do online is unreal." For instance, we can lose, gain, or spend real money online. If we learn a language online, we're still learning. Emotions aroused by virtual experiences are genuine emotions. In the NEUROSCIENCE NEWS article linked above, Boellstorff says, "Virtual worlds are online places of culture that impact life in the physical world."

In Second Life, an architect and clothing designer who can no longer create their arts in the material world can do so virtually. Fran, an 88-year-old woman with Parkinson's, dances and practices tai chi online. She maintains that her "friends in Second Life are just as real as friends in real life." Amazingly, she found that her physical strength actually improved as a result of her activities in Second Life. Some scientists credit this phenomenon to mirror neurons, while others are dubious of this explanation, but Fran does seem to have derived concrete benefits from immersing herself in her avatar's experiences. Jadyn, who loved hiking but can't do it in the physical world anymore, created a virtual equivalent of Yosemite in Second Life. Boellstorff designed an island called Ethnographia, where visitors "use art and building tools to work through their difficulties." As he explains it, "Instead of writing about your experience, you can build your own experience."

As far as visual realism is concerned, avatars still fall into the Uncanny Valley, however. They look like dolls or, at best, obvious CGI characters, rather than live people. You can view a sample by Googling "Second Life avatar images." But no doubt this limitation will be overcome in time.

Living inside a virtual world is a frequent motif in science fiction. I can imagine a future in which severely disabled people might choose to spend most of their time in Second Life or a next-generation equivalent. If the technology improves enough, some people might even "move into" the virtual world permanently (with the care and upkeep of their bodies provided for, of course).

The current plot thread on the TV series MARVEL'S AGENTS OF SHIELD features a similar virtual environment built by the antagonist, called the Framework, so advanced that it feels in every way like the real world. The antagonist has captured a SHIELD member, placed her in a permanent coma, and imprisoned her mind in the Framework, where (according to him) she's perfectly happy. He has also lured one of SHIELD's potential allies to his side by promising her a life within the virtual world as an alternative to her terminal illness. The SHIELD genius who created the prototype of the Framework as a safe combat training environment agonizes over his unintended role in the villain's acts. Another character agrees with him, declaring, "The line between scientist and mad scientist is paper thin." While that statement runs counter to the optimistic, science-positive worldview of classic SF, the importance of anticipating consequences remains valid, and every new technology has both good and bad uses. Second Life may function as an "escape from reality" for some people but a portal to a more fulfilling life for others.

Margaret L. Carter

Carter's Crypt