Meta presented AI for decoding brain activity - ForkLog

# Meta unveils AI for decoding brain activity

On June 29, Meta introduced a system that reconstructs typed text from brain activity recordings without surgical intervention. According to the company, average word-level accuracy reached 61%, compared to about 8% for previous non-invasive methods.

Brain2Qwerty v2 was trained on approximately 22,000 sentences. Nine healthy volunteers participated in the experiment: each spent 10 hours in a magnetoencephalography scanner and typed heard phrases on a keyboard.

"Instead of manually created schemes for detecting neural events, we use end-to-end deep learning to decode data directly from raw brain signals," Meta stated.

Brain2Qwerty v2 works with continuous brain activity recording and reconstructs the entire sentence. To refine the result, the system uses a fine-tuned language model that considers semantic context. The best participant achieved 78% word-level accuracy. The system decoded more than half of their sentences with one error or none.

The development is not yet ready for clinical or household use. Brain2Qwerty v2 was tested on healthy volunteers who actually typed text. For people who have lost speech or the ability to move, this approach needs to be verified separately.

Also, the system does not yet work as full-fledged real-time text input. The model processes the entire sentence, so the user does not see each word immediately after typing it.

The project is related to the Digital Brain Project, an initiative to create open neuroscience datasets. The program budget is $5 million.

Recall that in September 2022, Meta published a study on a "brain decoder" that uses artificial intelligence to convert thoughts into speech.