A major breakthrough in understanding visual thoughts has recently been achieved by Dutch researchers. Thanks to sophisticated artificial intelligence (AI), it is now possible to reconstruct mental images based solely on an individual's brain activity.
Illustrative image Pixabay
Scientists at Radboud University in the Netherlands have developed an AI capable of recreating images seen by individuals by analyzing their brain activity. The results of their study, pre-published in
bioRxiv, reveal remarkable accuracy in the reconstruction of mental images. This outcome is based on data obtained from two distinct studies.
In the first study, volunteers underwent functional magnetic resonance imaging (fMRI) while viewing photographs. The second study involved a macaque whose brain activity was recorded using implanted electrodes as it observed AI-generated images. The data from these experiments were used to train the AI to reconstruct the original images.
The AI used incorporates predictive attention mechanisms (PAM) to focus on the most relevant brain signals. By processing these signals with a generative adversarial network, the AI manages to create images visually close to the originals. The best results were obtained with images viewed by the macaque, whose brain activity, recorded directly by electrodes, is more precise than fMRI data, which is often disrupted by noise.
Umut Güçlü, one of the principal researchers, emphasizes that these reconstructions are the most precise ever achieved. He explains that the AI, by interpreting brain signals, learns to focus its attention on the essential aspects to reconstruct the viewed images. The ability to faithfully recreate what an individual perceives could pave the way for more advanced brain implants, allowing for a more natural restoration of sight.
Reconstruction results.
The top block shows ten examples from the B2G dataset (GAN-generated stimuli), and the bottom block ten examples from the GOD dataset (natural stimuli). The top rows present the original stimuli, the middle rows the PAM (P) model reconstructions, and the last rows the linear decoder (L) reconstructions.
The potential applications of this technology are vast. In addition to restoring vision, this breakthrough could transform communication for visually impaired individuals and improve neural interfaces. According to the researchers, the rapid advances in generative modeling could further enhance the accuracy of visual reconstructions in the near future.
Article author: Cédric DEPOND