Concept2Brain: AI Predicts Human Brain Waves From Pictures and Text

How well did the predictions match real brains?

The team first ran a 10-round cross-validation check, repeatedly training on 80 percent of the photographs, monitoring overfitting on 10 percent, and predicting brain waves for the held-out 10 percent.

When predicting responses to unseen photographs, the synthetic brain waves closely matched real recordings, with alignment emerging around 100 milliseconds after picture onset, when visual signals reach the cortex, and remaining strong until roughly 700 milliseconds. One exception was an erotic photograph that correlated negatively with real data, which the authors suggest may reflect CLIP's built-in content-safety guardrails distorting the image's representation.

Given its 360 training photographs, the model reproduced early visual waves and the late positive potential, a slow wave over parietal sensors between 400 and 800 milliseconds linked to emotional engagement. It also reproduced a drop in 10-hertz alpha waves after 300 milliseconds, and matched each of the 88 virtual participants most closely to their own real-world counterpart.

The tests then widened:

• On 1,078 unseen emotional photographs from a standardized library, the simulated late positive potential showed a modest positive association with standard human arousal ratings.
• On fearful and neutral expressions from 140 actors in a facial-expression database, the model generated larger face-processing waves around 170 to 200 milliseconds and larger late emotional waves for fearful faces, despite never being trained on facial-expression labels.

"We know from existing literature that facial stimuli activate the right fusiform face area around 170 ms (the N170 component)," Santos-Mayo said. "Despite the unsupervised setup, the model spontaneously and accurately reproduced this 170 ms component."

In external tests, the synthetic waves aligned with magnetoencephalography recordings (which capture the tiny magnetic fields produced by brain activity) from 4 volunteers viewing 1,854 object concepts, and correlated with fMRI activity in 3 volunteers viewing 720 concepts, in early visual cortex around 75 milliseconds and in a face-processing region around 190 milliseconds. Against an independent dataset of 40 people viewing 40 faces and 40 cars, the model reproduced the scalp map distinguishing faces from cars. The authors note, however, that associations across both external datasets were weaker than within-dataset tests.

What about text prompts?

Because CLIP handles both images and text, the researchers tested 120 short written descriptions sorted into pleasant, neutral, and unpleasant categories, plus prompts such as "a picture of an alligator." Pleasant and unpleasant descriptions prompted larger late emotional waves between 400 and 800 milliseconds than neutral ones. Using two statements about climate change, the model produced stronger responses to one emphasizing harmful consequences than to one describing mitigation strategies, a pattern consistent with negativity bias, the tendency for negative information to draw a stronger response.

"The resulting performance was remarkably similar to that achieved with real pictures, highlighting that our model opens a promising avenue for cross-modal brain signal generation," Santos-Mayo said. The paper reports the emotion, face, and text comparisons descriptively, without formal statistical tests.

What are the limitations?

The caveats are substantial. The model was trained exclusively on people viewing photographs, not reading sentences, so text inputs estimate how a brain might respond to a related visual concept rather than actual language comprehension. Text-prompted predictions are less validated and may be noisier than picture-based outputs. The model produces smoothed waveforms resembling averaged trials, so it cannot capture the trial-to-trial variability of raw single EEG recordings or model underlying cellular mechanisms.

"Synthetic data cannot replace empirical data collected from human participants, nor can it be used to draw definitive scientific conclusions," Santos-Mayo cautioned. "Its purpose is to provide a fast method for testing hypotheses or estimating sample sizes prior to real-world data collection."

The training base of 88 participants and 360 photographs limits the diversity of virtual brains and visual concepts, and because the architecture depends on CLIP, that system's blind spots, biases, and content filters can carry into the generated brain waves, though the framework allows CLIP to be swapped out in future versions.

"This model cannot replace a real human brain, nor can it generate a synthetic 'brain-like' mind. The current state of the field is nowhere near that capability," Santos-Mayo emphasized. "Concept2Brain makes much more modest claims: it serves as a solid proof of concept and a technical milestone in brain signal simulation, but it poses no threat to society, nor should it trigger dystopian fears."

The team plans future upgrades trained on larger datasets with more participants, diverse stimuli, additional neuroimaging modalities, and updated AI components. The study was authored by Alejandro Santos-Mayo, Faith Gilbert, Arash Mirifar, Anna-Lena Tebbe, Ruogu Fang, Mingzhou Ding, and Andreas Keil.