Attempting dream decoding with generalizable visual EEG encoding models

Poster Session C - Sunday, April 14, 2024, 5:00 – 7:00 pm EDT, Sheraton Hall ABC

Qiaorong Yu¹ (qiaorong.yu@balliol.ox.ac.uk), Remington Mallett², Michelle Carr²; ¹University of Oxford, ²University of Montreal

The realm of dreams remains relatively uncharted, largely due to the reliance of subjective reports to access dream content. This study aims to use electroencephalogram (EEG) to decode visual dream contents. We leveraged an encoding model which generates EEG signals from deep neural network (DNN) feature maps of visual images. The model was trained on a large EEG dataset of waking EEG responses to 18,540 naturalistic images (THINGS2). We first investigated the model’s generalizability by testing it on another waking EEG dataset built from the same image database (THINGS1). The encoding model successfully generalised across diverse data collection conditions by training on visual perception from THINGS2 and accurately decoding visual perception from THINGS1. Distinct neural representations for different visual images were observed at 0.1-0.15 and 0.5 seconds following the presentation of visual stimuli. Second, we tested if the same model trained on waking perception would generalise to a novel database of EEG collected during dreaming (DREAM). Visual DNN feature maps were generated for each dream report by converting dream reports to images using text2image AI (Stable Diffusion XL 1.0). A permutation test and representative similarity analysis (RSA) were conducted on both the waking and dream EEG dataset. The correlation between the dream EEG signals and AI-generated dream images was insignificant, though we observed relatively higher decoding for REM dreams than non-REM dreams. Future improvements to the model might include a larger dream database or more specific tuning of the encoding model to capture the differences between waking and dreaming perception.

Topic Area: OTHER