Unofficial minimal re-implementation of the paper "Brain Decoding: Toward Real-Time Reconstruction of Visual Perception" from FAIR, Meta
-
🚧 Preprocess and CLIP training are implemented but unable to reproduce the top-5 accuracy reported in the paper. (paper: 10%, this repo: 3.1%, where chance is 0.21% for large test set)
-
Currently using the preprocessed data in Hebart et al., 2023, whose pipeline is slightly different from the Meta paper.
-
Currently only CLIP-Vision encoding is supported.
-
-
🚧 Generation module to be implemented.
-
Download the THINGS-MEG dataset from here and place it where you want.
-
Download the THINGS dataset from here and place it where you want.
-
Run preprocessing.
python preproc.py thingsmeg_root={path to the THINGS-MEG dataset directory with / at the end} things_root={path to the THINGS dataset directory with / at the end}- Run CLIP training.
python train_clip.py thingsmeg_root={path to the THINGS-MEG dataset directory with / at the end} things_root={path to the THINGS dataset directory with / at the end}