Image classification and reconstruction from low-density EEG
Last page edit: July 17, 2024.
🥵 Reconstruction of perceived images from the brain is currently a hot topic in brain imaging space.
💸💸💸 However, previous approaches have predominantly relied on stationary, costly equipment like MEG, fMRI or high‐density EEG, limiting the real‐world availability and applicability of such projects.
❌ Additionally, several EEG‐based paradigms have utilized artifactual, rather than stimulus‐related information yielding flawed classification and reconstruction results.
🎯 Our goal was to reduce the cost of the decoding paradigm, while increasing its flexibility. Therefore, we investigated whether the classification of an image category and the reconstruction of the image itself is possible from the visually evoked brain activity measured by a portable EEG.
In a nutshell:
🎆 600 images: 20 classes - 30 images/class
👯♂️ 9 subjects
⏰ 6 hours-recording/subject
🧠 8 channel EEG over occipital locations only
💵 <$1K setup cost and <$4K GPU cost for classification and reconstruction
⏳<10 minutes setup time
🦾 paper, code, data, model - documented and available
Main takeaways:
✅ 5 classification models with our setup reaching an average accuracy of 34.4% for 20 image classes on hold‐out test recordings.
✅ After fine‐tuning, we reconstructed images from the test set with a 1000 trial 50‐class top‐1 accuracy of 35.3%.
Selection of good reconstructions of each image class from the base test set. Ground truth (GT) is presented on the left with the reconstructed image to the right for each column, respectively. The presented ground truth images, except for the human face image, were taken from the ImageNet Large Scale Visual Recognition Challenge dataset which permits usage for non- commercial research. The human face image was taken from the Human Faces dataset published under CC0 license.