EEG-DL is a Deep Learning (DL) library written by TensorFlow for EEG Tasks (Signals) Classification. It provides the latest DL algorithms and keeps updated.
The supported models include
| No. | Model | Codes |
|---|---|---|
| 1 | Deep Neural Networks | DNN |
| 2 | Convolutional Neural Networks [Paper] [Tutorial] | CNN |
| 3 | Deep Residual Convolutional Neural Networks [Paper] | ResNet |
| 4 | Thin Residual Convolutional Neural Networks [Paper] | Thin ResNet |
| 5 | Densely Connected Convolutional Neural Networks [Paper] | DenseNet |
| 6 | Fully Convolutional Neural Networks [Paper] | FCN |
| 7 | One Shot Learning with Siamese Networks (CNNs Backbone) [Paper] [Tutorial] |
Siamese Networks |
| 8 | Graph Convolutional Neural Networks [Paper] [Presentation] |
GCN / Graph CNN |
| 9 | Deep Residual Graph Convolutional Neural Networks [Paper] | ResGCN |
| 10 | Densely Connected Graph Convolutional Neural Networks | DenseGCN |
| 11 | Bayesian Convolutional Neural Network via Variational Inference [Paper] |
Bayesian CNNs |
| 12 | Recurrent Neural Networks [Paper] | RNN |
| 13 | Attention-based Recurrent Neural Networks [Paper] | RNN with Attention |
| 14 | Bidirectional Recurrent Neural Networks [Paper] | BiRNN |
| 15 | Attention-based Bidirectional Recurrent Neural Networks [Paper] | BiRNN with Attention |
| 16 | Long-short Term Memory [Paper] | LSTM |
| 17 | Attention-based Long-short Term Memory [Paper] | LSTM with Attention |
| 18 | Bidirectional Long-short Term Memory [Paper] | BiLSTM |
| 19 | Attention-based Bidirectional Long-short Term Memory [Paper] | BiLSTM with Attention |
| 20 | Gated Recurrent Unit [Paper] | GRU |
| 21 | Attention-based Gated Recurrent Unit [Paper] | GRU with Attention |
| 22 | Bidirectional Gated Recurrent Unit [Paper] | BiGRU |
| 23 | Attention-based Bidirectional Gated Recurrent Unit [Paper] | BiGRU with Attention |
| 24 | Attention-based BiLSTM + GCN [Paper] | Attention-based BiLSTM GCN |
| 25 | Transformer [Paper] [Paper] | Transformer |
| 26 | Transfer Learning with Transformer (This code is only for reference!) (You can modify the codes to fit your data.) |
Stage 1: Pre-training Stage 2: Fine Tuning |
- (Under Any Python Environment) Download the EEG Motor Movement/Imagery Dataset via this script.
$ python MIND_Get_EDF.py
- (Under Python 2.7 Environment) Read the .edf files (One of the raw EEG signals formats) and save them into Matlab .m files via this script. FYI, this script must be executed under the Python 2 environment (Python 2.7 is recommended) due to some Python 2 syntax. If using Python 3 environment to run the file, there might be no error, but the labels of EEG tasks would be totally messed up.
I used a conda environment with Python 2.7 using
$ conda create --name EEG2.7 python=2.7in cmd.
Then I ran the python script using the line below and it created a .mat dataset of 10 subjects for every 64 channels. I have applied a Notch Filter and Butterworth Band-pass filter in this process.
$ python Extract-Raw-Data-Into-Matlab-Files.py
- Preprocessed the Dataset via the Matlab and save the data into the Excel files (training_set, training_label, test_set, and test_label) via Preprocess_EEG_Dataset.m with regards to different models. FYI, every lines of the Excel file is a sample, and the columns can be regarded as features, e.g., 4096 columns mean 64 channels X 64 time points. Later, the models will reshape 4096 columns into a Matrix with the shape 64 channels X 64 time points. You should can change the number of columns to fit your own needs, e.g., the real dimension of your own Dataset. Because the matlab script was running out of memory while trying to save the large dataset as an Excel file I modified the script above with a function to save the excels in chunks:
Then I used a python script to concatenate the chunks files into 3 sets and 3 labels .csv files.
-
(Prerequsites) Train and test deep learning models under the Python 3.6 Environment (Highly Recommended) for EEG signals / tasks classification via the EEG-DL library, which provides multiple SOTA DL models.
First, I needed to create another conda environment using
$ conda create --name EEG3.6 python=3.6with TensorFlow GPU version 1.13.1
Python Version: Python 3.6 (Recommended)
TensorFlow Version: TensorFlow 1.13.1
Use the below command to install TensorFlow GPU Version 1.13.1:
$ pip install --upgrade --force-reinstall tensorflow-gpu==1.13.1 --userAfter installing tensorflow-gpu 1.13.1 it came with CUDA 11.2 as default which isn't compatible with my version of tensorflow. So I unninstalled it manually and then nstalled the CUDA Toolkit 10.0 and cuDNN 7.6.x using conda:
conda install -c anaconda cudatoolkit=10.0 cudnn=7.6.5
To train the CNN model on my database I ran the main-CNN.py :
python main-CNN.py
i)Training number #1 was made on a 20-subjects database which resulted in an OOM(Out of Memory). This error typically occurs when your GPU runs out of memory during the model training or evaluation process.
ii)Training number #2. I reduced the subject to 10 and then it trained for 300 iterations (num_epoch = 300 ). The following output is obtained:
Iter 0, Testing Accuracy: 0.47142857, Training Accuracy: 0.51
Iter 0, Testing Loss: 0.74734324, Training Loss: 0.744785
Learning rate is 1e-04
Iter 1, Testing Accuracy: 0.48333332, Training Accuracy: 0.54
Iter 1, Testing Loss: 0.5743346, Training Loss: 0.56509113
Learning rate is 1e-04
Iter 2, Testing Accuracy: 0.4845238, Training Accuracy: 0.56
Iter 2, Testing Loss: 0.5004847, Training Loss: 0.49093938
Learning rate is 1e-04
Iter 3, Testing Accuracy: 0.4952381, Training Accuracy: 0.53
Iter 3, Testing Loss: 0.47017473, Training Loss: 0.46240148
Learning rate is 1e-04
Iter 4, Testing Accuracy: 0.4940476, Training Accuracy: 0.65
Iter 4, Testing Loss: 0.45639592, Training Loss: 0.43720686
Learning rate is 1e-04
Iter 5, Testing Accuracy: 0.48214287, Training Accuracy: 0.54
Iter 5, Testing Loss: 0.45120627, Training Loss: 0.432128
Learning rate is 1e-04
......
......
......
Iter 295, Testing Accuracy: 0.51785713, Training Accuracy: 1.0
Iter 295, Testing Loss: 0.21398115, Training Loss: 0.034377642
Learning rate is 3.125e-06
Iter 296, Testing Accuracy: 0.51785713, Training Accuracy: 1.0
Iter 296, Testing Loss: 0.21302587, Training Loss: 0.0339612
Learning rate is 3.125e-06
Iter 297, Testing Accuracy: 0.5190476, Training Accuracy: 1.0
Iter 297, Testing Loss: 0.21510491, Training Loss: 0.035236362
Learning rate is 3.125e-06
Iter 298, Testing Accuracy: 0.52738094, Training Accuracy: 1.0
Iter 298, Testing Loss: 0.21519342, Training Loss: 0.03485203
Learning rate is 3.125e-06
Iter 299, Testing Accuracy: 0.5107143, Training Accuracy: 1.0
Iter 299, Testing Loss: 0.21810618, Training Loss: 0.03480571
Learning rate is 3.125e-06
Iter 300, Testing Accuracy: 0.50714284, Training Accuracy: 1.0
Iter 300, Testing Loss: 0.21795654, Training Loss: 0.03447302
Learning rate is 3.125e-06
The Summary and Checkpoints Files of the trained model will be saved here:
The output of my main-CNN is showing that the model is learning, but the performance is not ideal. The training accuracy reaches 1.0, which suggests that the model is overfitting the training data. The testing accuracy, on the other hand, is quite low, fluctuating between around 0.47 and 0.52.
For future tries, I will adjust the architecture of the CNN by adding or removing layers, changing the number of filters, or modifying the filter sizes or add regularization methods like L1, L2, or Dropout to prevent overfitting. Another idea is to use a cloud-base service to train the model for more subjects than 10.
At the root of the project, you will see:
├── Download_Raw_EEG_Data
│ ├── Extract-Raw-Data-Into-Matlab-Files.py
│ ├── MIND_Get_EDF.py
│ ├── README.md
│ └── electrode_positions.txt
├── Draw_Photos
│ ├── Draw_Accuracy_Photo.m
│ ├── Draw_Box_Photo.m
│ ├── Draw_Confusion_Matrix.py
│ ├── Draw_Loss_Photo.m
│ ├── Draw_ROC_and_AUC.py
│ └── figure_boxplot.m
├── LICENSE
├── Logo.png
├── MANIFEST.in
├── Models
│ ├── DatasetEEG
│ │ ├── all_data.csv
│ │ ├── all_labels.csv
│ │ ├── test_label.csv
│ │ ├── test_set.csv
│ │ ├── training_label.csv
│ │ └── training_set.csv
│ ├── DatasetAPI
│ │ └── DataLoader.py
│ ├── Evaluation_Metrics
│ │ └── Metrics.py
│ ├── Initialize_Variables
│ │ └── Initialize.py
│ ├── Loss_Function
│ │ └── Loss.py
│ ├── Network
│ │ ├── BiGRU.py
│ │ ├── BiGRU_with_Attention.py
│ │ ├── BiLSTM.py
│ │ ├── BiLSTM_with_Attention.py
│ │ ├── BiRNN.py
│ │ ├── BiRNN_with_Attention.py
│ │ ├── CNN.py
│ │ ├── DNN.py
│ │ ├── DenseCNN.py
│ │ ├── Fully_Conv_CNN.py
│ │ ├── GRU.py
│ │ ├── GRU_with_Attention.py
│ │ ├── LSTM.py
│ │ ├── LSTM_with_Attention.py
│ │ ├── RNN.py
│ │ ├── RNN_with_Attention.py
│ │ ├── ResCNN.py
│ │ ├── Siamese_Network.py
│ │ ├── Thin_ResNet.py
│ │ └── lib_for_GCN
│ │ ├── DenseGCN_Model.py
│ │ ├── GCN_Model.py
│ │ ├── ResGCN_Model.py
│ │ ├── coarsening.py
│ │ └── graph.py
│ ├── __init__.py
│ ├── main-BiGRU-with-Attention.py
│ ├── main-BiGRU.py
│ ├── main-BiLSTM-with-Attention.py
│ ├── main-BiLSTM.py
│ ├── main-BiRNN-with-Attention.py
│ ├── main-BiRNN.py
│ ├── main-CNN.py
│ ├── main-DNN.py
│ ├── main-DenseCNN.py
│ ├── main-DenseGCN.py
│ ├── main-FullyConvCNN.py
│ ├── main-GCN.py
│ ├── main-GRU-with-Attention.py
│ ├── main-GRU.py
│ ├── main-LSTM-with-Attention.py
│ ├── main-LSTM.py
│ ├── main-RNN-with-Attention.py
│ ├── main-RNN.py
│ ├── main-ResCNN.py
│ ├── main-ResGCN.py
│ ├── main-Siamese-Network.py
│ └── main-Thin-ResNet.py
├── NEEPU.png
├── Preprocess_EEG_Data
│ ├── For-CNN-based-Models
│ │ └── make_dataset.m
│ ├── For-DNN-based-Models
│ │ └── make_dataset.m
│ ├── For-GCN-based-Models
│ │ └── make_dataset.m
│ ├── For-RNN-based-Models
│ │ └── make_dataset.m
│ └── For-Siamese-Network-One-Shot-Learning
│ └── make_dataset.m
├── README.md
├── Saved_Files
│ └── README.md
├── requirements.txt
└── setup.py
If you find our library useful, please considering citing our papers in your publications. We provide a BibTeX entry below.
@article{hou2022gcn,
title = {{GCNs-Net}: A Graph Convolutional Neural Network Approach for Decoding Time-Resolved EEG Motor Imagery Signals},
author = {Hou, Yimin and Jia, Shuyue and Lun, Xiangmin and Hao, Ziqian and Shi, Yan and Li, Yang and Zeng, Rui and Lv, Jinglei},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
volume = {},
number = {},
pages = {1-12},
year = {Sept. 2022},
doi = {10.1109/TNNLS.2022.3202569}
}
@article{hou2020novel,
title = {A Novel Approach of Decoding EEG Four-class Motor Imagery Tasks via Scout {ESI} and {CNN}},
author = {Hou, Yimin and Zhou, Lu and Jia, Shuyue and Lun, Xiangmin},
journal = {Journal of Neural Engineering},
volume = {17},
number = {1},
pages = {016048},
year = {Feb. 2020},
publisher = {IOP Publishing},
doi = {10.1088/1741-2552/ab4af6}
}
@article{hou2022deep,
title = {Deep Feature Mining via the Attention-Based Bidirectional Long Short Term Memory Graph Convolutional Neural Network for Human Motor Imagery Recognition},
author = {Hou, Yimin and Jia, Shuyue and Lun, Xiangmin and Zhang, Shu and Chen, Tao and Wang, Fang and Lv, Jinglei},
journal = {Frontiers in Bioengineering and Biotechnology},
volume = {9},
year = {Feb. 2022},
url = {https://www.frontiersin.org/article/10.3389/fbioe.2021.706229},
doi = {10.3389/fbioe.2021.706229},
ISSN = {2296-4185}
}
@article{Jia2020AttentionGCN,
title = {Attention-based Graph {ResNet} for Motor Intent Detection from Raw EEG signals},
author = {Jia, Shuyue and Hou, Yimin and Lun, Xiangmin and Lv, Jinglei},
journal = {arXiv preprint arXiv:2007.13484},
year = {2022}
}Our papers can be downloaded from:
- A Novel Approach of Decoding EEG Four-class Motor Imagery Tasks via Scout ESI and CNN
Codes and Tutorials for this work can be found here.
Overall Framework:
Proposed CNNs Architecture:
- GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals
Slides Presentation for this work can be found here.
- Deep Feature Mining via Attention-based BiLSTM-GCN for Human Motor Imagery Recognition
Slides Presentation for this work can be found here.
The library was created and open-sourced by Shuyue Jia, supervised by Prof. Yimin Hou, at the School of Automation Engineering, Northeast Electric Power University, Jilin, Jilin, China.