CN115316955A - A lightweight and fast decoding method for motor imagery EEG signals - Google Patents

Abstract

The invention discloses a light-weight and fast method for decoding electroencephalographic signals of motor imagery. The steps are as follows: S1. Build a deep learning model, wherein the deep learning model includes a spatiotemporal convolution module, a pooling module and a fully connected module; The spatiotemporal convolution module consists of a temporal convolution layer for reducing the amount of trainable parameters and a spatial depth convolution layer for reducing channel connections; the pooling module stacks pooling layers to reduce the dimension of the model and reduce complexity degree; the fully connected module is used for final classification; S2, preprocess the original EEG signal, and then use a deep learning model to classify and decode the preprocessed EEG signal. The present invention can obtain better decoding performance with less trainable parameters, and maintains a relative balance between classification accuracy and model complexity.

Description

A Lightweight and Fast Motor Imagery EEG Signal Decoding Method

technical field

The invention relates to the technical field of electroencephalogram signal analysis, in particular to a lightweight and fast motor imagery electroencephalogram signal decoding method.

Background technique

In motor imagery EEG decoding tasks, the most typical machine learning algorithms for manually extracting features are common spatial patterns, filter bank common spatial patterns, short-time Fourier transform, and principal component analysis. The basic idea of the CSP algorithm is to find a spatial filter that maximizes this distance in the motor imagery EEG four-category task. Similarly, the FBCSP method is an extension of the CSP technique and is often used in EEG decoding tasks. The algorithm selects and classifies features by extracting the optimal spatial features through a set of bandpass filters. At the same time, in past motor imagery EEG research, classification methods such as linear classifiers, support vector machines, multi-layer perceptrons, and random forests are often used. Although these methods have achieved good results in motor imagery EEG decoding tasks, they divide feature extraction and classification into two stages.

However, traditional machine learning algorithms have shortcomings in classification accuracy due to the bias caused by the need to manually extract features. With the wide application of deep learning methods in various fields and the ability to efficiently extract more meaningful features and achieve better results, convolutional neural networks have also emerged in EEG classification. In contrast, deep learning combines feature extraction and classification into a single step. According to the characteristics of the time series of EEG signals, the long-short-term memory network has the ability to extract time features, especially in the processing of time series, which has a unique advantage and has been extensively understood in the fields of speech recognition and natural language processing. By introducing a gate function in the unit structure, LSTM can solve the problem of data that ordinary RNN cannot learn due to too much relevant input information. The LSTM structure contains two important branches of storage unit and nonlinear gating unit. At present, some researches have tried to use BLSTM to memorize the relationship changes of two channels in a specific time to extract time-domain features and obtain good classification results. Surprisingly, experiments show that convolutional neural networks can better extract spatiotemporal and frequency domain features from motor imagery EEG. It is worth noting that in the research contributions that have appeared, most scholars use CNNs or fusion models to extract motor imagery EEG signals, but do not consider resource consumption and model computational complexity, especially when the complexity is high The fusion model of . takes up more trainable parameters, and their scheme does not visualize the extracted features and convolution kernel output.

Contents of the invention

The object of the present invention is to provide a lightweight and fast motor imagery EEG signal decoding method. The present invention can obtain better decoding performance with fewer trainable parameters, and maintains a relative balance between classification accuracy and model complexity.

Technical solution of the present invention: a light-weight and fast motor imagery EEG signal decoding method is carried out as follows:

S1, build a deep learning model, the deep learning model includes a spatiotemporal convolution module, a pooling module and a fully connected module; the spatiotemporal convolution module is composed of a temporal convolution layer for reducing the amount of trainable parameters and a channel for reducing A connected spatial depth convolutional layer is formed; the pooling module is a stacked pooling layer to reduce the dimension of the model and reduce the complexity; the fully connected module is used for the final classification;

S2. Preprocessing the original EEG signal, and then using the deep learning model to classify and decode the preprocessed EEG signal.

个时间卷积核之间的一维卷积公式如下： In the light-weight and fast motor imagery EEG signal decoding method described above, the spatio-temporal convolution module is used to extract the spatial and spectral features of the EEG input; the temporal convolution layer defines the kernel of the temporal convolution kernel through a parameterized function value, so that the kernel value description of the temporal convolutional layer belongs to the subset of the temporal filter, so as to reduce the amount of trainable parameters and reduce resource consumption, where the EEG signal and the first

The one-dimensional convolution formula between two temporal convolution kernels is as follows:

；

;

是第

个电极信号与第

个时间卷积核之间的一维卷积；

是总数；

表示第

个电极信号，

代表一维卷积沿着时间维度的过滤器的长度，

对应着时间卷积 核的数量；

是中间量； In the formula,

is the first

The first electrode signal and the first

One-dimensional convolution between temporal convolution kernels;

is the total;

Indicates the first

electrode signal,

Represents the length of the filter of the one-dimensional convolution along the time dimension,

Corresponding to the number of temporal convolution kernels;

is the intermediate quantity;

定义时间卷积核的内核值，频域范围内带通滤波的振幅表达 为： by parameterizing the function

Define the kernel value of the time convolution kernel, and the amplitude of the bandpass filter in the frequency domain is expressed as:

；

;

是频率；

是第

个带通滤波的截止频率的劣值；

是是第

个带通滤波 的截止频率的优值； In the formula:

is the frequency;

is the first

The inferior value of the cutoff frequency of a bandpass filter;

is the first

The optimal value of the cutoff frequency of a bandpass filter;

The magnitude of bandpass filtering in the time domain is as follows:

；

;

为正弦函数； In the formula:

is a sine function;

A deep spatial convolutional layer with filter size (1, 65) and 32 bandpass temporal filters is stacked after the temporal convolutional layer to learn spectral and spatial features.

In the aforementioned lightweight and fast motor imagery EEG signal decoding method, the pooling module takes the sampled signal as input with a size of (1, 109) and a step size of (1, 23) and investigates specific hyperparameters, and The output of the pooling module is transformed using a flattened layer and connected to a fully connected layer.

映射到真实的类以表示卷积的处理过程，函数中

代表 电极数；

为时间步长，

为总的输入数据；由softmax函数将输出转换成给定输出的每个 类标签的特定受试者条件概率表达为： In the aforementioned light-weight and fast motor imagery EEG signal decoding method, the fully connected module uses the softmax function to classify, uses the supervised learning algorithm to train hyperparameters, and generates the real output values corresponding to the four classifications of motor imagery; through the function

Mapped to a real class to represent the processing of convolution, in the function

represents the number of electrodes;

is the time step,

is the total input data; the output transformed by the softmax function into the subject-specific conditional probability of each class label of the given output is expressed as:

是标签的条件概率；

为第

个试验的输入数据，

；

表示 权重与偏置函数的参数；

为映射函数； In the formula:

is the conditional probability of the label;

for the first

The input data of a test,

;

Indicates the parameters of the weight and bias functions;

is the mapping function;

The sum of per-sample losses is minimized by assigning a high probability to the correctly output label, which is shown as:

；

;

为每次试验的输出类，

的值由负对数可能性决定，其具体计算表达式 如下： In the formula:

is the output class for each trial,

The value of is determined by the negative logarithmic likelihood, and its specific calculation expression is as follows:

。

.

In the aforementioned light-weight and fast motor imagery EEG signal decoding method, in step S2, the processing of the original EEG signal is to convert the original EEG signal into a channel of the cross-time sample C*T matrix, and then between 8Hz and 30Hz Filter with a bandpass filter, and then perform batch normalization on the filtered EEG signals.

The aforementioned lightweight and fast motor imagery EEG signal decoding method, the original EEG signal is expressed as:

；

;

代表第

个记录的试验总数，运动想象信号的第

次试验与输出类紧密相 连；对于一个给定的试验

，第

次输入试验的输出映射为输出类标签

，以分别指代运动想 象四分类任务。 In the formula:

On behalf of

The total number of recorded trials, the first number of motor imagery signals

Trials are closely linked to the output class; for a given trial

, No.

The output of the input trials is mapped to the output class label

, to refer to the four categories of motor imagery tasks.

The aforementioned lightweight and fast motor imagery EEG signal decoding method uses the BrainDecode framework to realize the training of the deep learning model; the original EEG signal is divided into 750 samples as the input of the deep learning model, and the value of each batch is initially set When the epochs execution standard exceeds the preset maximum value, the training is terminated; at the same time, the early stopping strategy is combined with the Adam optimizer to improve the classification accuracy.

The aforementioned light-weight and fast motor imagery EEG signal decoding method uses batch normalization regularization method to prevent over-fitting phenomenon.

Compared with the prior art, the present invention presents an efficient deep learning module for motor imagery EEG decoding, which can obtain better decoding performance with fewer trainable parameters. A relative balance is maintained between classification accuracy and model complexity. The invention greatly improves the motor imagery EEG decoding accuracy by taking variable parameters into consideration. According to the experimental results of specific EEG subjects, the present invention obtains an average classification accuracy of about 78.42%, which is more robust than many other SOA algorithms. Furthermore, the present invention can solve common problems based on EEG in neuroscience, paving the way for establishing practical clinical applications.

Description of drawings

Fig. 1 is a schematic diagram of a deep learning model of the present invention;

Fig. 2 is a schematic flow sheet of the present invention;

2a的训练集与测试集单次实验范式； Figure 3 is the data set BCI competition

2a training set and test set single experiment paradigm;

2a数据集上的所有受试者之间的不同输出类的平 均混淆矩阵； Figure 4 shows the FBCSP algorithm in the BCI competition

Average confusion matrix for different output classes across all subjects on the 2a dataset;

2a数据集上的所有受试者之间的不同输 出类的平均混淆矩阵； Figure 5 shows the Mixed-ConvNet model in the BCI competition

Average confusion matrix for different output classes across all subjects on the 2a dataset;

2a数据集上不同模型之间的解码精度与标准误差的性能比 较； Figure 6 is in the BCI competition

Performance comparison of decoding accuracy and standard error between different models on the 2a dataset;

Fig. 7 is a comparison result between the depth model of the present invention and other two SOA algorithms in different performance measurement indexes.

Detailed ways

The following examples further illustrate the present invention, but are not as the basis for limiting the present invention.

Embodiment 1: A light-weight and fast motor imagery EEG signal decoding method, as shown in Figure 1, is carried out as follows:

S1. Obtain the original EEG signal and build a deep learning model. The deep learning model includes a spatiotemporal convolution module, a pooling module, and a fully connected module; the spatiotemporal convolution module is composed of temporal convolutions used to reduce the amount of trainable parameters layer and a spatial depth convolution layer for reducing channel connections; the pooling module is a stacked pooling layer to reduce the dimension of the model and reduce complexity; the fully connected module is used for the final classification;

S2. Preprocessing the original EEG signal, and then using the deep learning model to classify and decode the preprocessed EEG signal.

Specifically, the original EEG signal is expressed as:

；

;

代表第

个记录的试验总数，运动想象信号的第

次试验与输出类紧密相 连；对于一个给定的试验

，

；其中

代表电极数；

为时间步长，

为总 的输入数据，第

次输入试验的输出映射为输出类标签

，以分别指代运动想象四分类任 务。 In the formula:

On behalf of

The total number of recorded trials, the first number of motor imagery signals

Trials are closely linked to the output class; for a given trial

,

;in

represents the number of electrodes;

is the time step,

For the total input data, the

The output of the input trials is mapped to the output class label

, to refer to the four categories of motor imagery tasks.

As shown in Figure 2, the preprocessing of the original EEG signal is to convert the original EEG signal into a channel of a C*T matrix of samples across time, and then apply a bandpass filter between 8Hz and 30Hz, and then filter the filtered The EEG signals were batch normalized. In the preprocessing process, noise and artifacts are filtered for the original EEG signal, and three EOG bad channels are eliminated. In the present invention, random initialization is performed from a uniform distribution of frequency bands of interest in [0,38] or [4,38] Hz, and these frequencies are updated within this range.

In this embodiment, the global hyperparameters used by each layer of the deep learning model, the output shape, the number of parameters and the detailed information of the activation function are described in Table 1-3.

Table 1 is the input of the deep learning model and the spatio-temporal convolution module

Table 2 is the pooling module

Table 3 is the fully connected module

表示混合卷积神经网 络的dropout rate. 而且，

意味着EEG信号特征分类的数量。与此同时，采样频率为 250Hz，实验还使用了ELU指数线性单元激活函数。 Tables 1-3 have shown the detailed hyperparameters of the deep learning model of the present invention. Tables 1-3 focus on the main modules of the present invention's ingenious application of the time sinusoidal convolution layer model, the convolution or pooling layers used by each module, the global hyperparameters of each layer, the output shape, and the trainable parameters of each layer. parameters and the activation function used. Where C represents the number of channels, T represents the number of samples, K and F represent the number and size of convolution kernels, S and P represent the step size and pooling size, D represents the multiplier in the depth convolution layer, and m represents the use of BatchNorm parameter values for the regularization method, and

Indicates the dropout rate of the hybrid convolutional neural network. And,

means the number of EEG signal feature categories. At the same time, the sampling frequency is 250Hz, and the experiment also uses the ELU exponential linear unit activation function.

In the field of EEG signals and processing, CNNs are often used to deal with high-latitude data problems, and all neurons of the convolution window, that is, the kernel size, are given bias values and weights. Instead, CNNs subtly deepen the idea of weight sharing, learning a set of weights and a single bias value to apply to hidden layer neurons, the mathematical method of this process is expressed as follows:

；

;

表示隐藏层中第

个过滤器的第

个神经元的激活输出，

对应着使用的 激活函数，

指代过滤器

的共享偏置值，

是内核大小，

是共享权重向量，

意旨预知 神经元的输出向量，

表示转置操作。 in,

represents the hidden layer

filter's

The activation output of a neuron,

Corresponding to the activation function used,

Referral filter

The shared bias value of ,

is the kernel size,

is the shared weight vector,

Intention to predict the output vector of the neuron,

Represents a transpose operation.

个时间卷积 核之间的一维卷积公式如下： In order to extract the spatial and spectral features of the EEG input, temporal and spatial convolutional layers are designed respectively. The temporal convolution layer defines the kernel value of the temporal convolution kernel through a parameterized function, so that the kernel value description of the temporal convolution layer belongs to a subset of temporal filters, so as to reduce the amount of trainable parameters and reduce resource consumption. electrical signal and the

The one-dimensional convolution formula between two temporal convolution kernels is as follows:

；

;

是第

个电极信号与第

个时间卷积核之间的一维卷积；

是总数；

表示第

个电极信号，

代表一维卷积沿着时间维度的过滤器的长度，

对应着时间卷积 核的数量；

是中间量； In the formula,

is the first

The first electrode signal and the first

One-dimensional convolution between temporal convolution kernels;

is the total;

Indicates the first

electrode signal,

Represents the length of the filter of the one-dimensional convolution along the time dimension,

Corresponding to the number of temporal convolution kernels;

is the intermediate quantity;

定义时间卷积核的内核值，为了描述频域范围内的带通滤波， 其振幅表达为： by parameterizing the function

Define the kernel value of the time convolution kernel. In order to describe the bandpass filtering in the frequency domain, its amplitude is expressed as:

；

;

是频率；

是第

个带通滤波的截止频率的劣值；

是是第

个带通滤波 的截止频率的优值； In the formula:

is the frequency;

is the first

The inferior value of the cutoff frequency of a bandpass filter;

is the first

The optimal value of the cutoff frequency of a bandpass filter;

This calculation process reduces the amount of trainable parameters for each kernel of the time convolutional layer in the time domain. Similarly, the amplitude of the bandpass filter in the time domain is as follows:

；

;

为正弦函数； In the formula:

is a sine function;

A deep spatial convolutional layer with filter size (1, 65) and 32 bandpass temporal filters is stacked after the temporal convolutional layer to learn spectral and spatial features. In order to prevent the occurrence of overfitting, the present invention flexibly applies the batch normalization regularization technique to the feature map dimension.

The output of the space-time convolution module is used as the input of the pooling module. The pooling layer reduces the dimension of each feature map while retaining the original meaningful information. The spatial pooling can be divided into several types of subsampling and downsampling. The most famous of these are the max pooling and average pooling methods. The present invention sets a batch normalization technique after each convolutional layer and applies a Dropout method after a pooling layer. In the average pooling module, we sample the signal with a size of (1, 109) and a stride of (1, 23) as input and investigate specific hyperparameters.

Finally, the deep learning model also makes full use of the flattening layer to transform the output of the above modules and connect to the fully connected layer, resulting in an extracted 1D feature vector. The fully connected module uses the softmax function to classify, utilizes a supervised learning algorithm to train hyperparameters, and produces the real output values corresponding to the four classifications of motor imagery; through the function mapping to the real class to represent the convolution process, the softmax function will The output transforms into the subject-specific conditional probability of each class label for a given output expressed as:

是标签的条件概率；

为第

个试验的输入数据，；

表示权重与偏置函数 的参数，

是指数函数；

为映射函数；

代表电极数；

为时间步长，

为总的输入数据 In the formula:

is the conditional probability of the label;

for the first

The input data of a test,;

Indicates the parameters of the weight and bias functions,

is an exponential function;

is the mapping function;

represents the number of electrodes;

is the time step,

For the total input data

The sum of per-sample losses is minimized by assigning a high probability to the correctly output label, which is shown as:

；

;

为每次试验的输出类，

的值由负对数可能性决定，其具体计算表达式 如下： In the formula:

is the output class for each trial,

The value of is determined by the negative logarithmic likelihood, and its specific calculation expression is as follows:

。

.

与预处理过的脑电信号

为例， 当

与

分别是矩阵

和

的第i行，那么归一化操作显而易见地由如下公式表示：The present invention skillfully utilizes the BrainDecode framework proposed by Schirrmeister, which is completely used for processing motor imagery EEG classification tasks, to realize the training of Mixed-ConvNet. Since the EEG varies among subjects and changes with time, the present invention sets a series of variable hyperparameters to search for the best parameter setting value for each subject. The 750 samples of the original 22 EEG channel signals are used as the input of the model. At first, the value of each batch is set to 58. When the epochs execution standard exceeds the preset maximum value, the training is terminated. At the same time, the early stopping strategy is combined with the Adam optimizer to improve the classification accuracy of the model, and the frequency band with a low frequency of 0 or 4 Hz is randomly selected in the distribution of interest. In order to make the model training more stable and reliable, the batch normalization regularization technology is to prevent The key to the overfitting phenomenon and set its value to 0.99. raw EEG signal

with preprocessed EEG signals

For example, when

and

are the matrices

and

The i-th line of , then the normalization operation is obviously expressed by the following formula:

；

;

与

分别代表平均值与标准差操作。当正则化完输出结果之后， 巧妙运用激活函数并且本研究使用的函数为指数线性单元与线性激活函数。其中，ELU的计 算公式可以表达为： in,

and

represent the mean and standard deviation operations, respectively. After regularizing the output results, the activation function is cleverly used and the functions used in this study are exponential linear units and linear activation functions. Among them, the calculation formula of ELU can be expressed as:

（9）

(9)

The dropout technique randomly sets the output of the previous layer to 0 during each training update and this model sets a dropout rate of 0.5 after the average pooling layer.

In order to verify the accuracy of the classification decoding of the present invention, the present invention compares the experimental results with some SOA algorithms on the data set BCI competition 2a and visualizes the convolution kernel itself, and visualizes the results after convolution to help understand the convolution kernel. The key role of deeply identifying which parts of the image classification problem through the heat map is the visualization of class activation and visualization of hidden layers and features.

1. Dataset description

As one of the most common data sets in the field of motor imagery EEG classification, the BCI competition 2a was collected from 9 subjects experimentally by the University of Graz using 22 EEG electrodes (excluding 3 EOG channels) at a sampling frequency of 250Hz A publicly available dataset. Special reminder, in the experimental research of the present invention, because the training data of the 4th subject of this original data set is missing part of the test samples, so the other eight subjects except the 4th subject are used to carry out experiment. After removing the three additional EOG channels, noise and artifacts were filtered using a bandpass filter at 0.5 to 100 Hz. The subjects in this data set performed four different motor imagery tasks of left hand, right hand, feet and tongue movement. The training set and the test set completed the experimental records on two different days. Each set of experiments contained 288 trials, of which Each type of task contains 72 times, the time of motor imagery is 4s, and each trial generates 750 sample points. The experimental paradigm is shown in Figure 3, including the "Beep" sound and the 2-second time cross of "Fixation cross". , the next 1.25 seconds left, right, up and down "Cue" arrow prompts (respectively corresponding to the left hand, right hand, feet and tongue of motor imagery), 3 to 6 seconds is the key "Motor imagery" motor imagery time And the last 6 to 7.5 seconds is the "Break" short rest period after the cross hour. Previous attempts to classify data related to motor imagery have shown that the individual subjects of the data set have relatively large individual differences, and the decoding effect of measuring them on the model also varies from person to person. Moreover, in order to implement the early stopping strategy in the first step of the optimization process, the training set is further divided into a small training set and a validation set, where the validation set accounts for 20% of the total training set.

2. Performance evaluation index

To evaluate the decoding performance of the proposed method, the test data is retained and ground truth predictions are made on the labeled data. The experiment uses the following performance indicators to evaluate the proposed model, among which the classification accuracy is the most frequently adopted measurement indicator, and the prediction value and recall rate are also important methods to identify good or bad algorithm structures. The formulas of these three common indicators are as follows :

；

;

；

;

；

;

是真正例，

是真负例，

是假正例，

是假负例。除此以外，科恩卡 kappa值也是较常使用的计算指标，其公式展示如下： in

is a real example,

is a true negative example,

is a false positive,

is a false negative. In addition, Konka's kappa value is also a commonly used calculation indicator, and its formula is shown as follows:

；

;

代表观察到的一致性的比例精度，

暗指随机猜测值的概率或者精度。

分数值是最终经常采纳的指标，需要结合预测值与查全率两者共同计算得知，具体公式 表达如下： in,

represents the scaled precision of the observed agreement,

The probability or precision of a random guess value is implied.

The score value is the index that is often adopted in the end. It needs to be calculated by combining the predicted value and the recall rate. The specific formula is expressed as follows:

。

.

3. Compare the results

2a上相比较的实 验对比结果详细记录在表4中（忽略了部分训练数据缺失的受试者4）。 The deep learning model proposed by the present invention competes with similar SOA algorithms in the data set BCI competition

The results of the experimental comparisons on 2a are detailed in Table 4 (subject 4 with missing part of the training data is ignored).

Table 4

In order to improve persuasiveness, the experiment selected the traditional machine learning algorithm FBCSP and deep learning ShallowConvNet, C2CM, DeepConvNet, WaSFConvNet, CM-ConvNet, AMSI-ConvNet, Proposed-model models, and implemented them on this four-category motor imagery EEG dataset A series of comparative experiments were carried out. The filter bank common spatial mode is the most classic traditional manual feature extraction algorithm of motor imagery EEG, and combined with the linear discriminant analysis classifier to complete the task of the four-category data set, the decoding accuracy of 68.59% was obtained on this data set. Fig. 4 and Fig. 5 have respectively given the average confusion matrix of the different output classes between FBCSP algorithm and all subjects of the present invention respectively, all have demonstrated the average classification accuracy of " Left " and " Right " motor imagery task in the figure Classified higher than both "Foot" and "Tongue". It is worth noting that the proposed Mixed-ConvNet model is higher than the FBCSP+rLDA structure in the four categories of motor imagery, especially in the two categories of "Foot" and "Tongue". electrical characteristics. It is important to note that this structure is about 24% higher than the traditional FBCSP algorithm in terms of "Tongue" classification and achieves an average decoding accuracy of 80.4%, which proves that the proposed Mixed-ConvNet decoding model is highly efficient in feature extraction. high.

In addition, it is surprisingly found from Table 4 that Shallow-ConvNet, the most widely used shallow model in the field of motor imagery EEG, and the two-dimensional convolutional scene model C2CM, which is similar to most two-dimensional convolutional structures, were obtained on this data set. The average classification accuracies of 74.05% and 75.43% were achieved. At the same time, the Deep-ConvNet model built by Schirrmeister et al. stacks several layers of convolution and pooling layers on the basis of the shallow structure. On the contrary, it not only increases the number of trainable parameters of the model but also has the same In comparison, the average classification accuracy is reduced by about 4%. It is worth mentioning that the model WaSF-ConvNet, which uses the wavelet kernel to directly consider the spectral power modulation of the EEG signal, contains two specific convolutional layers, followed by a pooling layer, plus a dropout layer With five complete parts of the fully connected layer, the number of parameters is significantly reduced in the process of learning features and only a relatively low accuracy of 68.96% is obtained on this four-category EEG dataset. Furthermore, the AMSI-ConvNet structure proposes two variants of the space-time convolution module to verify the feasibility of the decoding performance. Further benchmark tests show that the average classification of 76.27% has been obtained on this EEG competition dataset. precision. Compared with other baseline algorithms, the average decoding accuracy of the model proposed by the present invention is about 10% higher than the traditional FBCSP and its standard deviation is relatively small. It is observed that the structure has higher robustness and stability.

2a数据 集上的解码精度实验对比如图6所示，其中图6中反映此数据集上八个受试者之间的性能的 误差线代表标准差（Std）。图中观察发现，Mixed-ConvNet模型的性能优于其他SOA方法，通 过充分使用正弦时间卷积模块，此算法比Shallow-ConvNet与Deep-ConvNet模型分别高于 达到4%以及8%精度，并且此模型在该数据集上的标准差也小于其他SOA算法，实验论证表明 时空卷积模块的确提升了脑电解码性能。从单独受试者方向考虑模型的性能，展示的模型 与其他SOA算法相比较而言，特定受试者1、5和9在此结构上性能表现最佳，分别达到了 88.54%、71.87%以及85.76%平均解码精度，这完全可以验证出提出的结构足以运用于振荡 运动想象脑电解码。表3充分展示了BCI竞赛

2a数据集上采用特定受试者方法的几种同类 SOA运动想象算法与提出的模型的每位受试者之间的kappa值结果比较。Further, the deep learning model proposed by the present invention competes with other baseline algorithms in the BCI competition

The experimental comparison of decoding accuracy on the 2a dataset is shown in Fig. 6, where the error bars in Fig. 6 reflecting the performance among eight subjects on this dataset represent the standard deviation (Std). It is observed in the figure that the performance of the Mixed-ConvNet model is better than other SOA methods. By fully using the sinusoidal time convolution module, this algorithm is 4% and 8% higher than the Shallow-ConvNet and Deep-ConvNet models, respectively, and this The standard deviation of the model on this data set is also smaller than that of other SOA algorithms. The experimental demonstration shows that the spatio-temporal convolution module does improve the performance of EEG decoding. Considering the performance of the model from the perspective of individual subjects, compared with other SOA algorithms, the performance of specific subjects 1, 5 and 9 on this structure is the best, reaching 88.54%, 71.87% and The average decoding accuracy is 85.76%, which can fully verify that the proposed structure is sufficient for oscillatory motor imagery EEG decoding. Table 3 fully demonstrates the BCI competition

Comparison of kappa-value results per subject between several comparable SOA motor imagery algorithms with subject-specific methods on the 2a dataset and the proposed model.

table 5

2a数据集上的Precision、Recall值以及F1分数的结果比较： From Table 5, it is not difficult to find that the proposed model has obtained the optimal performance evaluation index kappa value and reached 0.72. The highest kappa value was obtained in . In Figure 7, the model Mixed-ConvNet (deep learning model) proposed by the present invention and the traditional algorithm FBCSP and the typical shallow structure Shallow-ConvNet three models are shown in different performance metrics of classification accuracy, kappa value and F1 score value. The comparison results between (the histograms of each evaluation index, from top to bottom are Shallow-ConvNet, FBCSP and Mixed-ConvNet), it is not difficult to distinguish from the figure, the model proposed by the present invention is in the three evaluation indexes. Performs better than other SOA algorithms (ie FBCSP). Moreover, with the help of Precision and Recall values and other performance metrics to fully verify the competitiveness of the proposed model, Table 6 briefly illustrates the proposed model's competition with several other baseline algorithms BCI

Comparison of Precision, Recall values and F1 scores on the 2a dataset:

Table 6

Careful observation of the table shows that the overall average of the proposed lightweight structure is better than other comparative SOA algorithms in these three performance indicators.

Considering the complexity of the model, the detailed comparison results between the deep learning structure proposed by the present invention and the amount of trainable parameters, training time and average decoding accuracy used by several other baseline algorithms on the BCI competition 2a data set are shown in Table 7 Shown:

Table 7

It can be seen from Table 7 that in the comparison of several shallow and deep fusion algorithms, the channel mapping mixed-size convolutional neural network combined with the amplitude perturbation data enhancement method uses a large number of trainable hyperparameters and reaches 8.36×10 ^5. Under the premise that the model structure is relatively complex, it consumes a lot of resources and increases the calculation cost. However, the average decoding accuracy on this data set does not exceed the proposed model. In addition, the deep convolutional model Deep-ConvNet, the classic shallow structure Shallow-ConvNet, and the hybrid stacked network AMSI-ConvNet have the best performance in terms of the number of trainable parameters, the average training time of subjects (hh:mm:ss) and the average decoding accuracy. The balance between the proposed algorithm is not better. Our proposed model only introduces the minimum number of trainable parameters 8324 and achieves the highest average classification accuracy. The average training time between eight subjects on this dataset takes less than 10 minutes. This is enough to prove that the proposed lightweight structure can be applied to real brain-computer interface application scenarios.

To sum up, the present invention proposes and demonstrates an efficient deep learning module for motor imagery EEG decoding. The deep learning module can obtain better decoding performance with fewer trainable parameters, and the classification accuracy A relative balance is maintained between model complexity. The invention greatly improves the motor imagery EEG decoding accuracy by taking variable parameters into consideration. According to the experimental results of specific EEG subjects, the present invention obtains an average classification accuracy of about 78.42%, which is more robust than many other SOA algorithms. Furthermore, the present invention can solve common problems based on EEG in neuroscience, paving the way for establishing practical clinical applications.

Claims (8)

1. A lightweight and fast motor imagery EEG signal decoding method, characterized in that: proceed as follows: S1, build a deep learning model, the deep learning model includes a spatiotemporal convolution module, a pooling module and a fully connected module; the spatiotemporal convolution module is composed of a temporal convolution layer for reducing the amount of trainable parameters and a channel for reducing A connected spatial depth convolutional layer is formed; the pooling module is a stacked pooling layer to reduce the dimension of the model and reduce the complexity; the fully connected module is used for the final classification; S2. Preprocessing the original EEG signal, and then using the deep learning model to classify and decode the preprocessed EEG signal. 2. The light-weight and fast motor imagery EEG signal decoding method according to claim 1, characterized in that: the space-time convolution module is used to extract the space and spectral features of the EEG input; the time convolution layer passes A parameterized function defines the kernel value of the time convolution kernel, so that the kernel value of the time convolution layer describes a subset of the time filter, so as to reduce the amount of trainable parameters and reduce resource consumption. The EEG signal and the first

The one-dimensional convolution formula between two temporal convolution kernels is as follows:

; In the formula,

is the first

The first electrode signal and the first

One-dimensional convolution between temporal convolution kernels;

is the total;

Indicates the first

electrode signal,

Represents the length of the filter of the one-dimensional convolution along the time dimension,

Corresponding to the number of temporal convolution kernels;

is the intermediate quantity; by parameterizing the function

Define the kernel value of the time convolution kernel, and the amplitude of the bandpass filter in the frequency domain is expressed as:

; In the formula:

is the frequency;

is the first

The inferior value of the cutoff frequency of a bandpass filter;

is the first

The optimal value of the cutoff frequency of a bandpass filter; The magnitude of bandpass filtering in the time domain is as follows:

; In the formula:

is a sine function; A deep spatial convolutional layer with filter size (1, 65) and 32 bandpass temporal filters is stacked after the temporal convolutional layer to learn spectral and spatial features. 3. The light-weight and fast motor imagery EEG signal decoding method according to claim 1, characterized in that: the pooling module will sample The signal is taken as input and specific hyperparameters are investigated, and the output of the pooling module is transformed using a flattening layer and connected to a fully connected layer. 4. The light-weight and fast motor imagery EEG signal decoding method according to claim 3, characterized in that: the fully connected module uses a softmax function to classify, utilizes a supervised learning algorithm to train hyperparameters, and produces four classifications of motor imagery The corresponding real output value; through the function

Mapped to a real class to represent the processing of convolution, in the function

represents the number of electrodes;

is the time step,

is the total input data; the output transformed by the softmax function into the subject-specific conditional probability of each class label of the given output is expressed as:

In the formula:

is the conditional probability of the label;

for the first

The input data of a test,

;

Indicates the parameters of the weight and bias functions,

is an exponential function;

is the mapping function; The sum of per-sample losses is minimized by assigning a high probability to the correctly output label, which is shown as:

; In the formula:

is the output class for each trial,

The value of is determined by the negative logarithmic likelihood, and its specific calculation expression is as follows:

. 5. The light-weight and fast motor imagery EEG signal decoding method according to claim 1, characterized in that: in step S2, processing the original EEG signal is converting the original EEG signal into a cross-time sample C*T The channels of the matrix are then filtered with a bandpass filter between 8Hz and 30Hz, and then batch normalization is performed on the filtered EEG signals. 6. the lightweight and fast motor imagery EEG signal decoding method according to claim 5, characterized in that: the original EEG signal is expressed as:

; In the formula:

On behalf of

The total number of recorded trials, the first number of motor imagery signals

Trials are closely linked to the output class; for a given trial

, No.

The output of the input trials is mapped to the output class label

, to refer to the four categories of motor imagery tasks. 7. The lightweight and fast motor imagery EEG signal decoding method according to claim 6, characterized in that: utilize the BrainDecode framework to realize the training of the deep learning model; divide the original EEG signal into 750 samples as deep learning For the input of the model, the value of each batch is initially set to 58. When the epochs execution standard exceeds the preset maximum value, the training is terminated; at the same time, the early stopping strategy is combined with the Adam optimizer to improve the classification accuracy. 8. The light-weight and fast motor imagery EEG signal decoding method according to claim 7, characterized in that: a batch normalization regularization method is used to prevent over-fitting from occurring.

Images