CN116088686A - Electroencephalogram tracing motor imagery brain-computer interface training method and system - Google Patents
Electroencephalogram tracing motor imagery brain-computer interface training method and system Download PDFInfo
- Publication number
- CN116088686A CN116088686A CN202310083118.4A CN202310083118A CN116088686A CN 116088686 A CN116088686 A CN 116088686A CN 202310083118 A CN202310083118 A CN 202310083118A CN 116088686 A CN116088686 A CN 116088686A
- Authority
- CN
- China
- Prior art keywords
- motor imagery
- signal
- brain
- signals
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Neurology (AREA)
- Computational Linguistics (AREA)
- Neurosurgery (AREA)
- Dermatology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a motor imagery brain-computer interface training method and system for brain-computer tracing, which comprise the steps of acquiring multichannel brain-computer signals, carrying out real-time tracing analysis according to the multichannel brain-computer signals through a preset deep neural network tracing model to obtain a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed of a cerebral cortex, carrying out double-sample-t test on the first source space signal when the motor imagery is executed and the second source space signal when the motor imagery is not executed to obtain a double-sample-t test result, carrying out weighted average on a third source space signal of an activated brain region of a preset motor imagery task according to the double-sample-t test result to obtain a motor imagery neural signal mark, presenting a multi-mode training feedback signal according to the motor imagery neural signal mark, optimizing a trainer's motor imagery strategy according to the multi-mode training feedback signal, improving the positioning accuracy of the traced brain region and realizing targeted motor imagery training.
Description
Technical Field
The invention relates to the technical field related to electroencephalogram signal processing, in particular to a motor imagery brain-computer interface training method and system for electroencephalogram tracing.
Background
The brain-computer interface (Brain computer interface, BCI) is a special human-computer interaction paradigm that is independent of peripheral nerves and muscles and enables direct communication between the human brain and computer equipment. Common brain-computer interface paradigms include steady-state visual evoked potential (Steady state visual evoked potential, SSVEP), event-related potential (Event related potential, ERP), and Motor Imagery (MI) paradigms. SSVEP and ERP belong to the excitation brain-computer interface paradigm, and an external stimulation device is needed to collect the brain-computer signals generated when the test is subjected to external stimulation. The MI model is a spontaneous brain-computer interface model, does not need external stimulation, and the acquired brain-computer signal is completely derived from the psychological activities of the tested, is more close to 'conscious' control, is a model more in line with the definition of BCI, and has wide application prospect. The MI paradigm refers to a test subject that moves in a location of the brain that is intended to resemble the body, without actually doing the movement. The physiological basis of the brain-computer interface system based on motor imagery is: when a person imagines movements of different parts of the body, different functional areas of the brain are activated accordingly, so that electroencephalogram signals with different characteristics are generated, including Event-related desynchronization (Event-related desynchronization, ERD) and Event-related synchronization phenomena (Event-related synchronization, ERS). ERD phenomenon means that when an MI task is performed, a region related to the cerebral cortex is activated, a specific frequency amplitude of brain wave signals is reduced, and energy is reduced. The ERS phenomenon means that when the execution of MI task ends or enters a brain resting state, the amplitude of brain electrical signals in the relevant region increases, and the energy increases. When imagining the movement of different parts of the body, different positions of the cerebral cortex are activated, the ERD phenomenon appears on the brain electrical signals of different brain areas, and the ERD components in the brain area nerve signals of different positions of the brain are detected to perform pattern analysis, so that the movement intention of a tested person is identified.
MI training is of great importance for recovery of dyskinesia patients, and can help improve recovery of patients' locomotor ability and daily locomotor ability. In addition, the tested MI task is a controller of the whole MI-BCI system, the performance of the MI-BCI is seriously dependent on the performance of the tested MI, but the MI capacity difference of different people is obvious due to the individual difference of MI mental activities, and the improvement of the tested MI capacity through MI training is also of great significance for improving the performance of the MI-BCI system. However, most of the current MI training methods are based on brain electrical signals acquired in a scalp electrode sensor space, energy, power spectral density and frequency band characteristics of the brain electrical signals are calculated, and the brain electrical signals are presented to a tested in a feedback mode to realize nerve feedback training. The training mode can only locate and display the neural activity of the scalp electrode position, can not locate the neural activity of each brain region of the brain related to the MI task, has volume conduction effect in the process of conducting signals of the cerebral cortex to the scalp position, has little difference in modes of different MI tasks on the scalp electrode signals, and can not realize the targeted neural feedback training related to the specific MI task.
Disclosure of Invention
The present invention aims to at least solve the technical problems existing in the prior art. Therefore, the invention provides a motor imagery brain-computer interface training method and system for electroencephalogram tracing, which can improve the accuracy of tracing brain region positioning and realize targeted motor imagery training.
The invention provides a motor imagery brain-computer interface training method for electroencephalogram tracing, which comprises the following steps of:
acquiring multichannel electroencephalogram signals;
performing real-time tracing analysis according to the multichannel electroencephalogram signals through a preset deep neural network tracing model to obtain a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed of the cerebral cortex;
performing double-sample-t test on the first source space signal when the motor imagery is executed and the second source space signal when the motor imagery is not executed to obtain a double-sample-t test result;
according to the double-sample-t test result, carrying out weighted average on third source space signals of the activated brain region of the preset motor imagery task to obtain motor imagery nerve signal marks;
and presenting a multi-modal training feedback signal according to the motor imagery nerve signal mark, and optimizing a motor imagery strategy of a trainer according to the multi-modal training feedback signal.
According to the embodiment of the invention, at least the following technical effects are achieved:
according to the method, multichannel brain electrical signals are obtained, real-time tracing analysis is carried out according to the multichannel brain electrical signals through a preset deep neural network tracing model, a first source space signal when motor imagery is executed and a second source space signal when motor imagery is not executed of a cerebral cortex are obtained, double-sample-t test is carried out on the first source space signal when motor imagery is executed and the second source space signal when motor imagery is not executed, a double-sample-t test result is obtained, weighted average is carried out on a third source space signal of an activated brain region of a preset motor imagery task according to the double-sample-t test result, a motor imagery neural signal mark is obtained, a multi-mode training feedback signal is presented according to the motor imagery neural signal mark, a motor imagery strategy of a trainer is optimized according to the multi-mode training feedback signal, the positioning accuracy of the traced brain region is improved, and targeted motor imagery training is achieved.
According to some embodiments of the invention, after the acquiring the multichannel electroencephalogram signal, the method further comprises:
the method comprises the steps of collecting multichannel electroencephalogram signals through 64-channel electroencephalogram signal collecting electrodes, a signal amplifier and a signal transmitter, and storing the multichannel electroencephalogram signals in a data buffer area, wherein the data buffer area adopts a first-in first-out queue structure;
extracting the multichannel electroencephalogram signals acquired in a period of time from the data buffer;
and removing band-stop filtering and high-frequency noise of 50Hz power frequency noise and noise caused by head movement and eye movement in the multichannel electroencephalogram signals to obtain the preprocessed electroencephalogram signals.
According to some embodiments of the invention, a deep neural network traceability model is trained by:
dividing the individual head model into a plurality of small areas to obtain a divided individual head model, wherein each small area comprises a plurality of grid points;
synthesizing a simulated activation source signal of a cerebral cortex source space through a nerve group model, sparsely distributing the simulated activation source signal in the cerebral cortex source space, and randomly changing the position of the simulated activation source signal to obtain a simulated activation source signal sample with different sparse cortex distribution;
calculating a forward conduction matrix of the simulated activation source signal sample from the cerebral cortex source space to an observation electrode space according to the divided individual head model;
calculating according to the analog activation source signal sample and the forward conduction matrix to obtain a scalp electrode signal corresponding to the analog activation source signal sample, and forming a source space-scalp electroencephalogram signal pair by the scalp electrode signal corresponding to the analog activation source signal sample;
off-line decomposing the acquired brain electrical data by an independent component analysis method to obtain head movement and eye electrical noise signals;
adding scalp electrode signals corresponding to the analog activation source signal samples into the head movement and eye electrical noise signals to obtain noisy multi-lead scalp electroencephalogram signals;
and constructing an initial deep neural network traceability model, and training the initial deep neural network traceability model according to the noisy multi-lead scalp electroencephalogram signal and the source space-scalp electroencephalogram signal pair through a gradient return learning algorithm to obtain the deep neural network traceability model.
According to some embodiments of the present invention, before performing weighted average on the third source spatial signal of the activated brain region of the preset motor imagery task according to the double-sample-t test result to obtain a motor imagery neural signal mark, the motor imagery brain-computer interface training method of electroencephalogram tracing further includes:
acquiring multichannel electroencephalogram signals of a plurality of users for performing motor imagery task experiments, and denoising and preprocessing the multichannel electroencephalogram signals of the plurality of users for performing motor imagery task experiments to obtain preprocessed electroencephalogram signals of the plurality of users;
calculating the motor imagery task recognition accuracy of each user through a co-space mode algorithm according to the multichannel electroencephalogram signals of the plurality of users for performing the motor imagery task experiments;
inputting the preprocessed brain electrical signals of the plurality of users with the motor imagery task recognition accuracy rates ranked in front into a deep neural network traceability model to obtain first source space signals of the plurality of users when the motor imagery is executed;
the brain region which is jointly activated by the first source space signals when the motor imagery is executed of the cerebral cortex of the plurality of users is selected to be set as the activated brain region of the motor imagery task.
According to some embodiments of the invention, the presenting a multimodal training feedback signal from the motor imagery neural signal markers comprises:
constructing visual and auditory nerve feedback signals according to the motor imagery nerve signal markers;
and transmitting and presenting the visual and audible nerve feedback signals through an HDMI video interface or an audio interface to obtain the multi-mode training feedback signals.
According to some embodiments of the invention, after the multi-modal training feedback signal is presented according to the motor imagery neural signal marker, the motor imagery brain-computer interface training method of electroencephalogram tracing further includes:
taking the average activation degree of the activated brain region of the motor imagery task as a weight, calculating and obtaining weighted average energy and amplitude information of the activated brain region of the motor imagery task, and taking the weighted average energy and amplitude information of the activated brain region of the motor imagery task as an activation level requirement of a target brain region;
counting the times that the multi-mode training feedback signal reaches the activation level requirement of the target brain region within the preset cycle time to obtain the cycle standard reaching times;
judging whether the period reaching the standard times exceeds a preset threshold times, and if so, presenting a front report pattern of the smile picture.
According to some embodiments of the invention, the sampling frequency of the 64-lead electroencephalogram signal acquisition electrode is 1000Hz, the electrode position is according to the international 10/20 system standard, the reference electrode is FCz, the ground is AFz, and the resistance value of the electrode is kept below 10 kiloohms.
In a second aspect of the present invention, there is provided an electroencephalogram tracing motor imagery brain-computer interface training system, the electroencephalogram tracing motor imagery brain-computer interface training system comprising:
the electroencephalogram signal acquisition module is used for acquiring multichannel electroencephalogram signals;
the real-time tracing analysis module is used for carrying out real-time tracing analysis according to the multichannel electroencephalogram signals through a preset deep neural network tracing model to obtain a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed of the cerebral cortex;
the double-sample-t test module is used for carrying out double-sample-t test on the first source space signal when the motor imagery is executed and the second source space signal when the motor imagery is not executed to obtain a double-sample-t test result;
the motor imagery neural signal mark calculation module is used for carrying out weighted average on third source space signals of the activated brain region of the preset motor imagery task according to the double-sample-t test result to obtain a motor imagery neural signal mark;
the feedback signal presentation module is used for presenting a multi-mode training feedback signal according to the motor imagery nerve signal mark and optimizing a motor imagery strategy of a trainer according to the multi-mode training feedback signal.
According to the system, a multichannel brain electrical signal is obtained, real-time tracing analysis is carried out according to the multichannel brain electrical signal through a preset deep neural network tracing model, a first source space signal when motor imagery is executed and a second source space signal when motor imagery is not executed of a cerebral cortex are obtained, double-sample-t test is carried out on the first source space signal when motor imagery is executed and the second source space signal when motor imagery is not executed, a double-sample-t test result is obtained, weighted average is carried out on a third source space signal of an activated brain region of a preset motor imagery task according to the double-sample-t test result, a motor imagery neural signal mark is obtained, a multi-mode training feedback signal is presented according to the motor imagery neural signal mark, a motor imagery strategy of a trainer is optimized according to the multi-mode training feedback signal, the positioning accuracy of the traced brain region is improved, and targeted motor imagery training is realized.
In a third aspect of the present invention, there is provided an electroencephalogram traceable motor imagery brain-computer interface training electronic device, comprising at least one control processor and a memory for communication connection with the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform the motor imagery brain-computer interface training method of brain-computer traceability described above.
In a fourth aspect of the present invention, there is provided a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described motor imagery brain-computer interface training method of brain-electrical tracing.
It should be noted that the advantages of the second to fourth aspects of the present invention and the prior art are the same as those of the motor imagery brain-computer interface training system for tracing brain and electricity and the prior art, and will not be described in detail herein.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a flow chart of a motor imagery brain-computer interface training method of brain-computer traceability according to an embodiment of the present invention;
fig. 2 is a schematic overall flow diagram of a motor imagery brain-computer interface training method of brain-computer traceability according to an embodiment of the present invention;
fig. 3 is a flowchart of a motor imagery brain-computer interface training system of brain-computer traceability according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
In the description of the present invention, the description of first, second, etc. is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, it should be understood that the direction or positional relationship indicated with respect to the description of the orientation, such as up, down, etc., is based on the direction or positional relationship shown in the drawings, is merely for convenience of describing the present invention and simplifying the description, and does not indicate or imply that the apparatus or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
In the description of the present invention, unless explicitly defined otherwise, terms such as arrangement, installation, connection, etc. should be construed broadly and the specific meaning of the terms in the present invention can be determined reasonably by a person skilled in the art in combination with the specific content of the technical solution.
MI training is of great importance for recovery of dyskinesia patients, and can help improve recovery of patients' locomotor ability and daily locomotor ability. In addition, the tested MI task is a controller of the whole MI-BCI system, the performance of the MI-BCI is seriously dependent on the performance of the tested MI, but the MI capacity difference of different people is obvious due to the individual difference of MI mental activities, and the improvement of the tested MI capacity through MI training is also of great significance for improving the performance of the MI-BCI system. However, most of the current MI training methods are based on brain electrical signals acquired in a scalp electrode sensor space, energy, power spectral density and frequency band characteristics of the brain electrical signals are calculated, and the brain electrical signals are presented to a tested in a feedback mode to realize nerve feedback training. The training mode can only locate and display the neural activity of the scalp electrode position, can not locate the neural activity of each brain region of the brain related to the MI task, has volume conduction effect in the process of conducting signals of the cerebral cortex to the scalp position, has little difference in modes of different MI tasks on the scalp electrode signals, and can not realize the targeted neural feedback training related to the specific MI task.
In order to solve the technical defects, referring to fig. 1 and 2, the invention also provides a motor imagery brain-computer interface training method for tracing brain electricity, which comprises the following steps:
step S101, acquiring multichannel brain electrical signals;
step S1012, performing real-time tracing analysis according to multichannel brain electrical signals through a preset deep neural network tracing model to obtain a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed of the cerebral cortex;
step S103, performing double-sample-t test on the first source space signal when the motor imagery is executed and the second source space signal when the motor imagery is not executed to obtain a double-sample-t test result;
step S104, carrying out weighted average on third source space signals of the activated brain region of the preset motor imagery task according to the double-sample-t test result to obtain motor imagery nerve signal marks;
According to the method, multichannel brain electrical signals are obtained, real-time tracing analysis is carried out according to the multichannel brain electrical signals through a preset deep neural network tracing model, a first source space signal when motor imagery is executed and a second source space signal when motor imagery is not executed of a cerebral cortex are obtained, double-sample-t test is carried out on the first source space signal when motor imagery is executed and the second source space signal when motor imagery is not executed, a double-sample-t test result is obtained, weighted average is carried out on a third source space signal of an activated brain region of a preset motor imagery task according to the double-sample-t test result, a motor imagery neural signal mark is obtained, a multi-mode training feedback signal is presented according to the motor imagery neural signal mark, a motor imagery strategy of a trainer is optimized according to the multi-mode training feedback signal, the positioning accuracy of the traced brain region is improved, and targeted motor imagery training is achieved.
In some embodiments, after acquiring the multichannel electroencephalogram signal, further comprising:
the multi-channel electroencephalogram signals are collected through the 64-channel electroencephalogram signal collecting electrode, the signal amplifier and the signal transmitter and stored in a data buffer area, wherein the data buffer area adopts a first-in first-out queue structure;
extracting multichannel electroencephalogram signals acquired within a period of time from a data buffer area;
and removing band-stop filtering of 50Hz power frequency noise, high-frequency noise and noise caused by head movement and eye movement in the multichannel electroencephalogram signals to obtain the preprocessed electroencephalogram signals.
In some embodiments, the deep neural network traceability model is trained by:
dividing the individual head model into a plurality of small areas to obtain a divided individual head model, wherein each small area comprises a plurality of grid points;
synthesizing a simulated activation source signal of a cerebral cortex source space through a nerve group model, sparsely distributing the simulated activation source signal in the cerebral cortex source space, and randomly changing the position of the simulated activation source signal to obtain a simulated activation source signal sample with different sparse cortex distribution;
calculating a forward conduction matrix of the simulated activation source signal sample from the cerebral cortex source space to the observation electrode space according to the divided individual head model;
calculating according to the analog activation source signal sample and the forward conduction matrix to obtain scalp electrode signals corresponding to the analog activation source signal sample, and forming a source space-scalp electroencephalogram signal pair by the scalp electrode signals corresponding to the analog activation source signal sample;
off-line decomposing the acquired brain electrical data by an independent component analysis method to obtain head movement and eye electrical noise signals;
adding scalp electrode signals corresponding to the analog activation source signal samples into head movement and eye electrical noise signals to obtain noisy multi-lead scalp electroencephalogram signals;
and constructing an initial deep neural network traceability model, and training the initial deep neural network traceability model according to the noisy multi-lead scalp electroencephalogram signal and the source space-scalp electroencephalogram signal pair through a gradient return learning algorithm to obtain the deep neural network traceability model.
In some embodiments, before performing weighted average on the third source spatial signal of the activated brain region of the preset motor imagery task according to the double-sample-t test result to obtain the motor imagery neural signal mark, the motor imagery brain-computer interface training method of the brain electricity tracing further includes:
acquiring multichannel electroencephalogram signals of a plurality of users for performing motor imagery task experiments, and denoising and preprocessing the multichannel electroencephalogram signals of the plurality of users for performing motor imagery task experiments to obtain preprocessed electroencephalogram signals of the plurality of users;
calculating the motor imagery task recognition accuracy of each user through a co-space mode algorithm according to the multi-channel electroencephalogram signals of the plurality of users for performing the motor imagery task experiments;
inputting preprocessed brain electrical signals of a plurality of users of the users with the motor imagery task recognition accuracy rates ranked in front of the motor imagery task recognition accuracy rates into a deep neural network traceability model to obtain first source space signals when the motor imagery is executed by the cerebral cortex of the users;
the brain region commonly activated by the first source space signal when the motor imagery is performed of the cerebral cortex of the selected plurality of users is set as the activated brain region of the motor imagery task.
In some embodiments, presenting a multimodal training feedback signal from motor imagery neural signal markers includes:
constructing visual and auditory nerve feedback signals according to motor imagery nerve signal markers;
and transmitting and presenting visual and auditory nerve feedback signals through the HDMI video interface or the audio interface to obtain a multi-mode training feedback signal.
In some embodiments, after presenting the multimodal training feedback signal according to the motor imagery neural signal marker, the motor imagery brain-computer interface training method of brain electrical tracing further includes:
taking the average activation degree of the activated brain region of the motor imagery task as a weight, calculating to obtain weighted average energy and amplitude information of the activated brain region of the motor imagery task, and taking the weighted average energy and amplitude information of the activated brain region of the motor imagery task as an activation level requirement of a target brain region;
counting the times that the multi-mode training feedback signal reaches the activation level requirement of the target brain region in the preset cycle time to obtain the cycle standard reaching times;
judging whether the period reaching the standard times exceeds the preset threshold times, and if so, displaying the front report pattern of the smile picture.
In some embodiments, the sampling frequency of the 64-lead electroencephalogram signal acquisition electrode is 1000Hz, the electrode position is in reference to the international 10/20 system standard, the reference electrode is FCz, the ground is AFz, and the resistance value of the electrode is kept below 10 kiloohms.
The brain-computer interface type brain-computer signal provided by the invention is used as a noninvasive physiological signal, and the motor imagery is used as a spontaneous brain-computer interface type, so that the brain-computer interface type brain-computer signal can be suitable for almost all people. And brain regions related to each type of motor imagery task can be determined by tracing the brain electrical signals to the brain cortex source space, and compared with a system for performing feedback training by using scalp electrodes, the system can more accurately determine training and regulation targets, and realize targeted motor imagery training;
in some embodiments, if the pre-experiment and the tested person of the training experiment are set to be the same person, the method can be further expanded into an individual training mode aiming at each trainer, and the accuracy of tracing brain area positioning can be further improved by adopting the individual training mode and adopting an individual head model in tracing calculation.
For ease of understanding by those skilled in the art, a set of motor imagery training experimental data is provided below:
in a quiet data acquisition room, the trainer sits on a chair, faces the display, and wears headphones.
Each trainer performed four types of motor imagery tasks including left hand, right hand, tongue, and bipedal motor imagery, the trainer performed 160 trials altogether, each type of motor imagery task including 40 trials.
The system comprises 4 sessions, wherein each session comprises 4 run, each run is 10 test motor imagery tasks, the 4 run experiments are sequentially 10 test left-hand motor imagery tasks, 10 test right-hand motor imagery tasks, 10 test tongue motor imagery tasks and 10 test bipedal motor imagery tasks.
Each test motor imagery task experiment comprises a resting state of 3s and motor imagery task execution time of 5s, after each run10 test motor imagery tasks are executed, a trainer closes eyes to rest for 1 minute, and then starts to execute the next run experiment. The execution time of each session is about 9 minutes, and the rest time of 2 minutes is included between the two sessions, so that the whole motor imagery task training experiment of the trainer is about 42 minutes;
before the run experiment starts, the display displays the category of the motor imagery task to be executed later, and the user is reminded to start executing the motor imagery task by the prompt tone.
The method comprises the steps that multichannel electroencephalogram signals of each type of motor imagery tasks to be tested are collected through an electroencephalogram collection device and sent to a computing unit through a wireless module, and the computing unit stores the transmitted multichannel electroencephalogram signals in a data buffer area;
the computing unit refreshes the data buffer in real time, and the length of the electroencephalogram signals stored in the data buffer is 500ms.
The calculation unit performs preprocessing such as filtering and the like on the brain electrical signals of the data buffer region 500ms, then inputs a trained deep neural network traceability model to perform traceability analysis, and calculates to obtain source signals of target brain regions activated by each category of motor imagery tasks.
Based on the target brain region source signals activated by each motor imagery task of the trainer, constructing visual/auditory nerve feedback signals, displaying the source signals of the target brain region activated by the trainer and the activation degree of the target brain region on a display in real time, and displaying the auditory feedback signals on a headset.
And counting the number of times that the source space level of the activation target brain region of each run training experiment of the trainer meets the activation level requirement determined by the pre-experiment, and if the number of times of each run experiment (10 times) exceeds 50%, presenting smile pictures on a display after each run experiment as a return.
And counting smile picture times obtained by a trainer in the whole experiment process, wherein the smile picture times exceeds 50% of all run times (16), and considering that a motor imagery nerve feedback training experiment of the trainer is successful, and the trainer has better motor imagery ability after being trained by the motor imagery brain-computer interface training system.
In addition, referring to fig. 3, an embodiment of the present invention provides a motor imagery brain-computer interface training system for electroencephalogram tracing, which includes an electroencephalogram signal acquisition module 1100, a real-time tracing analysis module 1200, a double-sample-t test module 1300, a motor imagery neural signal mark calculation module 1400 and a feedback signal presentation module 1500, wherein:
the electroencephalogram signal acquisition module 1100 is used for acquiring multichannel electroencephalogram signals;
the real-time tracing analysis module 1200 is configured to perform real-time tracing analysis according to a multichannel electroencephalogram signal through a preset deep neural network tracing model, so as to obtain a first source space signal when the cortex performs motor imagery and a second source space signal when the cortex does not perform motor imagery;
the double-sample-t-test module 1300 is used for performing double-sample-t-test on a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed, so as to obtain a double-sample-t-test result;
the motor imagery neural signal mark calculation module 1400 is configured to perform weighted average on third source spatial signals of the activated brain region of the preset motor imagery task according to the double-sample-t test result, so as to obtain motor imagery neural signal marks;
the feedback signal presenting module 1500 is configured to present a multimodal training feedback signal according to the motor imagery neural signal marker, and optimize a motor imagery strategy of a trainer according to the multimodal training feedback signal.
According to the system, a multichannel brain electrical signal is obtained, real-time tracing analysis is carried out according to the multichannel brain electrical signal through a preset deep neural network tracing model, a first source space signal when motor imagery is executed and a second source space signal when motor imagery is not executed of a cerebral cortex are obtained, double-sample-t test is carried out on the first source space signal when motor imagery is executed and the second source space signal when motor imagery is not executed, a double-sample-t test result is obtained, weighted average is carried out on a third source space signal of an activated brain region of a preset motor imagery task according to the double-sample-t test result, a motor imagery neural signal mark is obtained, a multi-mode training feedback signal is presented according to the motor imagery neural signal mark, a motor imagery strategy of a trainer is optimized according to the multi-mode training feedback signal, the positioning accuracy of the traced brain region is improved, and targeted motor imagery training is realized.
It should be noted that, the system embodiment and the above-mentioned system embodiment are based on the same inventive concept, so that the relevant content of the above-mentioned method embodiment is also applicable to the system embodiment, and is not repeated here.
The application also provides a motor imagery brain-computer interface training electronic equipment that brain electricity traced to source, include: memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing when executing the computer program: the motor imagery brain-computer interface training method based on the electroencephalogram tracing.
The processor and the memory may be connected by a bus or other means.
The memory, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs as well as non-transitory computer executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory remotely located relative to the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The non-transitory software program and instructions required to implement the motor imagery brain-computer interface training method of electroencephalogram tracing of the above embodiments are stored in the memory, and when executed by the processor, the motor imagery brain-computer interface training method of electroencephalogram tracing in the above embodiments is performed, for example, the method steps S101 to S105 in fig. 1 described above are performed.
The present application also provides a computer-readable storage medium storing computer-executable instructions for performing: the motor imagery brain-computer interface training method based on the electroencephalogram tracing.
The computer-readable storage medium stores computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described electronic device embodiment, which may cause the processor to perform the motor imagery brain-computer interface training method of brain-electric tracing in the above-described embodiment, for example, to perform the method steps S101 to S105 in fig. 1 described above.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program elements or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program elements or other data in a modulated data signal such as a carrier wave or other transport mechanism and may include any information delivery media.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of one of ordinary skill in the art without departing from the spirit of the present invention.
Claims (10)
1. The motor imagery brain-computer interface training method for the electroencephalogram tracing is characterized by comprising the following steps of:
acquiring multichannel electroencephalogram signals;
performing real-time tracing analysis according to the multichannel electroencephalogram signals through a preset deep neural network tracing model to obtain a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed of the cerebral cortex;
performing double-sample-t test on the first source space signal when the motor imagery is executed and the second source space signal when the motor imagery is not executed to obtain a double-sample-t test result;
according to the double-sample-t test result, carrying out weighted average on third source space signals of the activated brain region of the preset motor imagery task to obtain motor imagery nerve signal marks;
and presenting a multi-modal training feedback signal according to the motor imagery nerve signal mark, and optimizing a motor imagery strategy of a trainer according to the multi-modal training feedback signal.
2. The method for training a motor imagery brain-computer interface according to claim 1, wherein after the acquiring the multichannel brain-computer signal, further comprises:
the method comprises the steps of collecting multichannel electroencephalogram signals through 64-channel electroencephalogram signal collecting electrodes, a signal amplifier and a signal transmitter, and storing the multichannel electroencephalogram signals in a data buffer area, wherein the data buffer area adopts a first-in first-out queue structure;
extracting the multichannel electroencephalogram signals acquired in a period of time from the data buffer;
and removing band-stop filtering and high-frequency noise of 50Hz power frequency noise and noise caused by head movement and eye movement in the multichannel electroencephalogram signals to obtain the preprocessed electroencephalogram signals.
3. The motor imagery brain-computer interface training method of brain electricity tracing according to claim 2, wherein the deep neural network tracing model is trained by:
dividing the individual head model into a plurality of small areas to obtain a divided individual head model, wherein each small area comprises a plurality of grid points;
synthesizing a simulated activation source signal of a cerebral cortex source space through a nerve group model, sparsely distributing the simulated activation source signal in the cerebral cortex source space, and randomly changing the position of the simulated activation source signal to obtain a simulated activation source signal sample with different sparse cortex distribution;
calculating a forward conduction matrix of the simulated activation source signal sample from the cerebral cortex source space to an observation electrode space according to the divided individual head model;
calculating according to the analog activation source signal sample and the forward conduction matrix to obtain a scalp electrode signal corresponding to the analog activation source signal sample, and forming a source space-scalp electroencephalogram signal pair by the scalp electrode signal corresponding to the analog activation source signal sample;
off-line decomposing the acquired brain electrical data by an independent component analysis method to obtain head movement and eye electrical noise signals;
adding scalp electrode signals corresponding to the analog activation source signal samples into the head movement and eye electrical noise signals to obtain noisy multi-lead scalp electroencephalogram signals;
and constructing an initial deep neural network traceability model, and training the initial deep neural network traceability model according to the noisy multi-lead scalp electroencephalogram signal and the source space-scalp electroencephalogram signal pair through a gradient return learning algorithm to obtain the deep neural network traceability model.
4. The method for training a motor imagery brain-computer interface of brain-computer traceability according to claim 3, wherein before performing weighted average on third source spatial signals of an activated brain region of a preset motor imagery task according to the double-sample-t test result to obtain motor imagery neural signal marks, the method for training a motor imagery brain-computer interface of brain-computer traceability further comprises:
acquiring multichannel electroencephalogram signals of a plurality of users for performing motor imagery task experiments, and denoising and preprocessing the multichannel electroencephalogram signals of the plurality of users for performing motor imagery task experiments to obtain preprocessed electroencephalogram signals of the plurality of users;
calculating the motor imagery task recognition accuracy of each user through a co-space mode algorithm according to the multichannel electroencephalogram signals of the plurality of users for performing the motor imagery task experiments;
inputting the preprocessed brain electrical signals of the plurality of users with the motor imagery task recognition accuracy rates ranked in front into a deep neural network traceability model to obtain first source space signals of the plurality of users when the motor imagery is executed;
the brain region which is jointly activated by the first source space signals when the motor imagery is executed of the cerebral cortex of the plurality of users is selected to be set as the activated brain region of the motor imagery task.
5. The motor imagery brain-computer interface training method of claim 4, wherein the presenting of the multimodal training feedback signal according to the motor imagery neural signal signature includes:
constructing visual and auditory nerve feedback signals according to the motor imagery nerve signal markers;
and transmitting and presenting the visual and audible nerve feedback signals through an HDMI video interface or an audio interface to obtain the multi-mode training feedback signals.
6. The method according to claim 5, wherein after the multi-modal training feedback signal is presented according to the motor imagery neural signal marker, the motor imagery brain-computer interface training method for brain tracing further comprises:
taking the average activation degree of the activated brain region of the motor imagery task as a weight, calculating and obtaining weighted average energy and amplitude information of the activated brain region of the motor imagery task, and taking the weighted average energy and amplitude information of the activated brain region of the motor imagery task as an activation level requirement of a target brain region;
counting the times that the multi-mode training feedback signal reaches the activation level requirement of the target brain region within the preset cycle time to obtain the cycle standard reaching times;
judging whether the period reaching the standard times exceeds a preset threshold times, and if so, presenting a front report pattern of the smile picture.
7. The motor imagery brain-computer interface training method of brain-computer traceability according to claim 6, wherein the sampling frequency of the 64-lead brain-computer signal collecting electrode is 1000Hz, the electrode position is according to international 10/20 system standard, the reference electrode is FCz, the ground is AFz, and the resistance value of the electrode is kept below 10 kiloohms.
8. The motor imagery brain-computer interface training system of brain electricity tracing is characterized in that the motor imagery brain-computer interface training method system of brain electricity tracing:
the electroencephalogram signal acquisition module is used for acquiring multichannel electroencephalogram signals;
the real-time tracing analysis module is used for carrying out real-time tracing analysis according to the multichannel electroencephalogram signals through a preset deep neural network tracing model to obtain a first source space signal when the motor imagery is executed and a second source space signal when the motor imagery is not executed of the cerebral cortex;
the double-sample-t test module is used for carrying out double-sample-t test on the first source space signal when the motor imagery is executed and the second source space signal when the motor imagery is not executed to obtain a double-sample-t test result;
the motor imagery neural signal mark calculation module is used for carrying out weighted average on third source space signals of the activated brain region of the preset motor imagery task according to the double-sample-t test result to obtain a motor imagery neural signal mark;
the feedback signal presentation module is used for presenting a multi-mode training feedback signal according to the motor imagery nerve signal mark and optimizing a motor imagery strategy of a trainer according to the multi-mode training feedback signal.
9. The motor imagery brain-computer interface training device for the electroencephalogram tracing is characterized by comprising at least one control processor and a memory, wherein the memory is used for being in communication connection with the at least one control processor; the memory stores instructions executable by the at least one control processor to enable the at least one control processor to perform a motor imagery brain-computer interface training method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized by: the computer-readable storage medium stores computer-executable instructions for causing a computer to perform a motor imagery brain-computer interface training method of brain-computer traceability according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310083118.4A CN116088686B (en) | 2023-01-17 | 2023-01-17 | Electroencephalogram tracing motor imagery brain-computer interface training method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310083118.4A CN116088686B (en) | 2023-01-17 | 2023-01-17 | Electroencephalogram tracing motor imagery brain-computer interface training method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116088686A true CN116088686A (en) | 2023-05-09 |
| CN116088686B CN116088686B (en) | 2023-07-25 |
Family
ID=86213718
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310083118.4A Active CN116088686B (en) | 2023-01-17 | 2023-01-17 | Electroencephalogram tracing motor imagery brain-computer interface training method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116088686B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116719411A (en) * | 2023-06-01 | 2023-09-08 | 之江实验室 | Information display method and device based on deep brain muscle functional coupling |
| CN118940154A (en) * | 2024-10-14 | 2024-11-12 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | A magnetoencephalogram tracing method based on deep neural network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765920A (en) * | 2019-10-18 | 2020-02-07 | 西安电子科技大学 | Motor imagery classification method based on convolutional neural network |
| US20210251555A1 (en) * | 2018-08-03 | 2021-08-19 | Rehabswift Pty Ltd | Stroke Rehabilitation Method and System Using a Brain-Computer Interface (BCI) |
| CN113918008A (en) * | 2021-08-30 | 2022-01-11 | 北京大学 | Brain-computer interface system based on source space brain magnetic signal decoding and application method |
| CN115562488A (en) * | 2022-10-08 | 2023-01-03 | 上海前瞻创新研究院有限公司 | Motor imagery brain-computer interface communication method, device, system, medium and equipment |
-
2023
- 2023-01-17 CN CN202310083118.4A patent/CN116088686B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210251555A1 (en) * | 2018-08-03 | 2021-08-19 | Rehabswift Pty Ltd | Stroke Rehabilitation Method and System Using a Brain-Computer Interface (BCI) |
| CN110765920A (en) * | 2019-10-18 | 2020-02-07 | 西安电子科技大学 | Motor imagery classification method based on convolutional neural network |
| CN113918008A (en) * | 2021-08-30 | 2022-01-11 | 北京大学 | Brain-computer interface system based on source space brain magnetic signal decoding and application method |
| CN115562488A (en) * | 2022-10-08 | 2023-01-03 | 上海前瞻创新研究院有限公司 | Motor imagery brain-computer interface communication method, device, system, medium and equipment |
Non-Patent Citations (1)
| Title |
|---|
| 李鹏海;王丽余;刘瀛涛;刘斌;马春雷;: "下肢运动想象和运动执行的EEG节律特性研究", 仪器仪表学报, no. 03 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116719411A (en) * | 2023-06-01 | 2023-09-08 | 之江实验室 | Information display method and device based on deep brain muscle functional coupling |
| CN116719411B (en) * | 2023-06-01 | 2024-06-21 | 之江实验室 | Information display method and device based on deep brain muscle functional coupling |
| CN118940154A (en) * | 2024-10-14 | 2024-11-12 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | A magnetoencephalogram tracing method based on deep neural network |
| CN118940154B (en) * | 2024-10-14 | 2024-12-20 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Brain magnetic map tracing method based on deep neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116088686B (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210012675A1 (en) | Teaching method and teaching device for improving attention, and computer readable storage medium | |
| US20240023892A1 (en) | Method and system for collecting and processing bioelectrical signals | |
| CN116088686B (en) | Electroencephalogram tracing motor imagery brain-computer interface training method and system | |
| CN104965584B (en) | Mixing brain-machine interface method based on SSVEP and OSP | |
| Pfurtscheller et al. | 15 years of BCI research at Graz University of Technology: current projects | |
| Trejo et al. | Brain-computer interfaces for 1-D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials | |
| CN102016757B (en) | Method for controlling device by using brain wave and brain wave interface system | |
| US20070060830A1 (en) | Method and system for detecting and classifying facial muscle movements | |
| Rohani et al. | BCI inside a virtual reality classroom: a potential training tool for attention | |
| CN109620257B (en) | Mental state intervention and regulation system based on biofeedback and its working method | |
| CN105708436A (en) | Mental regulation method and device | |
| CN103251417B (en) | Method for representing and identifying entrepreneurial potential electroencephalogram signals | |
| US20100094156A1 (en) | System and Method for Biofeedback Administration | |
| CN107402635B (en) | Mental health adjusting method and system combining brain waves and virtual reality | |
| CN108335728A (en) | A kind of name of calling out for disturbance of consciousness patient promotees system of waking up | |
| Pun et al. | Brain-computer interaction research at the Computer Vision and Multimedia Laboratory, University of Geneva | |
| CN115376695A (en) | Method, system and device for neuropsychological assessment and intervention based on augmented reality | |
| Punsawad et al. | Multi‐command SSAEP‐based BCI system with training sessions for SSVEP during an eye fatigue state | |
| CN113397547A (en) | Film watching evaluation service system and method based on physiological data | |
| KR20160031187A (en) | System for psychotherapy by using neurofeedback | |
| Milekovic et al. | Volitional control of single-electrode high gamma local field potentials by people with paralysis | |
| CN105700687A (en) | Single-trial electroencephalogram P300 component detection method based on folding HDCA algorithm | |
| CN110811548A (en) | A memory state evaluation method, system, device and storage medium | |
| CN107510451B (en) | pitch perception ability objective assessment method based on brainstem auditory evoked potentials | |
| WO2020139108A1 (en) | Method for conducting cognitive examinations using a neuroimaging system and a feedback mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |