CN117075741B - Consciousness interaction communication method and system - Google Patents
Consciousness interaction communication method and system Download PDFInfo
- Publication number
- CN117075741B CN117075741B CN202311338300.6A CN202311338300A CN117075741B CN 117075741 B CN117075741 B CN 117075741B CN 202311338300 A CN202311338300 A CN 202311338300A CN 117075741 B CN117075741 B CN 117075741B
- Authority
- CN
- China
- Prior art keywords
- electroencephalogram
- waveform
- eyeball
- preset
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004891 communication Methods 0.000 title claims abstract description 16
- 238000000034 method Methods 0.000 title claims abstract description 14
- 230000003993 interaction Effects 0.000 title claims description 9
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 93
- 230000004424 eye movement Effects 0.000 claims abstract description 17
- 230000002452 interceptive effect Effects 0.000 claims abstract description 10
- 210000004556 brain Anatomy 0.000 claims description 53
- 238000012790 confirmation Methods 0.000 claims description 14
- 208000003443 Unconsciousness Diseases 0.000 abstract description 8
- 230000002159 abnormal effect Effects 0.000 abstract description 4
- 238000012545 processing Methods 0.000 abstract description 2
- 230000005284 excitation Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 208000007204 Brain death Diseases 0.000 description 4
- 210000005036 nerve Anatomy 0.000 description 4
- 210000000133 brain stem Anatomy 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 210000001508 eye Anatomy 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000029058 respiratory gaseous exchange Effects 0.000 description 2
- 206010011224 Cough Diseases 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 241001282135 Poromitra oscitans Species 0.000 description 1
- 206010048232 Yawning Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 230000003930 cognitive ability Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000002503 metabolic effect Effects 0.000 description 1
- 230000004060 metabolic process Effects 0.000 description 1
- 230000008035 nerve activity Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000035764 nutrition Effects 0.000 description 1
- 235000016709 nutrition Nutrition 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 230000002269 spontaneous effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
- G06F2218/14—Classification; Matching by matching peak patterns
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Neurology (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Neurosurgery (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Dermatology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a consciousness interactive communication method and a consciousness interactive communication system, which belong to the technical field of data processing, and comprise the steps of displaying problem images; collecting the eyeball direction; judging whether the first position data are consistent; judging whether the waveform of the electroencephalogram is matched with a pre-stored first electroencephalogram, and outputting text data corresponding to the first position data if the waveform of the electroencephalogram is matched with the pre-stored first electroencephalogram. The invention realizes the double check of two electric signals by the double judgment of the eyeball direction and the waveform of the electroencephalogram, so that the accuracy of the output text data is higher. In addition, the eye movement instrument can be used for positioning whether the electroencephalogram is in a selected time period or not, so that erroneous judgment caused by unconscious electroencephalogram abnormal fluctuation and erroneous judgment caused by unconscious eye movement stay direction are avoided.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a conscious interaction communication method.
Background
Plant humans (PVS) are special human body states similar to the living states of plants. Apart from retaining some intrinsic neural reflex and metabolic capacity for substance and energy, cognitive abilities (including the ability to be self-existing) have been completely lost without any active activity. Also known as the state of the plant material, irreversible coma. The brainstem of the plant human still has the functions of digesting and absorbing and utilizing the energy to maintain the metabolism of the human body, including respiration, heartbeat, blood pressure, etc. when the brain stem of the plant human is used for delivering nutrition into the human body. Some natural reflections can be generated by external stimulus, such as cough, sneeze, yawning, etc. However, the body has no specific high-grade nerve activity such as consciousness, perception and thinking. Electroencephalogram presents a spurious waveform. The plant state is different from brain death, which refers to total brain death including brainstem. The brain dead person has no spontaneous respiration, and the electroencephalogram is in a straight line.
However, some plant persons are treated and recovered to enable certain electroencephalogram fluctuations and/or to enable autonomous control of eye movements. But cannot communicate speech. In order to communicate with the consciousness interaction of the patient, the characteristic part of the electroencephalogram or the eye movement instrument is extracted to mark the consciousness form, so that certain interaction is realized.
However, in the prior art, the analysis results of the analysis of the pure electroencephalogram and the analysis of the pure eye tracker have larger errors, and compared with direct frontal speech communication, the problem of larger recognition accuracy exists, so that a larger obstacle is brought to interaction.
In addition, neither the eye tracker nor the electroencephalogram can have unintentional staring somewhere or unintentional waveform jumping, and one of the results is taken as the final interaction result, so that the accuracy is too low.
Therefore, there is a need for a conscious interactive communication method and system capable of overcoming the above technical problems.
Disclosure of Invention
The present invention is directed to a conscious interactive communication method for solving the above problems in the prior art.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides a consciousness interactive communication method, which comprises the following steps of
Displaying a question image, wherein the question image comprises text data, a first button symbol and a second button symbol;
collecting the direction of the eyeball and outputting the direction as the direction of the eyeball;
judging whether the eyeball direction is consistent with first position data corresponding to one of a pre-stored first button or a pre-stored second button, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and if so, outputting text data corresponding to the first position data;
the collecting the eyeball direction and outputting the eyeball direction as the eyeball direction comprises the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the acquiring the electroencephalogram within the preset time comprises the following steps:
the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;
acquiring an electroencephalogram within a preset time;
the judging whether the waveform of the electroencephalogram is matched with the prestored first brain wave comprises the following steps:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments from two adjacent peaks to the peaks, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold, if not, deleting the first segments, and if so, taking the abscissa between the peaks of the adjacent first segments as a first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
The invention provides a system for conscious interaction communication method, which comprises
An image display module for displaying a question image, wherein the question image includes text data, a first button symbol, and a second button symbol;
the eye movement instrument is used for collecting the direction of the eyeball and outputting the direction of the eyeball;
the processor is used for judging whether the eyeball direction is consistent with the first position data corresponding to one of the pre-stored first buttons or the pre-stored second buttons, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and outputting text data corresponding to the first position data if the waveform of the electroencephalogram is matched with the pre-stored first brain wave;
the collecting the eyeball direction and outputting the eyeball direction as the eyeball direction comprises the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the step of acquiring the electroencephalogram within the preset time comprises the following steps:
the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;
acquiring an electroencephalogram within a preset time;
the judging whether the waveform of the electroencephalogram is matched with the prestored first brain wave comprises the following steps:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments from two adjacent peaks to the peaks, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold, if not, deleting the first segments, and if so, taking the abscissa between the peaks of the adjacent first segments as a first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
The beneficial effects are that:
the invention realizes the double check of two electric signals by the double judgment of the eyeball direction and the waveform of the electroencephalogram, so that the accuracy of the output text data is higher. In addition, the eye movement instrument can be used for positioning whether the electroencephalogram is in a selected time period or not, so that erroneous judgment caused by unconscious electroencephalogram abnormal fluctuation and erroneous judgment caused by unconscious eye movement stay direction are avoided.
Drawings
FIG. 1 is a first flowchart of a conscious interactive communication method according to the present invention;
FIG. 2 is a second flowchart of a conscious interactive communication method according to the present invention;
FIG. 3 is a schematic waveform diagram of an electroencephalogram;
fig. 4 is a schematic diagram of a real object demonstration state.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly described below with reference to the accompanying drawings and the description of the embodiments or the prior art, and it is obvious that the following description of the structure of the drawings is only some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art. It should be noted that the description of these examples is for aiding in understanding the present invention, but is not intended to limit the present invention.
Examples:
as shown in fig. 1, 3 and 4, the present embodiment provides a conscious interactive communication method, which includes
Displaying a question image, wherein the question image comprises text data, a first button symbol and a second button symbol;
collecting the direction of the eyeball and outputting the direction as the direction of the eyeball;
judging whether the eyeball direction is consistent with first position data corresponding to one of a pre-stored first button or a pre-stored second button, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first electroencephalogram, and outputting text data corresponding to the first position data if the waveform of the electroencephalogram is matched with the pre-stored first electroencephalogram.
The invention realizes the double check of two electric signals by the double judgment of the eyeball direction and the waveform of the electroencephalogram, so that the accuracy of the output text data is higher. In addition, the eye movement instrument can be used for positioning whether the electroencephalogram is in a selected time period or not, so that erroneous judgment caused by unconscious electroencephalogram abnormal fluctuation and erroneous judgment caused by unconscious eye movement stay direction are avoided.
In use, a person interacting with the patient may dictate the question and prepare a corresponding question image, or write the question directly on the question image as shown in fig. 4.
The manner of collecting the eyeball direction may be: the direction of the eyeball is collected by an eye movement instrument. The eye movement instrument is the prior art.
Wherein the "output text data corresponding to the first position data", that is, "output text data corresponding to the first button or the second button corresponding to the first position data", that is, the first button and the second button may correspond to text data respectively as described in the following table:
the invention is provided with a database, wherein the database is used for pre-storing first position data corresponding to a first button symbol and a second button symbol; and the database is also used for pre-storing first symbol characteristics corresponding to the first standard electric wave.
Referring to fig. 1, 3 and 4, the collecting the direction of the eyeball direction and outputting the direction as the eyeball direction includes the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the step of acquiring the electroencephalogram within the preset time comprises the following steps of:
the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;
and acquiring an electroencephalogram within a preset time.
The invention uses the initial time of the eyeball stagnation as the judging condition of the acquired electroencephalogram, which is convenient for a user to see the text data of the first button or the second button firstly, thus the user can initiate ideas after seeing the text data corresponding to the first button and understanding the text data, thus intercepting the electroencephalogram in the preset time corresponding to the eyeball direction, further leading the acquired electroencephalogram to have pertinence, and filtering a plurality of unnecessary electroencephalogram time periods for judging the waveform of the electroencephalogram.
In the problem image, the first button is arranged on the left side, the second button is arranged on the right side, text data corresponding to the first button is arranged in the first button, and text data corresponding to the second button is arranged in the second button.
Wherein the preset time and the confirmation threshold are the same, and may be 1 second, 2 seconds, 3 seconds, 4 seconds or 5 seconds, that is, the time period after the eyeball starts to look is valid.
Of course, the start point of the time axis of the preset time is the same as the start point of the time axis of the time held in the preset direction region, but the time held in the preset direction region may be longer than the preset time by 1 second or 2 seconds or 3 seconds, for example, the preset time is 3 seconds, representing three meter seconds seen by the eyeball, and the time held in the preset direction region may be 4 seconds or 5 seconds or 6 seconds, representing an electroencephalogram within 4 seconds or 5 seconds or 6 seconds after the start of the acquisition of the eyeball, thereby increasing the accuracy of the acquisition.
The preset direction area can be only one area on the left side, one area on the right side, and the two areas are all arranged; or, 4 areas are respectively arranged at the upper, lower, left and right sides.
Of course, as a modified structure, referring to fig. 1, 3 and 4, the collecting the direction of the eyeball and outputting the direction as the eyeball direction includes the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the step of acquiring the electroencephalogram within the preset time comprises the following steps of:
the end point of the time axis of the preset time is the same as the end point of the time axis of the time kept in the preset direction area; and acquiring an electroencephalogram within a preset time.
The invention uses the stop time of the stop of the eyeball to be taken as the judging condition of the collected electroencephalogram, which is convenient for a user to see the text data of the first button or the second button firstly, thus the user can initiate ideas of the text data after seeing the text data corresponding to the first button and understanding the text data, thus the electroencephalogram in the preset time corresponding to the eyeball direction is intercepted, the collected electroencephalogram has more pertinence, and a plurality of unnecessary electroencephalogram time periods are filtered for judging the waveform of the electroencephalogram.
The end point of the time axis of the time kept in the preset direction region is understood to be the end point at the point of time when the time kept in the preset direction region just exceeds the confirmation threshold.
The preset time and the time kept in the preset direction area may be the same, and may be 1 second, 2 seconds, 3 seconds, 4 seconds, or 5 seconds, that is, a period of time after the eyeball starts to look is valid.
Of course, the start point of the time axis of the preset time is the same as the end point of the time axis of the time held in the preset direction region, but the time held in the preset direction region may be longer than the preset time by 1 second or 2 seconds or 3 seconds, for example, the preset time is 3 seconds, representing three meter seconds seen by the eyeball, and the time held in the preset direction region may be 4 seconds or 5 seconds or 6 seconds, representing an electroencephalogram before 1 second or 2 seconds or 3 seconds at which the acquisition is desired to control the eyeball to start to see, thereby increasing the accuracy of the acquisition.
Wherein, referring to fig. 2, 3 and 4, the determining whether the waveform of the electroencephalogram matches the pre-stored first brain waves includes:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments by taking two adjacent peaks to peaks as the first segments, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold value, if not, deleting the first segments, if so, taking the abscissa between the peaks of the adjacent first segments as a first length, and deleting the first lengths of two adjacent sides of the deleted first segments in the first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
The invention matches the first brain wave with the waveform of the electroencephalogram in the grid matching mode, so as to find out the first brain wave which is stored in the database and is most matched with the waveform of the existing electroencephalogram. In addition, the grid mode selected by the invention is emphasized, the peak and trough of the detected electroencephalogram waveform are different due to different nerve reaction force or excitation degree of different people, and only the first brain waves prestored by longer conventional excitation degree or reaction force are stored in the database, and the first brain waves contain the first characters corresponding to the first brain waves. Then, if the first brain wave that is the best match with it is to be found, the influence of the excitation degree and reaction force of the present person on the waveform of the present electroencephalogram must be considered. Therefore, grids with corresponding sizes are configured according to the distances between the peaks of different electroencephalograms, so that statistics of the number of grids is performed, and the matched first brain waves corresponding to the number of grids can be more reasonably counted.
The first threshold and the second threshold may be the minimum distances between the peaks of all pre-stored first brain waves counted in the same manner.
The number of grids occupied by the misaligned area between the waveform of the electroencephalogram and the first electroencephalogram is counted, and the number can indicate whether the waveform difference between the waveform of the electroencephalogram and the first electroencephalogram is larger or smaller, and it is to be noted that the number of grids occupied is calculated by rounding, that is, less than half of one grid is occupied, no statistics is counted, and if more than half of one grid is occupied, the number of grids is calculated to occupy one grid.
The maximum amplitude point of the peaks or troughs of all the first segments is understood to be the maximum amplitude point of the troughs if the peaks are (1, 10) and the troughs are (3, -20), that is to say, the maximum amplitude point of the peaks or troughs of all the first segments is counted as the point at which the absolute value of the maximum amplitude is the largest.
The minimum distance between the abscissas of adjacent peaks of the electroencephalogram is configured as the length of the grid,
the square of the grid length can be understood as the area of the grid, and the grid length configured in the mode can be better suitable for different human bodies.
Wherein "deleting the first segment is understood as" summing "and deleting the first lengths of the adjacent sides of the deleted first segment in the first length", deleting the first segment directly in the waveform diagram, and the positions of other first segments are not moving, that is, if deleting the first segment, it is generally impossible to count the adjacent peaks of the two sides of the first segment as the minimum first length.
If yes, judging whether the time of the eyeball direction towards the first position data corresponding to one of the first button or the second button exceeds a confirmation threshold value;
of course, as a modification structure, referring to fig. 2, 3 and 4, the determining whether the waveform of the electroencephalogram matches the pre-stored first electroencephalogram includes:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments by taking wave crests to wave troughs, judging whether one of the wave crest amplitude or the wave trough amplitude of the first segments is larger than a first preset threshold value, if not, deleting the first segments, if so, taking the abscissa between wave troughs of adjacent first segments as a first length, and deleting the first lengths of two adjacent sides of the deleted first segments in the first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
The invention matches the first brain wave with the waveform of the electroencephalogram in the grid matching mode, so as to find out the first brain wave which is stored in the database and is most matched with the waveform of the existing electroencephalogram. In addition, the grid mode selected by the invention is emphasized, the peak and trough of the detected electroencephalogram waveform are different due to different nerve reaction force or excitation degree of different people, and only the first brain waves prestored by longer conventional excitation degree or reaction force are stored in the database, and the first brain waves contain the first characters corresponding to the first brain waves. Then, if the first brain wave that is the best match with it is to be found, the influence of the excitation degree and reaction force of the present person on the waveform of the present electroencephalogram must be considered. Therefore, grids with corresponding sizes are configured according to the distances between the troughs of different electroencephalograms, so that statistics of the number of grids is performed, and the matched first brain waves corresponding to the number of grids can be more reasonably counted.
The first threshold and the second threshold may be the minimum distances between the peaks and the troughs of all pre-stored first brain waves counted in the same manner.
The number of grids occupied by the misaligned area between the waveform of the electroencephalogram and the first electroencephalogram is counted, and the number can indicate whether the waveform difference between the waveform of the electroencephalogram and the first electroencephalogram is larger or smaller, and it is to be noted that the number of grids occupied is calculated by rounding, that is, less than half of one grid is occupied, no statistics is counted, and if more than half of one grid is occupied, the number of grids is calculated to occupy one grid.
The maximum amplitude point of the peaks or troughs of all the first segments is understood to be the maximum amplitude point of the troughs if the peaks are (1, 10) and the troughs are (3, -20), that is to say, the maximum amplitude point of the peaks or troughs of all the first segments is counted as the point at which the absolute value of the maximum amplitude is the largest.
The minimum distance between the abscissas of the adjacent wave troughs of the electroencephalogram is configured as the length of the grid,
the square of the grid length can be understood as the area of the grid, and the grid length configured in the mode can be better suitable for different human bodies.
Wherein "deleting the first segment is understood as" and "deleting the first lengths on adjacent sides of the deleted first segment in the first length", deleting the first segment directly in the waveform diagram, and the positions of other first segments are not moving, that is, if deleting the first segment, adjacent wave troughs on two sides of the first segment cannot be counted as the minimum first length.
The invention refers to FIGS. 1, 3 and 4, a system for a conscious interactive communication method, comprising
An image display module for displaying a question image, wherein the question image includes text data, a first button symbol, and a second button symbol;
the eye movement instrument is used for collecting the direction of the eyeball and outputting the direction of the eyeball;
the processor is used for judging whether the eyeball direction is consistent with the first position data corresponding to one of the pre-stored first buttons or the pre-stored second buttons, and if so, acquiring an electroencephalogram in a preset time; and judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and if so, outputting text data corresponding to the first position data.
The invention realizes the double check of two electric signals by the double judgment of the eyeball direction and the waveform of the electroencephalogram, so that the accuracy of the output text data is higher. In addition, the eye movement instrument can be used for positioning whether the electroencephalogram is in a selected time period or not, so that erroneous judgment caused by unconscious electroencephalogram abnormal fluctuation and erroneous judgment caused by unconscious eye movement stay direction are avoided.
The manner of collecting the eyeball direction may be: the direction of the eyeball is collected by an eye movement instrument. The eye movement instrument is the prior art.
Wherein the "output text data corresponding to the first position data", that is, "output text data corresponding to the first button or the second button corresponding to the first position data", that is, the first button and the second button may correspond to text data respectively as described in the following table:
the invention is provided with a database, wherein the database is used for pre-storing first position data corresponding to a first button symbol and a second button symbol; and the database is also used for pre-storing first symbol characteristics corresponding to the first standard electric wave.
Referring to fig. 1, 3 and 4, the collecting the direction of the eyeball direction and outputting the direction as the eyeball direction includes the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the step of acquiring the electroencephalogram within the preset time comprises the following steps of:
the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area; and acquiring an electroencephalogram within a preset time.
The invention uses the initial time of the eyeball stagnation as the judging condition of the acquired electroencephalogram, which is convenient for a user to see the text data of the first button or the second button firstly, thus the user can initiate ideas after seeing the text data corresponding to the first button and understanding the text data, thus intercepting the electroencephalogram in the preset time corresponding to the eyeball direction, further leading the acquired electroencephalogram to have pertinence, and filtering a plurality of unnecessary electroencephalogram time periods for judging the waveform of the electroencephalogram.
In the problem image, the first button is arranged on the left side, the second button is arranged on the right side, text data corresponding to the first button is arranged in the first button, and text data corresponding to the second button is arranged in the second button.
Wherein the preset time and the confirmation threshold are the same, and may be 1 second, 2 seconds, 3 seconds, 4 seconds or 5 seconds, that is, the time period after the eyeball starts to look is valid.
Of course, the start point of the time axis of the preset time is the same as the start point of the time axis of the time held in the preset direction region, but the time held in the preset direction region may be longer than the preset time by 1 second or 2 seconds or 3 seconds, for example, the preset time is 3 seconds, representing three meter seconds seen by the eyeball, and the time held in the preset direction region may be 4 seconds or 5 seconds or 6 seconds, representing an electroencephalogram within 4 seconds or 5 seconds or 6 seconds after the start of the acquisition of the eyeball, thereby increasing the accuracy of the acquisition.
The preset direction area can be only one area on the left side, one area on the right side, and the two areas are all arranged; or, 4 areas are respectively arranged at the upper, lower, left and right sides.
Of course, as a modified structure, referring to fig. 1, 3 and 4, the collecting the direction of the eyeball and outputting the direction as the eyeball direction includes the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the step of acquiring the electroencephalogram within the preset time comprises the following steps of:
the end point of the time axis of the preset time is the same as the end point of the time axis of the time kept in the preset direction area; and acquiring an electroencephalogram within a preset time.
The invention uses the stop time of the stop of the eyeball to be taken as the judging condition of the collected electroencephalogram, which is convenient for a user to see the text data of the first button or the second button firstly, thus the user can initiate ideas of the text data after seeing the text data corresponding to the first button and understanding the text data, thus the electroencephalogram in the preset time corresponding to the eyeball direction is intercepted, the collected electroencephalogram has more pertinence, and a plurality of unnecessary electroencephalogram time periods are filtered for judging the waveform of the electroencephalogram.
The end point of the time axis of the time kept in the preset direction region is understood to be the end point at the point of time when the time kept in the preset direction region just exceeds the confirmation threshold.
The preset time and the time kept in the preset direction area may be the same, and may be 1 second, 2 seconds, 3 seconds, 4 seconds, or 5 seconds, that is, a period of time after the eyeball starts to look is valid.
Of course, the end point of the time axis of the preset time is the same as the end point of the time axis of the time held in the preset direction region, but the time held in the preset direction region may be longer than the preset time by 1 second or 2 seconds or 3 seconds, for example, the preset time is 3 seconds, representing three meter seconds seen by the eyeball, and the time held in the preset direction region may be 4 seconds or 5 seconds or 6 seconds, representing an electroencephalogram before 1 second or 2 seconds or 3 seconds at which the acquisition is desired to control the eyeball to start to see, thereby increasing the accuracy of the acquisition.
Wherein, referring to fig. 2, 3 and 4, the determining whether the waveform of the electroencephalogram matches the pre-stored first brain waves includes:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments by taking two adjacent peaks to peaks as the first segments, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold value, if not, deleting the first segments, if so, taking the abscissa between the peaks of the adjacent first segments as a first length, and deleting the first lengths of two adjacent sides of the deleted first segments in the first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
The invention matches the first brain wave with the waveform of the electroencephalogram in the grid matching mode, so as to find out the first brain wave which is stored in the database and is most matched with the waveform of the existing electroencephalogram. In addition, the grid mode selected by the invention is emphasized, the peak and trough of the detected electroencephalogram waveform are different due to different nerve reaction force or excitation degree of different people, and only the first brain waves prestored by longer conventional excitation degree or reaction force are stored in the database, and the first brain waves contain the first characters corresponding to the first brain waves. Then, if the first brain wave that is the best match with it is to be found, the influence of the excitation degree and reaction force of the present person on the waveform of the present electroencephalogram must be considered. Therefore, grids with corresponding sizes are configured according to the distances between the peaks of different electroencephalograms, so that statistics of the number of grids is performed, and the matched first brain waves corresponding to the number of grids can be more reasonably counted.
The first threshold and the second threshold may be the minimum distances between the peaks of all pre-stored first brain waves counted in the same manner.
The number of grids occupied by the misaligned area between the waveform of the electroencephalogram and the first electroencephalogram is counted, and the number can indicate whether the waveform difference between the waveform of the electroencephalogram and the first electroencephalogram is larger or smaller, and it is to be noted that the number of grids occupied is calculated by rounding, that is, less than half of one grid is occupied, no statistics is counted, and if more than half of one grid is occupied, the number of grids is calculated to occupy one grid.
The maximum amplitude point of the peaks or troughs of all the first segments is understood to be the maximum amplitude point of the troughs if the peaks are (1, 10) and the troughs are (3, -20), that is to say, the maximum amplitude point of the peaks or troughs of all the first segments is counted as the point at which the absolute value of the maximum amplitude is the largest.
The minimum distance between the abscissas of adjacent peaks of the electroencephalogram is configured as the length of the grid,
the square of the grid length can be understood as the area of the grid, and the grid length configured in the mode can be better suitable for different human bodies.
Wherein "deleting the first segment is understood as" summing "and deleting the first lengths of the adjacent sides of the deleted first segment in the first length", deleting the first segment directly in the waveform diagram, and the positions of other first segments are not moving, that is, if deleting the first segment, it is generally impossible to count the adjacent peaks of the two sides of the first segment as the minimum first length.
If so, judging whether the time of the eyeball direction towards the first position data corresponding to one of the first button or the second button exceeds a confirmation threshold.
Of course, as a modification structure, referring to fig. 1, 3 and 4, the determining whether the waveform of the electroencephalogram matches the pre-stored first electroencephalogram includes:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments by taking wave crests to wave troughs, judging whether one of the wave crest amplitude or the wave trough amplitude of the first segments is larger than a first preset threshold value, if not, deleting the first segments, if so, taking the abscissa between wave troughs of adjacent first segments as a first length, and deleting the first lengths of two adjacent sides of the deleted first segments in the first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
The invention matches the first brain wave with the waveform of the electroencephalogram in the grid matching mode, so as to find out the first brain wave which is stored in the database and is most matched with the waveform of the existing electroencephalogram. In addition, the grid mode selected by the invention is emphasized, the peak and trough of the detected electroencephalogram waveform are different due to different nerve reaction force or excitation degree of different people, and only the first brain waves prestored by longer conventional excitation degree or reaction force are stored in the database, and the first brain waves contain the first characters corresponding to the first brain waves. Then, if the first brain wave that is the best match with it is to be found, the influence of the excitation degree and reaction force of the present person on the waveform of the present electroencephalogram must be considered. Therefore, grids with corresponding sizes are configured according to the distances between the troughs of different electroencephalograms, so that statistics of the number of grids is performed, and the matched first brain waves corresponding to the number of grids can be more reasonably counted.
The first threshold and the second threshold may be the minimum distances between the peaks and the troughs of all pre-stored first brain waves counted in the same manner.
The number of grids occupied by the misaligned area between the waveform of the electroencephalogram and the first electroencephalogram is counted, and the number can indicate whether the waveform difference between the waveform of the electroencephalogram and the first electroencephalogram is larger or smaller, and it is to be noted that the number of grids occupied is calculated by rounding, that is, less than half of one grid is occupied, no statistics is counted, and if more than half of one grid is occupied, the number of grids is calculated to occupy one grid.
The maximum amplitude point of the peaks or troughs of all the first segments is understood to be the maximum amplitude point of the troughs if the peaks are (1, 10) and the troughs are (3, -20), that is to say, the maximum amplitude point of the peaks or troughs of all the first segments is counted as the point at which the absolute value of the maximum amplitude is the largest.
The minimum distance between the abscissas of the adjacent wave troughs of the electroencephalogram is configured as the length of the grid,
the square of the grid length can be understood as the area of the grid, and the grid length configured in the mode can be better suitable for different human bodies.
Wherein "deleting the first segment is understood as" and "deleting the first lengths on adjacent sides of the deleted first segment in the first length", deleting the first segment directly in the waveform diagram, and the positions of other first segments are not moving, that is, if deleting the first segment, adjacent wave troughs on two sides of the first segment cannot be counted as the minimum first length.
Finally, it should be noted that: the foregoing description is only of the preferred embodiments of the invention and is not intended to limit the scope of the invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (2)
1. A conscious interaction communication method is characterized by comprising the following steps of
Displaying a question image, wherein the question image comprises text data, a first button symbol and a second button symbol;
collecting the direction of the eyeball and outputting the direction as the direction of the eyeball;
judging whether the eyeball direction is consistent with first position data corresponding to one of a pre-stored first button or a pre-stored second button, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and if so, outputting text data corresponding to the first position data;
the collecting the eyeball direction and outputting the eyeball direction as the eyeball direction comprises the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the acquiring the electroencephalogram within the preset time comprises the following steps:
the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;
acquiring an electroencephalogram within a preset time;
the judging whether the waveform of the electroencephalogram is matched with the prestored first brain wave comprises the following steps:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments from two adjacent peaks to the peaks, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold, if not, deleting the first segments, and if so, taking the abscissa between the peaks of the adjacent first segments as a first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
2. A system for a conscious interactive communication method, comprising
An image display module for displaying a question image, wherein the question image includes text data, a first button symbol, and a second button symbol;
the eye movement instrument is used for collecting the direction of the eyeball and outputting the direction of the eyeball;
the processor is used for judging whether the eyeball direction is consistent with the first position data corresponding to one of the pre-stored first buttons or the pre-stored second buttons, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and outputting text data corresponding to the first position data if the waveform of the electroencephalogram is matched with the pre-stored first brain wave;
the collecting the eyeball direction and outputting the eyeball direction as the eyeball direction comprises the following steps:
collecting the real-time direction of the eyeball direction;
judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;
the step of acquiring the electroencephalogram within the preset time comprises the following steps:
the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;
acquiring an electroencephalogram within a preset time;
the judging whether the waveform of the electroencephalogram is matched with the prestored first brain wave comprises the following steps:
pre-storing first brain waves corresponding to the text data;
dividing the waveform of the electroencephalogram into first segments from two adjacent peaks to the peaks, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold, if not, deleting the first segments, and if so, taking the abscissa between the peaks of the adjacent first segments as a first length;
judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;
overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;
counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311338300.6A CN117075741B (en) | 2023-10-17 | 2023-10-17 | Consciousness interaction communication method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311338300.6A CN117075741B (en) | 2023-10-17 | 2023-10-17 | Consciousness interaction communication method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117075741A CN117075741A (en) | 2023-11-17 |
| CN117075741B true CN117075741B (en) | 2023-12-12 |
Family
ID=88715624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311338300.6A Active CN117075741B (en) | 2023-10-17 | 2023-10-17 | Consciousness interaction communication method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117075741B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144238A (en) * | 2018-05-14 | 2019-01-04 | 孙佳楠 | A kind of man-machine interactive system and its exchange method based on eye electricity coding |
| CN110837298A (en) * | 2019-10-31 | 2020-02-25 | 福建心动生物科技有限公司 | Brain wave remote control training system and method based on concentration and vision |
| CN112034977A (en) * | 2019-06-04 | 2020-12-04 | 陈涛 | Method for MR intelligent glasses content interaction, information input and recommendation technology application |
| CN112957014A (en) * | 2021-02-07 | 2021-06-15 | 广州大学 | Pain detection and positioning method and system based on brain waves and neural network |
| CN113662563A (en) * | 2021-09-02 | 2021-11-19 | 潍坊医学院 | Electroencephalogram data storage and transmission method and system based on neural network |
| CN116098633A (en) * | 2022-12-12 | 2023-05-12 | 首都医科大学附属北京天坛医院 | Minimally invasive intervention brain-computer interface ideation brain control system for consciousness disturbance |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105528084A (en) * | 2016-01-21 | 2016-04-27 | 京东方科技集团股份有限公司 | Display control device, display control method thereof and display control system |
| US11647956B2 (en) * | 2018-03-19 | 2023-05-16 | Neurofeedback-Partner GmbH | Electroencephalogram system and method |
-
2023
- 2023-10-17 CN CN202311338300.6A patent/CN117075741B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144238A (en) * | 2018-05-14 | 2019-01-04 | 孙佳楠 | A kind of man-machine interactive system and its exchange method based on eye electricity coding |
| CN112034977A (en) * | 2019-06-04 | 2020-12-04 | 陈涛 | Method for MR intelligent glasses content interaction, information input and recommendation technology application |
| CN110837298A (en) * | 2019-10-31 | 2020-02-25 | 福建心动生物科技有限公司 | Brain wave remote control training system and method based on concentration and vision |
| CN112957014A (en) * | 2021-02-07 | 2021-06-15 | 广州大学 | Pain detection and positioning method and system based on brain waves and neural network |
| CN113662563A (en) * | 2021-09-02 | 2021-11-19 | 潍坊医学院 | Electroencephalogram data storage and transmission method and system based on neural network |
| CN116098633A (en) * | 2022-12-12 | 2023-05-12 | 首都医科大学附属北京天坛医院 | Minimally invasive intervention brain-computer interface ideation brain control system for consciousness disturbance |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117075741A (en) | 2023-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP0504945A2 (en) | Device for simultaneous diagnosis and quantitative analysis of apnea and other diseases | |
| CN105105714A (en) | Sleep period separating method and system | |
| EP2916724A1 (en) | Method and device for determining vital parameters | |
| CN104984475B (en) | The rehabilitation equipment that suppression Parkinson's tranquillization based on skin reflex principle trembles | |
| DE112018003005T5 (en) | FATIGUE RECOVERY SUPPORT DEVICE | |
| CN101779955A (en) | Portable brain function biofeedback instrument | |
| CN102708288A (en) | Brain-computer interface based doctor-patient interaction method | |
| CN113855052B (en) | Nerve feedback intervention system and method based on positive idea meditation | |
| CN112328072A (en) | Multi-mode character input system and method based on electroencephalogram and electrooculogram | |
| CN106859617A (en) | A kind of many vital sign parameter collecting devices of Wearable and its parameter extracting method | |
| CN103211603A (en) | Mental stress detecting, tracking and feedback system | |
| CN117075741B (en) | Consciousness interaction communication method and system | |
| Yoon et al. | EEG spectral analysis in valence and arousal dimensions of emotion | |
| CN110688013A (en) | English keyboard spelling system and method based on SSVEP | |
| CN113546395A (en) | Intelligent exercise training system and training method | |
| CN211957134U (en) | Sleep quality monitoring and interaction system | |
| CN113359991A (en) | Intelligent brain-controlled mechanical arm auxiliary feeding system and method for disabled people | |
| CN113729732B (en) | Sleep quality monitoring system and method based on EEG signal | |
| CN112037916A (en) | Shared multifunctional sudden death prevention physiological information detection system and method thereof | |
| CN215993951U (en) | Sleep respiration monitoring belt | |
| Li et al. | Design and implementation of multimedia control system based on bluetooth and electrooculogram (EOG) | |
| CN117037571A (en) | AI-based artificial intelligence auxiliary communication system | |
| Dong et al. | An EOG signals recognition method based on improved threshold dual tree complex wavelet transform | |
| CN203802456U (en) | Bluetooth wireless sleep monitoring device | |
| OuYang et al. | An algorithm for reading activity recognition based on electrooculogram |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |