CN117075741B - A consciousness interactive communication method and system - Google Patents

Abstract

The invention discloses a consciousness interactive communication method and a consciousness interactive communication system, which belong to the technical field of data processing, and comprise the steps of displaying problem images; collecting the eyeball direction; judging whether the first position data are consistent; judging whether the waveform of the electroencephalogram is matched with a pre-stored first electroencephalogram, and outputting text data corresponding to the first position data if the waveform of the electroencephalogram is matched with the pre-stored first electroencephalogram. The invention realizes the double check of two electric signals by the double judgment of the eyeball direction and the waveform of the electroencephalogram, so that the accuracy of the output text data is higher. In addition, the eye movement instrument can be used for positioning whether the electroencephalogram is in a selected time period or not, so that erroneous judgment caused by unconscious electroencephalogram abnormal fluctuation and erroneous judgment caused by unconscious eye movement stay direction are avoided.

Description

A consciousness interactive communication method and system

Technical field

The invention belongs to the field of data processing technology, and specifically relates to a consciousness interactive communication method.

Background technique

Vegetative state (PVS) is a special human state similar to the living state of plants. In addition to retaining some instinctive nerve reflexes and the ability to metabolize matter and energy, cognitive abilities (including the ability to recognize one's own existence) have been completely lost, and there is no active activity. Also known as vegetative state and irreversible coma. The brainstem of a vegetative person is still functional. It can digest and absorb nutrients when transporting them to the body, and can use this energy to maintain the body's metabolism, including breathing, heartbeat, blood pressure, etc. Some instinctive reflexes can also occur in response to external stimuli, such as coughing, sneezing, yawning, etc. But the body no longer has consciousness, perception, thinking and other advanced neural activities unique to humans. The electroencephalogram showed spurious waveforms. The vegetative state is different from brain death, which refers to the death of the whole brain, including the brainstem. A brain-dead person has no spontaneous breathing and the electroencephalogram shows a straight line.

However, after treatment and recovery, some vegetative people can achieve certain electroencephalogram fluctuations and/or be able to control eye movements independently. But verbal communication is impossible. In order to facilitate conscious interactive communication with such patients, we usually extract the characteristic parts of the electroencephalogram or eye tracker to mark the ideology, so as to achieve a certain degree of interaction.

However, the analysis results of pure electroencephalogram analysis and pure eye tracker analysis in the existing technology have large errors. Compared with direct frontal verbal communication, there is still a problem of greater recognition accuracy. , thus bringing greater obstacles to interaction.

In addition, whether it is an eye tracker or an electroencephalogram, there will be unconscious staring at a certain place or unconscious waveform beating. Taking one of the results as the final interaction result is too low in accuracy.

Therefore, there is currently a need for a consciousness interactive communication method and system that can overcome the above technical problems.

Contents of the invention

The purpose of the present invention is to provide a consciousness interactive communication method to solve the above problems existing in the prior art.

In order to achieve the above objects, the present invention adopts the following technical solutions:

The present invention provides a consciousness interactive communication method, including

Display a question image, wherein the question image includes text data, a first button symbol and a second button symbol;

Collect the eyeball direction and output it as the eyeball direction;

Determine whether the eyeball direction is consistent with the first position data corresponding to one of the pre-stored first button or the second button. If so, collect the electroencephalogram within a preset time; determine whether the waveform of the electroencephalogram is consistent with the pre-stored first position data. The first brain waves match, and if they match, text data corresponding to the first position data is output;

The method of collecting the eyeball direction and outputting it as the eyeball direction includes the following steps:

Collect the real-time direction of the eyeball orientation;

Determine whether the cumulative duration of the eyeball orientation in the preset direction area exceeds a confirmation threshold, and if so, output the preset direction area as the eyeball direction;

The collection of EEG within a preset time includes:

The starting point of the timeline of the preset time is the same as the starting point of the timeline of the time remaining within the preset direction area;

Collect EEG within a preset time;

Determining whether the waveform of the electroencephalogram matches the prestored first brain wave includes:

Pre-store the first brain wave corresponding to the text data;

Divide the electroencephalogram waveform into a first segment from two adjacent wave peaks, and determine whether one of the peak amplitude or the trough amplitude of the first segment is greater than a first preset threshold; if not, Then delete the first segment, and if so, use the abscissa between the peaks of the adjacent first segments as the first length;

Determine whether the minimum first length is lower than the second preset threshold. If so, configure the electroencephalogram waveform in the grid, and configure the second preset threshold as the unit grid length. If not , then configure the minimum first length as the unit grid length;

Coincide the maximum amplitude point of the peak or trough of all the first segments of the EEG waveform with the lower left corner endpoint of one of the unit grids;

The number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first brain wave is counted, and the first electroencephalogram occupying the smallest number of grids is determined to match the waveform of the electroencephalogram. .

The present invention provides a system of consciousness interactive communication method, including

An image display module configured to display a question image, wherein the question image includes text data, a first button symbol and a second button symbol;

Eye tracker, which is used to collect the eyeball direction and output it as the eyeball direction;

A processor configured to determine whether the eyeball direction is consistent with the first position data corresponding to one of the prestored first button or the second button, and if so, collect the electroencephalogram within a preset time; and determine whether the brain Whether the waveform of the electrogram matches the pre-stored first brain wave, and if it matches, output text data corresponding to the first position data;

The method of collecting the eyeball direction and outputting it as the eyeball direction includes the following steps:

Collect the real-time direction of the eyeball orientation;

Determine whether the cumulative duration of the eyeball orientation in the preset direction area exceeds a confirmation threshold, and if so, output the preset direction area as the eyeball direction;

The collection of EEG within a preset time period includes:

The starting point of the timeline of the preset time is the same as the starting point of the timeline of the time remaining within the preset direction area;

Collect EEG within a preset time;

Determining whether the waveform of the electroencephalogram matches the prestored first brain wave includes:

Pre-store the first brain wave corresponding to the text data;

Divide the electroencephalogram waveform into a first segment from two adjacent wave peaks, and determine whether one of the peak amplitude or the trough amplitude of the first segment is greater than a first preset threshold; if not, Then delete the first segment, and if so, use the abscissa between the peaks of the adjacent first segments as the first length;

Determine whether the minimum first length is lower than the second preset threshold. If so, configure the electroencephalogram waveform in the grid, and configure the second preset threshold as the unit grid length. If not , then configure the minimum first length as the unit grid length;

Coincide the maximum amplitude point of the peak or trough of all the first segments of the EEG waveform with the lower left corner endpoint of one of the unit grids;

The number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first brain wave is counted, and the first electroencephalogram occupying the smallest number of grids is determined to match the waveform of the electroencephalogram. .

Beneficial effects:

The present invention realizes double verification of the two electrical signals through the above-mentioned double judgment of eyeball orientation and electroencephalogram waveform, so that the accuracy of the output text data is higher. In addition, the eye tracker can be used to determine whether the selected time period is EEG, thus avoiding misjudgments caused by abnormal fluctuations in EEG caused by unconsciousness and misjudgments caused by unconscious eye movement direction.

Description of the drawings

Figure 1 is a first flow chart of a consciousness interactive communication method according to the present invention;

Figure 2 is a second flow chart of a consciousness interactive communication method of the present invention;

Figure 3 is a schematic diagram of the waveform of EEG;

Figure 4 is a schematic diagram of the physical demonstration state.

Detailed ways

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts. It should be noted here that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

Example:

As shown in Figures 1, 3, and 4, this embodiment provides a consciousness interactive communication method, including

Display a question image, wherein the question image includes text data, a first button symbol and a second button symbol;

Collect the eyeball direction and output it as the eyeball direction;

Determine whether the eyeball direction is consistent with the first position data corresponding to one of the pre-stored first button or the second button. If so, collect the electroencephalogram within a preset time; determine whether the waveform of the electroencephalogram is consistent with the pre-stored first position data. The first brain waves match, and if they match, text data corresponding to the first position data is output.

The present invention realizes double verification of the two electrical signals through the above-mentioned double judgment of eyeball orientation and electroencephalogram waveform, so that the accuracy of the output text data is higher. In addition, the eye tracker can be used to determine whether the selected time period is EEG, thus avoiding misjudgments caused by abnormal fluctuations in EEG caused by unconsciousness and misjudgments caused by unconscious eye movement direction.

When used, the person interacting with the patient can dictate the problem and prepare the corresponding problem image, or directly write the problem on the problem image as shown in Figure 4.

Among them, the method of "collecting the eyeball direction" may be: collecting the eyeball direction through an eye tracker. The eye tracker is an existing technology.

Among them, "then output text data corresponding to the first position data", that is to say, "then output text data corresponding to the first button or second button corresponding to the first position data", that is, That is, the first button and the second button can respectively correspond to text data as described in the following table:

Among them, the present invention is provided with a database, which is used to pre-store the first position data corresponding to the first button symbol and the second button symbol; and the database is also used to pre-store the first symbol feature corresponding to the first standard radio wave.

Among them, referring to Figures 1, 3, and 4, collecting the eyeball direction and outputting it as the eyeball direction includes the following steps:

Collect the real-time direction of the eyeball orientation;

Determine whether the cumulative duration of the eyeball orientation in the preset direction area exceeds a confirmation threshold, and if so, output the preset direction area as the eyeball direction;

The method of collecting EEG within a preset time includes the following steps:

The starting point of the timeline of the preset time is the same as the starting point of the timeline of the time remaining within the preset direction area;

Collect EEG within a preset time.

The present invention collects the starting time when the eyeballs stagnate as a judgment condition for collecting electroencephalogram, which can facilitate the user to see the text data of the first button or the second button first, so that the user can see the text corresponding to the first button. After the data is collected, and after understanding the above text data, it triggers thoughts about it, thereby intercepting the EEG within the preset time corresponding to the above eyeball direction, so that the collected EEG is more targeted and is an EEG The waveform judgment filters out many unnecessary EEG time periods.

Wherein, in the question image, it can be configured that the left side is a first button and the right side is a second button. The first button is provided with text data corresponding to it, and the second button is provided with text data corresponding to it.

Wherein, the preset time and the confirmation threshold are the same, and they can be 1 second, 2 seconds, 3 seconds, 4 seconds or 5 seconds. That is to say, the time period after the eyeball starts looking is valid.

Of course, the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time remaining in the preset direction area, but the time staying in the preset direction area can be longer than the preset time by 1 second or 2 seconds. seconds or 3 seconds. For example, the preset time is 3 seconds, which means that the eyeballs have been looking for three meters for seconds, and the time remaining in the preset direction area can be 4 seconds, 5 seconds, or 6 seconds, which means that the eyeballs have been collected and started to look. Then the EEG within 4 seconds or 5 seconds or 6 seconds, thereby increasing the accuracy of the acquisition.

Among them, the preset direction area can be only one area on the left and one area on the right, a total of two areas; or one area each up, down, left, and right, a total of four areas.

Of course, as a deformation structure, it can also be, see Figures 1, 3, and 4. The method of collecting the eyeball direction and outputting it as the eyeball direction includes the following steps:

Collect the real-time direction of the eyeball orientation;

Determine whether the cumulative duration of the eyeball orientation in the preset direction area exceeds a confirmation threshold, and if so, output the preset direction area as the eyeball direction;

The method of collecting EEG within a preset time includes the following steps:

The end point of the time axis of the preset time is the same as the end point of the time axis remaining in the preset direction area; the electroencephalogram within the preset time is collected.

The present invention collects the end time when the eyeballs stagnate as a judgment condition for collecting electroencephalogram, which can facilitate the user to see the text data of the first button or the second button first, so that the user can see the text data corresponding to the first button. Finally, after understanding the above text data, it triggers thoughts about it, thereby intercepting the EEG within the preset time corresponding to the above eyeball direction, thereby making the collected EEG more targeted and providing the basis for EEG analysis. Waveform judgment filters out many unnecessary EEG time periods.

The end point of the time axis of the time remaining in the preset direction area can be understood as the time point when the time remaining in the preset direction area just exceeds the confirmation threshold.

Wherein, the preset time and the time to stay in the preset direction area can be the same, which can be 1 second, 2 seconds, 3 seconds, 4 seconds or 5 seconds, that is, the time period after the eyeballs start to look. efficient.

Of course, the starting point of the time axis of the preset time is the same as the end point of the time axis of the time remaining in the preset direction area, but the time staying in the preset direction area can be longer than the preset time by 1 second or 2 seconds. Seconds or 3 seconds. For example, the preset time is 3 seconds, which means that the eyeballs are looking for three meters for seconds, and the time remaining in the preset direction area can be 4 seconds, 5 seconds, or 6 seconds, which means that the eyeballs are collected and controlled. Start looking at the EEG 1 second or 2 seconds or 3 seconds ago, thereby increasing the accuracy of the collection.

Among them, referring to Figures 2, 3, and 4, the determination of whether the waveform of the electroencephalogram matches the prestored first brain wave includes:

Pre-store the first brain wave corresponding to the text data;

Divide the electroencephalogram waveform into a first segment from two adjacent wave peaks, and determine whether one of the peak amplitude or the trough amplitude of the first segment is greater than a first preset threshold; if not, , then delete the first segment. If so, use the abscissa between the peaks of the adjacent first segments as the first length, and delete the first length on both sides of the adjacent first segment. a length;

Determine whether the minimum first length is lower than the second preset threshold. If so, configure the electroencephalogram waveform in the grid, and configure the second preset threshold as the unit grid length. If not , then configure the minimum first length as the unit grid length;

Coincide the maximum amplitude point of the peak or trough of all the first segments of the EEG waveform with the lower left corner endpoint of one of the unit grids;

The number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first brain wave is counted, and the first electroencephalogram occupying the smallest number of grids is determined to match the waveform of the electroencephalogram. .

The present invention matches the first brain wave with the waveform of the electroencephalogram through the above grid matching method, thereby finding the first brain wave pre-stored in the database that best matches the existing electroencephalogram waveform. In addition, the grid method selected by the present invention needs to be emphasized. Since different people have different nerve reactions or different levels of excitement, the peaks and troughs of the detected EEG waveforms are also different, and what is stored in the database is only The first brain wave is pre-stored due to a longer conventional level of excitement or reaction, and the first brain wave contains the first text corresponding to it. Then, if we want to find the first brain wave that best matches it, we must take into account the impact of people's current level of excitement and reaction on the current EEG waveform. Therefore, by configuring grids of corresponding sizes according to the distances between the wave peaks of different EEGs, and counting the number of grids, it will be more reasonable to count the matching first brain waves corresponding to the number of grids.

Wherein, the first threshold and the second threshold may both be the minimum distance between the peaks of all pre-stored first brain waves calculated in the same manner.

Among them, the number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first electroencephalogram is counted. This number can indicate whether the waveform difference between the two waveforms is larger or smaller. It needs to be explained. Yes, the number of occupied grids is calculated based on rounding. That is to say, if it occupies less than half of a grid, it will not be counted. If it occupies more than half, it will be counted as occupying one grid.

Among them, the maximum amplitude point of all the peaks or troughs of the first segment can be understood as, if the peak is (1,10) and the trough is (3,-20), then the trough is the maximum amplitude point, that is, all the peaks and troughs of the first segment are The statistics of the maximum amplitude point of a wave peak or trough in a segment are the points with the largest absolute value of the maximum amplitude.

The minimum distance between the abscissas of adjacent wave peaks of the electroencephalogram is configured as the length of the grid,

Among them, the square of the grid length can be understood as the area of the grid. The grid length configured in the above way can better adapt to different human bodies.

Among them, "then delete the first segment can be understood as" and "and delete the first length on both sides of the adjacent two sides of the deleted first segment in the first length", and the first segment is directly deleted in the waveform diagram, Moreover, the positions of other first segments do not change. That is to say, if this first segment is deleted, the adjacent wave peaks on both sides of this first segment cannot generally be counted as the minimum first length.

If so, determine whether the time for which the eyeball is directed toward the first position data corresponding to one of the first button or the second button exceeds a confirmation threshold;

Of course, as a deformation structure, it can also be, see Figures 2, 3, and 4. Determining whether the waveform of the electroencephalogram matches the prestored first brain wave includes:

Pre-store the first brain wave corresponding to the text data;

Divide the electroencephalogram waveform into first segments from peak to trough, and determine whether one of the peak amplitude or trough amplitude of the first segment is greater than a first preset threshold; if not, delete the first segment. A segment, if so, use the abscissa between the troughs of adjacent first segments as the first length, and delete the first lengths on both adjacent sides of the deleted first segment in the first length;

Determine whether the minimum first length is lower than the second preset threshold. If so, configure the electroencephalogram waveform in the grid, and configure the second preset threshold as the unit grid length. If not , then configure the minimum first length as the unit grid length;

Coincide the maximum amplitude point of the peak or trough of all the first segments of the EEG waveform with the lower left corner endpoint of one of the unit grids;

The number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first brain wave is counted, and the first electroencephalogram occupying the smallest number of grids is determined to match the waveform of the electroencephalogram. .

The present invention matches the first brain wave with the waveform of the electroencephalogram through the above grid matching method, thereby finding the first brain wave pre-stored in the database that best matches the existing electroencephalogram waveform. In addition, the grid method selected by the present invention needs to be emphasized. Since different people have different nerve reactions or different levels of excitement, the peaks and troughs of the detected EEG waveforms are also different, and what is stored in the database is only The first brain wave is pre-stored due to a longer conventional level of excitement or reaction, and the first brain wave contains the first text corresponding to it. Then, if we want to find the first brain wave that best matches it, we must take into account the impact of people's current level of excitement and reaction on the current EEG waveform. Therefore, by configuring grids of corresponding sizes according to the distances between the troughs of different EEGs, and counting the number of grids, it will be more reasonable to count the matching first brain waves corresponding to the number of grids.

Wherein, the first threshold and the second threshold may both be the minimum distance between the peaks and troughs of all pre-stored first brain waves calculated in the same manner.

Among them, the number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first electroencephalogram is counted. This number can indicate whether the waveform difference between the two waveforms is larger or smaller. It needs to be explained. Yes, the number of occupied grids is calculated based on rounding. That is to say, if it occupies less than half of a grid, it will not be counted. If it occupies more than half, it will be counted as occupying one grid.

Among them, the maximum amplitude point of all the peaks or troughs of the first segment can be understood as, if the peak is (1,10) and the trough is (3,-20), then the trough is the maximum amplitude point, that is, all the peaks and troughs of the first segment are The statistics of the maximum amplitude point of a wave peak or trough in a segment are the points with the largest absolute value of the maximum amplitude.

The minimum spacing between the abscissas of adjacent troughs of the electroencephalogram is configured as the length of the grid,

Among them, the square of the grid length can be understood as the area of the grid. The grid length configured in the above way can better adapt to different human bodies.

Among them, "then delete the first segment can be understood as" and "and delete the first length on both sides of the adjacent two sides of the deleted first segment in the first length", and the first segment is directly deleted in the waveform diagram, Moreover, the positions of other first segments do not change. That is to say, if this first segment is deleted, the adjacent troughs on both sides of this first segment cannot generally be counted as the minimum first length.

Referring to Figures 1, 3, and 4 of the present invention, a system of consciousness interactive communication methods includes

An image display module configured to display a question image, wherein the question image includes text data, a first button symbol and a second button symbol;

Eye tracker, which is used to collect the eyeball direction and output it as the eyeball direction;

A processor configured to determine whether the eyeball direction is consistent with the first position data corresponding to one of the prestored first button or the second button, and if so, collect the electroencephalogram within a preset time; and determine whether the brain Whether the waveform of the electrogram matches the pre-stored first brain wave, if so, text data corresponding to the first position data is output.

The present invention realizes double verification of the two electrical signals through the above-mentioned double judgment of eyeball orientation and electroencephalogram waveform, so that the accuracy of the output text data is higher. In addition, the eye tracker can be used to determine whether the selected time period is EEG, thus avoiding misjudgments caused by abnormal fluctuations in EEG caused by unconsciousness and misjudgments caused by unconscious eye movement direction.

Among them, the method of "collecting the eyeball direction" may be: collecting the eyeball direction through an eye tracker. The eye tracker is an existing technology.

Among them, "then output text data corresponding to the first position data", that is to say, "then output text data corresponding to the first button or second button corresponding to the first position data", that is, That is, the first button and the second button can respectively correspond to text data as described in the following table:

Among them, the present invention is provided with a database, which is used to pre-store the first position data corresponding to the first button symbol and the second button symbol; and the database is also used to pre-store the first symbol feature corresponding to the first standard radio wave.

Among them, referring to Figures 1, 3, and 4, collecting the eyeball direction and outputting it as the eyeball direction includes the following steps:

Collect the real-time direction of the eyeball orientation;

Determine whether the cumulative duration of the eyeball orientation in the preset direction area exceeds a confirmation threshold, and if so, output the preset direction area as the eyeball direction;

The method of collecting EEG within a preset time includes the following steps:

The starting point of the timeline of the preset time is the same as the starting point of the timeline of time remaining in the preset direction area; the electroencephalogram within the preset time is collected.

The present invention collects the starting time when the eyeballs stagnate as a judgment condition for collecting electroencephalogram, which can facilitate the user to see the text data of the first button or the second button first, so that the user can see the text corresponding to the first button. After the data is collected, and after understanding the above text data, it triggers thoughts about it, thereby intercepting the EEG within the preset time corresponding to the above eyeball direction, so that the collected EEG is more targeted and is an EEG The waveform judgment filters out many unnecessary EEG time periods.

Wherein, in the question image, it can be configured that the left side is a first button and the right side is a second button. The first button is provided with text data corresponding to it, and the second button is provided with text data corresponding to it.

Wherein, the preset time and the confirmation threshold are the same, and they can be 1 second, 2 seconds, 3 seconds, 4 seconds or 5 seconds. That is to say, the time period after the eyeball starts looking is valid.

Of course, the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time remaining in the preset direction area, but the time staying in the preset direction area can be longer than the preset time by 1 second or 2 seconds. seconds or 3 seconds. For example, the preset time is 3 seconds, which means that the eyeballs have been looking for three meters for seconds, and the time remaining in the preset direction area can be 4 seconds, 5 seconds, or 6 seconds, which means that the eyeballs have been collected and started to look. Then the EEG within 4 seconds or 5 seconds or 6 seconds, thereby increasing the accuracy of the collection.

Among them, the preset direction area can be only one area on the left and one area on the right, a total of two areas; or one area each up, down, left, and right, a total of four areas.

Of course, as a deformation structure, it can also be, see Figures 1, 3, and 4. The method of collecting the eyeball direction and outputting it as the eyeball direction includes the following steps:

Collect the real-time direction of the eyeball orientation;

Determine whether the cumulative duration of the eyeball orientation in the preset direction area exceeds a confirmation threshold, and if so, output the preset direction area as the eyeball direction;

The method of collecting EEG within a preset time includes the following steps:

The end point of the time axis of the preset time is the same as the end point of the time axis remaining in the preset direction area; the electroencephalogram within the preset time is collected.

The present invention collects the end time when the eyeballs stagnate as a judgment condition for collecting electroencephalogram, which can facilitate the user to see the text data of the first button or the second button first, so that the user can see the text data corresponding to the first button. Finally, after understanding the above text data, it triggers thoughts about it, thereby intercepting the EEG within the preset time corresponding to the above eyeball direction, thereby making the collected EEG more targeted and providing the basis for EEG analysis. Waveform judgment filters out many unnecessary EEG time periods.

The end point of the time axis of the time remaining in the preset direction area can be understood as the time point when the time remaining in the preset direction area just exceeds the confirmation threshold.

Wherein, the preset time and the time to stay in the preset direction area can be the same, which can be 1 second, 2 seconds, 3 seconds, 4 seconds or 5 seconds, that is, the time period after the eyeballs start to look. efficient.

Of course, the end point of the time axis of the preset time is the same as the end point of the time axis of the time remaining in the preset direction area, but the time of staying in the preset direction area can be longer than the preset time by 1 second or 2 seconds. Seconds or 3 seconds. For example, the preset time is 3 seconds, which means that the eyeballs are looking for three meters for seconds, and the time remaining in the preset direction area can be 4 seconds, 5 seconds, or 6 seconds, which means that the eyeballs are collected and controlled. Start looking at the EEG 1 second or 2 seconds or 3 seconds ago, thereby increasing the accuracy of the collection.

Among them, referring to Figures 2, 3, and 4, the determination of whether the waveform of the electroencephalogram matches the prestored first brain wave includes:

Pre-store the first brain wave corresponding to the text data;

Divide the electroencephalogram waveform into a first segment from two adjacent wave peaks, and determine whether one of the peak amplitude or the trough amplitude of the first segment is greater than a first preset threshold; if not, , then delete the first segment. If so, use the abscissa between the peaks of the adjacent first segments as the first length, and delete the first length on both sides of the adjacent first segment. a length;

Determine whether the minimum first length is lower than the second preset threshold. If so, configure the electroencephalogram waveform in the grid, and configure the second preset threshold as the unit grid length. If not , then configure the minimum first length as the unit grid length;

Coincide the maximum amplitude point of the peak or trough of all the first segments of the EEG waveform with the lower left corner endpoint of one of the unit grids;

The number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first brain wave is counted, and the first electroencephalogram occupying the smallest number of grids is determined to match the waveform of the electroencephalogram. .

The present invention matches the first brain wave with the waveform of the electroencephalogram through the above grid matching method, thereby finding the first brain wave pre-stored in the database that best matches the existing electroencephalogram waveform. In addition, the grid method selected by the present invention needs to be emphasized. Since different people have different nerve reactions or different levels of excitement, the peaks and troughs of the detected EEG waveforms are also different, and what is stored in the database is only The first brain wave is pre-stored due to a longer conventional level of excitement or reaction, and the first brain wave contains the first text corresponding to it. Then, if we want to find the first brain wave that best matches it, we must take into account the impact of people's current level of excitement and reaction on the current EEG waveform. Therefore, by configuring grids of corresponding sizes according to the distances between the wave peaks of different EEGs, and counting the number of grids, it will be more reasonable to count the matching first brain waves corresponding to the number of grids.

Wherein, the first threshold and the second threshold may both be the minimum distance between the peaks of all pre-stored first brain waves calculated in the same manner.

Among them, the number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first electroencephalogram is counted. This number can indicate whether the waveform difference between the two waveforms is larger or smaller. It needs to be explained. Yes, the number of occupied grids is calculated based on rounding. That is to say, if it occupies less than half of a grid, it will not be counted. If it occupies more than half, it will be counted as occupying one grid.

Among them, the maximum amplitude point of all the peaks or troughs of the first segment can be understood as, if the peak is (1,10) and the trough is (3,-20), then the trough is the maximum amplitude point, that is, all the peaks and troughs of the first segment are The statistics of the maximum amplitude point of a wave peak or trough in a segment are the points with the largest absolute value of the maximum amplitude.

The minimum distance between the abscissas of adjacent wave peaks of the electroencephalogram is configured as the length of the grid,

Among them, the square of the grid length can be understood as the area of the grid. The grid length configured in the above way can better adapt to different human bodies.

Among them, "then delete the first segment can be understood as" and "and delete the first length on both sides of the adjacent two sides of the deleted first segment in the first length", and the first segment is directly deleted in the waveform diagram, Moreover, the positions of other first segments do not change. That is to say, if this first segment is deleted, the adjacent wave peaks on both sides of this first segment cannot generally be counted as the minimum first length.

If so, it is determined whether the time during which the eyeball is directed toward the first position data corresponding to one of the first button or the second button exceeds a confirmation threshold.

Of course, as a deformation structure, it can also be, see Figures 1, 3, and 4. Determining whether the waveform of the electroencephalogram matches the prestored first brain wave includes:

Pre-store the first brain wave corresponding to the text data;

Divide the electroencephalogram waveform into first segments from peak to trough, and determine whether one of the peak amplitude or trough amplitude of the first segment is greater than a first preset threshold; if not, delete the first segment. A segment, if so, use the abscissa between the troughs of adjacent first segments as the first length, and delete the first lengths on both adjacent sides of the deleted first segment in the first length;

Determine whether the minimum first length is lower than the second preset threshold. If so, configure the electroencephalogram waveform in the grid, and configure the second preset threshold as the unit grid length. If not , then configure the minimum first length as the unit grid length;

Coincide the maximum amplitude point of the peak or trough of all the first segments of the EEG waveform with the lower left corner endpoint of one of the unit grids;

The number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first brain wave is counted, and the first electroencephalogram occupying the smallest number of grids is determined to match the waveform of the electroencephalogram. .

The present invention matches the first brain wave with the waveform of the electroencephalogram through the above grid matching method, thereby finding the first brain wave pre-stored in the database that best matches the existing electroencephalogram waveform. In addition, the grid method selected by the present invention needs to be emphasized. Since different people have different nerve reactions or different levels of excitement, the peaks and troughs of the detected EEG waveforms are also different, and what is stored in the database is only The first brain wave is pre-stored due to a longer conventional level of excitement or reaction, and the first brain wave contains the first text corresponding to it. Then, if we want to find the first brain wave that best matches it, we must take into account the impact of people's current level of excitement and reaction on the current EEG waveform. Therefore, by configuring grids of corresponding sizes according to the distances between the troughs of different EEGs, and counting the number of grids, it will be more reasonable to count the matching first brain waves corresponding to the number of grids.

Wherein, the first threshold and the second threshold may both be the minimum distance between the peaks and troughs of all pre-stored first brain waves calculated in the same manner.

Among them, the number of grids occupied by the non-overlapping area between the waveform of the electroencephalogram and the first electroencephalogram is counted. This number can indicate whether the waveform difference between the two waveforms is larger or smaller. It needs to be explained. Yes, the number of occupied grids is calculated based on rounding. That is to say, if it occupies less than half of a grid, it will not be counted. If it occupies more than half, it will be counted as occupying one grid.

Among them, the maximum amplitude point of all the peaks or troughs of the first segment can be understood as, if the peak is (1,10) and the trough is (3,-20), then the trough is the maximum amplitude point, that is, all the peaks and troughs of the first segment are The statistics of the maximum amplitude point of a wave peak or trough in a segment are the points with the largest absolute value of the maximum amplitude.

The minimum spacing between the abscissas of adjacent troughs of the electroencephalogram is configured as the length of the grid,

Among them, the square of the grid length can be understood as the area of the grid. The grid length configured in the above way can better adapt to different human bodies.

Among them, "then delete the first segment can be understood as" and "and delete the first length on both sides of the adjacent two sides of the deleted first segment in the first length", and the first segment is directly deleted in the waveform diagram, Moreover, the positions of other first segments do not change. That is to say, if this first segment is deleted, the adjacent troughs on both sides of this first segment cannot generally be counted as the minimum first length.

Finally, it should be noted that the above descriptions are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (2)

1. A conscious interaction communication method is characterized by comprising the following steps of

Displaying a question image, wherein the question image comprises text data, a first button symbol and a second button symbol;

collecting the direction of the eyeball and outputting the direction as the direction of the eyeball;

judging whether the eyeball direction is consistent with first position data corresponding to one of a pre-stored first button or a pre-stored second button, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and if so, outputting text data corresponding to the first position data;

the collecting the eyeball direction and outputting the eyeball direction as the eyeball direction comprises the following steps:

collecting the real-time direction of the eyeball direction;

judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;

the acquiring the electroencephalogram within the preset time comprises the following steps:

the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;

acquiring an electroencephalogram within a preset time;

the judging whether the waveform of the electroencephalogram is matched with the prestored first brain wave comprises the following steps:

pre-storing first brain waves corresponding to the text data;

dividing the waveform of the electroencephalogram into first segments from two adjacent peaks to the peaks, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold, if not, deleting the first segments, and if so, taking the abscissa between the peaks of the adjacent first segments as a first length;

judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;

overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;

counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.

2. A system for a conscious interactive communication method, comprising

An image display module for displaying a question image, wherein the question image includes text data, a first button symbol, and a second button symbol;

the eye movement instrument is used for collecting the direction of the eyeball and outputting the direction of the eyeball;

the processor is used for judging whether the eyeball direction is consistent with the first position data corresponding to one of the pre-stored first buttons or the pre-stored second buttons, and if so, acquiring an electroencephalogram in a preset time; judging whether the waveform of the electroencephalogram is matched with a pre-stored first brain wave, and outputting text data corresponding to the first position data if the waveform of the electroencephalogram is matched with the pre-stored first brain wave;

the collecting the eyeball direction and outputting the eyeball direction as the eyeball direction comprises the following steps:

collecting the real-time direction of the eyeball direction;

judging whether the accumulated time length of the eyeball facing the preset direction area exceeds a confirmation threshold value or not, and if so, outputting the preset direction area as the eyeball direction;

the step of acquiring the electroencephalogram within the preset time comprises the following steps:

the starting point of the time axis of the preset time is the same as the starting point of the time axis of the time kept in the preset direction area;

acquiring an electroencephalogram within a preset time;

the judging whether the waveform of the electroencephalogram is matched with the prestored first brain wave comprises the following steps:

pre-storing first brain waves corresponding to the text data;

dividing the waveform of the electroencephalogram into first segments from two adjacent peaks to the peaks, judging whether one of the peak amplitude or the trough amplitude of the first segments is larger than a first preset threshold, if not, deleting the first segments, and if so, taking the abscissa between the peaks of the adjacent first segments as a first length;

judging whether the minimum first length is lower than a second preset threshold value, if so, configuring the waveform of the electroencephalogram in a grid, configuring the second preset threshold value as a unit grid length, and if not, configuring the minimum first length as the unit grid length;

overlapping the maximum amplitude points of the peaks or the troughs of all the first segments of the waveform of the electroencephalogram with the lower left corner end point of one unit grid;

counting the number of grids occupied by the area which is not overlapped between the waveform of the electroencephalogram and the first electroencephalogram, and judging the first electroencephalogram with the minimum number of grids to be matched with the waveform of the electroencephalogram.