JP6687639B2 - Information processing method, information processing device, program, and eyewear - Google Patents

Description

  The present invention relates to an information processing method, an information processing device, a program, and eyewear.

  There is known eyewear in which a plurality of electrodes are provided in a frame portion and an electro-oculography signal is obtained from each electrode (see, for example, Patent Document 1).

US Patent Application Publication No. 2004/0070729

  However, in the related art, when each electro-oculography signal is acquired using a plurality of electrodes, each electrode has the following problems. For example, when the electrode and the skin are not in sufficient contact, the contact resistance generated by the contact between the electrode and the skin becomes unstable. Due to this unstable contact resistance, the electro-oculography signal cannot be properly acquired. Note that this problem occurs not only in the electro-oculography signal but also in the biological signal. Further, the contact between the electrode and the skin is not limited to direct contact, but includes indirect contact via cosmetics or gel.

  Therefore, the disclosed technology aims to acquire an appropriate biomedical signal when acquiring each biomedical signal from a plurality of electrodes in which contact resistance may be unstable.

  An information processing method according to an aspect of the disclosed technology is an information processing method executed by a computer having a control unit, wherein the control unit has a plurality of biological electrodes whose positions are symmetrical with respect to a detection target of a biological signal. From each of the plurality of acquired biomedical signals, to identify the first biomedical signal having the largest variation within the predetermined time period from among the acquired plurality of biomedical signals, and the absolute value of the first biometric signal within the predetermined time period. On the basis of the above, the other biological signals within the predetermined period are corrected.

  According to the disclosed technology, it is possible to acquire an appropriate biomedical signal when acquiring each biomedical signal from a plurality of electrodes in which contact resistance may be unstable.

It is a perspective view showing an example from the front of glasses in an example. It is a perspective view showing an example from the back of glasses in an example. It is a block diagram which shows an example of the processing apparatus in an Example. It is a figure which shows roughly the contact position of the electrode with respect to a user. It is a figure which shows an example of a structure of the amplification part in an Example. It is a figure for explaining the reason for providing a buffer amplifier. It is a figure which shows the other example of a structure of the amplification part in an Example. It is a block diagram which shows an example of a structure of the external device in an Example. It is a figure which shows the example of each biological signal when the contact state of each biological electrode and skin is good. It is a figure which shows the example of each biomedical signal when the contact state of one bioelectrode is good and the contact state of the other bioelectrode is bad. It is a figure which shows two electro-oculography signals before correction. It is a figure which shows two electro-oculography signals after correction. It is a figure which shows the value of two biological signals before correction. It is a figure which shows the value of two biological signals after the correction process A. It is a figure which shows the value of two biological signals after the correction process B. 6 is a flowchart illustrating an example of a correction process in the embodiment. 6 is a flowchart illustrating an example of output processing according to the exemplary embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications and application of techniques not explicitly shown below. That is, the present invention can be implemented with various modifications without departing from the spirit of the present invention. Further, in the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. The drawings are schematic and do not necessarily match actual dimensions and ratios. The drawings may include portions having different dimensional relationships and ratios.

[Example]
In this embodiment, an electro-oculogram signal is taken as an example of the biological signal, and a pair of nose pads provided on the glasses is taken as an example of the position of the electrode that can theoretically measure the same signal. FIG. 1 is a perspective view showing an example from the front of the glasses 100 in the embodiment. FIG. 2 is a perspective view showing an example from the rear of the glasses 100 in the embodiment. The glasses 100 include a lens 110 and a frame 120. The glasses 100 and the frame 120 are an example of eyewear.

  The frame 120 supports the pair of lenses 110. The frame 120 includes a rim 122, an eyebrow portion (for example, a bridge) 124, an edge 126, a hinge 128, a temple 130, a modern 132, a pair of nose pads 140, a first electrode 152, and a second electrode. 154, a third electrode 156, an electric wire (not shown), the processing device 200, and an amplification section 250. Depending on the type of the eyeglasses 100, there is a case where there is no bridge portion of the frame by using a single lens. In this case, the part between the eyebrows of the single lens is the part between the eyebrows.

  The pair of nose pads 140 includes a right nose pad 142 and a left nose pad 144. The rim 122, the end piece 126, the hinge 128, the temple 130, and the modern piece 132 are provided in a left-right pair.

  The rim 122 holds the lens 110. The end piece 126 is provided on the outer side of the rim 122, and the hinge 128 rotatably holds the temple 130. The temple 130 presses the upper part of the user's ear to sandwich this part. The modern 132 is provided at the tip of the temple 130. The modern 132 contacts the upper part of the user's ear. The modern 132 does not necessarily need to be provided on the glasses 100.

  The first electrode 152 and the second electrode 154 are provided on the respective surfaces of the pair of nose pads 140 and detect the electrooculogram. For example, the first electrode 152 is provided on the right nose pad 142, and the second electrode 154 is provided on the left nose pad 144.

  The first electrode 152 detects the electrooculogram of the right eye of the user. The second electrode 154 detects the electrooculogram of the left eye of the user. Thus, the electrode for detecting the electro-oculography is provided on the surface of the nose pad that necessarily contacts the skin of the user. As a result, the burden on the user's skin can be reduced compared to the case where two pairs of electrodes are contacted around the user's eyes.

  The third electrode 156 is provided on the surface of the eyebrow portion 124 and detects the electrooculogram. The ground electrode (not shown) may not be provided or may be provided on the surface of the modern 132. When the glasses 100 do not have the modern 132, the ground electrode is provided in front of the temple 130. In the embodiment, the potentials detected by the first electrode 152, the second electrode 154, and the third electrode 156 may be based on the potential detected by the ground electrode.

  The processing device 200 may be provided in the temple 130, for example. This does not impair the design when the glasses 100 are viewed from the front. The installation position of the processing device 200 does not necessarily have to be the temple 130, but may be positioned in consideration of the balance when the glasses 100 are attached. The processing device 200 is connected to the amplification unit 250 via an electric wire. The processing device 200 and the amplification unit 250 may be connected wirelessly.

  The amplification unit 250 is provided in the vicinity of the first electrode 152, the second electrode 154, and the third electrode 156, and is connected to each electrode to be amplified via an electric wire. The amplification unit 250 acquires an electro-oculography signal indicating the electro-oculography detected by each electrode. For example, the amplification unit 250 amplifies the electro-oculography signal indicating the electro-oculography detected by the first electrode 152, the second electrode 154 and the third electrode 156.

  Further, if the amplification section 250 has a processing section that calculates an electro-oculography signal, the amplification section 250 may perform the adjustment processing on each electro-oculography signal before or after amplification. For example, the amplification section 250 may obtain a reference electro-oculography signal indicating the potential of the first electrode 152 with the third electrode 156 as a reference. Further, the amplification section 250 may obtain a reference electro-oculography signal indicating the potential of the second electrode 154 with respect to the third electrode 156. The signal amplified or processed by the amplification unit 250 is output to the processing device 200.

  The external device 300 is an information processing device having a communication function. For example, the external device 300 is a mobile communication terminal such as a mobile phone and a smart phone owned by a user, a personal computer, or the like. The external device 300 executes processing based on the electro-oculography signal received from the transmission unit 220 illustrated in FIG. For example, the external device 300 detects a blink or eye movement from the received electro-oculography signal. As a response when detecting a blink, the external device 300 issues a warning for preventing a drowsiness when detecting that the number of blinks of the user is increasing. Details of the external device 300 will be described later. In addition, the external device 300 may allow the application to be operated based on the detected eye movement. In addition, when the processing device 200 makes a predetermined determination based on the electro-oculography signal, the external device 300 may obtain a determination result from the processing device 200 and perform processing based on this determination result. The predetermined determination is, for example, a blink or a line of sight movement.

<Structure of processing device>
FIG. 3 is a block diagram illustrating an example of the processing device 200 according to the embodiment. As shown in FIG. 3, the processing device 200 includes a processing unit 210, a transmission unit 220, and a power supply unit 230. The first electrode 152, the second electrode 154, and the third electrode 156 are connected to the processing unit 210 via the amplification unit 250, for example. Note that the respective units of the processing device 200 may be provided in a pair of temples instead of being provided in one temple.

  The processing unit 210 includes, for example, a processor and a memory, acquires the amplified electro-oculography signal from the amplification unit 250, and processes it. For example, the processing unit 210 may process a reference electro-oculography signal indicating the potential of the first electrode 152 with respect to the third electrode 156. Note that the reference electro-oculography signal is given a “reference” for convenience of description, but is conceptually included in the electro-oculography signal. Further, the processing unit 210 may process the reference electro-oculography signal indicating the potential of the second electrode 154 with the third electrode 156 as a reference.

  At this time, the processing unit 210 performs processing on the right eye and the left eye based on the electro-oculogram detected from each electrode so as to obtain an electro-oculogram signal indicating vertical and / or horizontal movement of the eye. May be. For example, the processing unit 210 subtracts the potential of the second electrode 154 from the potential of the first electrode 152 to generate a vertical ocular potential signal, or the potential of the first electrode 152 and the potential of the second electrode 154. Alternatively, the horizontal electro-oculography signal may be generated by calculating the average of

  In addition, if the acquired electro-oculography signal is not digitized, the processing unit 210 performs digitization processing, or when the electro-oculography signal amplified from each electrode is acquired, the electro-oculography signal is adjusted. Do processing. In addition, the processing unit 210 may directly transmit the electro-oculography signal acquired from the amplification unit 250 to the transmission unit 220.

  The transmission unit 220 transmits the electro-oculography signal processed by the processing unit 210 to the external device 300. For example, the transmission unit 220 transmits the electro-oculography signal to the external device 300 by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wired communication. The power supply unit 230 supplies power to the processing unit 210, the transmission unit 220, and the amplification unit 250.

  FIG. 4 is a diagram schematically showing a contact position of the electrode with respect to the user. The first contact position 452 represents the contact position of the first electrode 152. The second contact position 454 represents the contact position of the second electrode 154. The third contact position 456 represents the contact position of the third electrode 156. The horizontal center line 460 represents a horizontal center line connecting the center of the right eye 402 and the center of the left eye 404. The vertical centerline 462 represents a centerline orthogonal to the horizontal centerline 460 at the centers of the right eye 402 and the left eye 404.

  The first contact position 452 and the second contact position 454 are preferably located below the horizontal center line 460. Further, it is desirable that the first contact position 452 and the second contact position 454 are arranged such that the line segment connecting the centers of the first contact position 452 and the second contact position 454 is parallel to the horizontal center line 460.

  Further, the first contact position 452 and the second contact position 454 are preferably arranged so that the distance from the first contact position 452 to the right eye 402 and the distance between the second contact position 454 and the left eye 404 are equal. . The first contact position 452 and the second contact position 454 are preferably separated from each other by a certain distance or more.

  The third contact position 456 is preferably located on the vertical centerline 462. Further, it is desirable that the third contact position 456 be located above the horizontal center line 460 and away from the first contact position 452 and the second contact position 454. Further, for example, the distance between the third contact position 456 and the right eye 402 is separated from the distance between the right eye 402 and the first contact position 452, and the distance between the left eye 404 and the left eye 404 is the second contact. It may be separated from the position 454.

  In the eyeball, the cornea side is positively charged and the retina side is negatively charged. Therefore, when the line of sight moves upward, the potential of the first electrode 152 based on the third electrode 156 and the potential of the second electrode 154 based on the third electrode 156 become negative. When the line of sight moves downward, the potential of the first electrode 152 based on the third electrode 156 and the potential of the second electrode 154 based on the third electrode 156 become positive.

  When the line of sight moves to the right, the potential of the first electrode 152 based on the third electrode 156 becomes negative and the potential of the second electrode 154 based on the third electrode 156 becomes positive. When the line of sight moves to the left, the potential of the first electrode 152 based on the third electrode 156 becomes positive and the potential of the second electrode 154 based on the third electrode 156 becomes negative.

  By detecting the potential of the first electrode 152 based on the third electrode 156 and the potential of the second electrode 154 based on the third electrode 156, it is possible to preferably reduce the influence of noise. The glabella portion 124 may be disposed at or near the upper end of the rim 122 in order to separate the third contact position 456 from the first contact position 452 and the second contact position 454 as much as possible. Further, the third electrode 156 may be provided above the center of the eyebrow portion 124. In this case, it is desirable to employ the wide eyebrow portion 124 as the arrangement position of the third electrode 156.

  The processing unit 210 does not detect the potential of the first electrode 152 based on the third electrode 156, but instead of the potential of the first electrode 152 based on the reference electrode, the third electrode based on the reference electrode. The potential at 156 may be reduced. In the same manner, the processing unit 210 does not detect the potential of the second electrode 154 based on the third electrode 156, but instead of detecting the potential of the second electrode 154 based on the reference electrode, the processing unit 210 refers to the reference electrode based on the potential. The potential of the three-electrode 156 may be reduced.

  A ground electrode may be used as the reference electrode. Further, a separate reference electrode may be provided at a position separated from the first electrode 152, the second electrode 154, and the third electrode 156 of the glasses 100. For example, the reference electrode may be provided on the right side modern 132. Further, the reference electrode may be provided on the right temple 130 at a portion in contact with the user's skin.

  Note that the process of subtracting the potential of the third electrode 156 from the potential of the first electrode 152 with reference to the reference electrode and the process of subtracting the potential of the third electrode 156 from the potential of the second electrode 154 with reference to the reference electrode are It may be executed by the processing unit 210, or may be executed by the amplification unit 250 or the external device 300. In this case, the signal indicating the potential of the processing target is amplified by the amplification unit 250.

<Structure of amplification section>
Next, the configuration of the amplification section 250 will be described. FIG. 5 is a diagram illustrating an example of the configuration of the amplification unit 250 according to the embodiment. As shown in FIG. 5, the amplification section 250 has a first amplifier 260 and a second amplifier 270. The first amplifier 260 is located before the second amplifier 270 and functions as a buffer amplifier. Hereinafter, the first amplifier 260 will also be referred to as a buffer amplifier 260. The second amplifier 270 is an amplifier that functions as a main amplifier. Hereinafter, the second amplifier 270 will also be referred to as the main amplifier 270. The signal amplified by the main amplifier 270 is output to the processing device 200 in a wired or wireless manner.

  It is desirable that the installation position of the amplification unit 250 is the part between the eyebrows 124. The amplification section 250 may be provided so as to be embedded in the eyebrow section 124. As described above, it is desirable to separate each electrode as much as possible, but the installation position of each electrode depends on the shape of the frame 120, so there is a limit even if it is separated.

  Therefore, the potential difference between the electrodes may not be sufficiently large, and if noise is mixed in the ocular potential signal indicating the small potential detected by the electrodes, the potential with sufficient accuracy may be detected. It will be difficult.

  Therefore, in the embodiment, the amplifying unit 250 is provided near the first electrode 152, the second electrode 154, and the third electrode 156 for the purpose of amplifying the detected electro-oculogram signal before noise is mixed therein. To be For example, it is preferable that the amplification section 250 is provided near the electrodes and in a portion between the eyebrows portion 124 where there is a space in the frame 120. Accordingly, it is possible to reduce a risk that noise is mixed and the accuracy of the electro-oculography signal is deteriorated while the electro-oculography signal detected by each electrode passes through the electric wire.

  Next, the reason why the buffer amplifier 260 is provided at the position preceding the main amplifier 270 will be described with reference to FIG. FIG. 6 is a diagram for explaining the reason for providing the buffer amplifier 260. The example shown in FIG. 6 uses the third electrode 156, but the same applies to the first electrode 152 and the second electrode 154.

Since the third electrode 156 touches human skin when the glasses 100 are worn, it may be considered that a resistance (contact resistance) R ₀ exists between the third electrode 156 and the ground. At this time, the resistance R ₀ is, for example, several 100 kΩ. Further, the main amplifier 270 has an internal resistance R ₁ . At this time, if a normal amplifier is used as the main amplifier 270, the internal resistance R ₁ is several tens kΩ to several hundreds kΩ.

Here, ideally, no current flows into the main amplifier 270, but if the internal resistance R ₁ is smaller than the resistance R ₀ , a current flows into the main amplifier 270 side. Then, the voltage Vi of the electrode and the voltage Vx of the main amplifier 270 are divided and observed. Therefore, a buffer amplifier 260 is provided at a position preceding the main amplifier 270 to prevent current from flowing into the main amplifier 270 side.

  FIG. 7 is a diagram illustrating another example of the configuration of the amplification unit according to the embodiment. The amplification unit shown in FIG. 7 is denoted by reference numeral 250A. The amplification unit 250A includes a buffer amplifier 260, a main amplifier 270, an A / D conversion unit 280, and a wireless communication unit 290. Since the buffer amplifier 260 and the main amplifier 270 have the same functions as shown in FIG. 5, the A / D conversion unit 280 and the wireless communication unit 290 will be mainly described below.

  The A / D converter 280 converts the signal amplified by the main amplifier 270 from analog to digital. The A / D conversion unit 280 outputs the digitally converted signal to the wireless communication unit 290.

  The wireless communication unit 290 transmits the digital signal converted by the A / D conversion unit 280 to the processing device 200 using wireless communication. Therefore, the wireless communication unit 290 functions as a transmission unit. The wireless communication unit 290 uses wireless communication such as Bluetooth (registered trademark) and wireless LAN. In addition, the wireless communication unit 290 may directly transmit the digital signal to the external device 300.

  In the embodiment, one buffer amplifier 260 and one main amplifier 270 are provided, but in this case, the electro-oculography signals from the respective electrodes may be amplified in a determined order. Further, a buffer amplifier 260 and a main amplifier 270 may be provided for each electrode.

<Structure of external device>
Next, the configuration of the external device 300 will be described. FIG. 8 is a block diagram illustrating an example of the configuration of the external device 300 according to the embodiment. As illustrated in FIG. 8, the external device 300 includes a communication unit 310, a storage unit 320, and a control unit 330.

  The communication unit 310 is, for example, a communication interface, and receives an electro-oculography signal and the like by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wire communication. The communication unit 310 outputs the electro-oculography signal and the like received from the transmission unit 204 of the processing device 200 to the storage unit 320 and / or the control unit 330.

  The storage unit 320 is, for example, a RAM (Random Access Memory), and stores data regarding the processing of the electro-oculography signal. The storage unit 320 includes, for example, a signal storage unit 322, and the signal storage unit 322 stores the electro-oculography signals of the right and left eyes processed by the control unit 330 and / or processed.

  The storage unit 320 also stores a program that causes a computer to execute a signal correction process described below. This program may be installed in the external device 300 via the Internet or a recording medium such as an SD card, or may be pre-installed. The storage unit that stores this program may be different from the storage unit 320.

  The control unit 330 includes, for example, a CPU (Central Processing Unit) and a memory, and controls each unit and performs various arithmetic processes. In the example illustrated in FIG. 8, the control unit 330 includes at least the acquisition unit 332, the identification unit 334, the correction unit 336, the detection unit 338, the comparison unit 340, and the output unit 342.

  The acquisition unit 332 acquires, from the storage unit 320 or the communication unit 310, an electro-oculography signal based on the electro-oculography detected by each electrode that contacts the eye area of the subject. Hereinafter, the acquired electro-oculography signal will be described by taking each electro-oculography signal from the first electrode 152 and the second electrode 154 as an example. In addition, each electro-oculography signal may be each electro-oculography signal from the first electrode 152 and the second electrode 154 based on the third electrode 156 or the ground electrode, for example.

  In addition, the electrodes that contact the periphery of the subject's eyes are provided at positions where the absolute values of the detected electro-oculography signals are theoretically approximately the same. In the case of the present embodiment, the first electrode 152 and the second electrode 154 provided on the pair of nose pads 140 are provided at positions where the absolute values of the detected electro-oculography signals are theoretically substantially the same. Equivalent to. This is because the positions of the first electrode 152 and the second electrode 154 with respect to each eye to be detected have symmetry. In addition, each electrode may be provided at positions at substantially the same distance from the detection target of the biological signal.

  The identifying unit 334 identifies the first electro-oculography signal having the largest variation within the predetermined period, out of the plurality of electro-oculography signals acquired by the acquisition unit 332. The largest variation means that the amplitude of the biological signal is the largest, for example. The predetermined period is, for example, a preset signal sampling period, a variation period of a biological signal detected by the detection unit 338 described later, a moving time window period, or the like.

  For example, the identifying unit 334 compares the amplitudes of the respective electro-oculography signals within a predetermined period, and identifies the electro-oculography signal having the maximum peak as the first electro-oculography signal.

Here, the reason for identifying the signal with the largest variation will be described. The detection accuracy of the biological signal including the electro-oculography signal changes depending on the contact resistance (for example, the resistance R ₀ shown in FIG. 6) that depends on the contact state between the skin and the electrode. For example, in the case of the present embodiment, if the electrode placed on the nose pad is in contact with the nose in an appropriate and stable manner, the value of the contact resistance will be small, and as a result, the measured ocular potential will be affected. Instead, the original electrooculogram can be measured.

  On the other hand, if the electrode installed on the nose pad is not in sufficient contact with the nose, the value of the contact resistance increases, and as a result, the voltage of the ocular potential that should be measured is lowered by the contact resistance and measured. Voltage becomes small. The unstable contact state between the electrode and the nose means that the contact position shifts or the pressure at the time of contact between the electrode and the nose changes.

  Therefore, a larger absolute value (or amplitude) of the electro-oculography signal during fluctuation indicates a state in which the contact resistance is stable, and this state is referred to as a preferable state. Since the application using the electro-oculography signal is basically designed on the premise of this preferable state, it is possible to provide a reliable processing result by using the biomedical signal acquired in the preferable state. Become.

  Normally, the absolute values of the electro-oculography signals from the first electrode 152 and the second electrode 154 should be the same, but depending on the contact state between the electrode and the nose, for example, one contact resistance may be It is stable at a low level, and the contact resistance on the other side is large, and it may be unstable. In this case, one of the electro-oculography signals is hardly affected by the contact resistance, so that the variation thereof is large, and the other electro-oculography signal is reduced by the influence of the contact resistance.

  Therefore, as described above, in a suitable state, the contact resistance is stable at a low level, so that the absolute value of the electro-oculography signal during fluctuation is large (the fluctuation is large), and the control unit 330 has a large absolute value. One electro-oculogram signal (first electro-oculography signal) is regarded as a signal acquired in an appropriate state, and other electro-oculography signals are corrected based on this first electro-oculography signal. In order to perform this correction, the control unit 330 has a correction unit 336.

  The correction unit 336 corrects another electro-oculography signal within the predetermined period based on the absolute value (or amplitude) of the first electro-oculography signal within the predetermined period identified by the identifying unit 334. The other electro-oculography signal is an electro-oculography signal different from the first electro-oculography signal among the acquired electro-oculography signals. As a result, the electro-oculography signal estimated not to be acquired in the preferable state can be the same signal as the electro-oculography signal estimated to be acquired in the preferable state.

The correction unit 336, the other eye potential signal, the same as the first eye potential signal, or the other eye potential signal, the same as the first eye potential signal obtained by inverting the positive and negative signs May be corrected as follows. Hereinafter, this correction process is also referred to as correction process A. Thereby, each electro-oculography signal can be made the same as the electro-oculography signal acquired in a suitable state regardless of positive / negative signs.

  Further, the correction unit 336 corrects other electro-oculography signals using a coefficient based on the ratio between the absolute value of the first electro-oculography signal at a predetermined time point and the absolute value of the other electro-oculography signal at the same time point. Good. Hereinafter, this correction process is also referred to as correction process B. For example, the correction unit 336 may obtain a coefficient from the average value of ratios within a predetermined period and multiply the other electro-oculography signal by the coefficient. Accordingly, it is possible to match the intensity of the electro-oculography signal estimated to be acquired in a suitable state while making use of the profile of the original data of other electro-oculography signals.

  The detection unit 338 detects a change in each electro-oculography signal acquired by the acquisition unit 332. For example, the detection unit 338 executes the processing of detecting the blink or the movement of the line of sight, and sets the period in which the blink or the movement of the line of sight is detected as the above-described predetermined period. Any method may be used as the method of detecting the blink or the movement of the line of sight. An example of the method of detecting the blink or the movement of the line of sight will be described later.

  On the condition that the detection unit 338 sets the predetermined period, the specifying unit 334 may perform the above-described first electro-oculography signal specifying process within the predetermined period. As a result, when the electro-oculography signal does not change, it is not necessary to correct the electro-oculography signal, so that wasteful processing can be eliminated and power saving can be achieved.

  The comparison unit 340 compares the fluctuations of the plurality of electro-oculography signals acquired by the acquisition unit 332. For example, the comparison unit 340 performs comparison in order to check whether or not the degree of variation in each electro-oculography signal is the same. That is, the comparison unit 340 calculates the ratio of the absolute values of the two electro-oculography signals at the same time point within the predetermined period among the plurality of electro-oculography signals, and determines whether the average value of the ratios is within the predetermined range. May be used as the comparison result. For example, if the two electro-oculography signals are the signal A and the signal B, whether or not the absolute value of the value (ratio) of the signal A / the signal B is within a predetermined range (for example, 0.7 to 1.3). It becomes a comparison result.

  The output unit 342 outputs predetermined information regarding the contact state of the electrodes based on the comparison result by the comparison unit 240. The predetermined information is, for example, a warning regarding the contact condition of the electrodes, information regarding the contact state, and information regarding the reliability of the determination result based on the electro-oculography signal.

  For example, the output unit 342 indicates that the comparison result indicates that the fluctuation of the electro-oculography signal from one electrode is larger than the fluctuation of the electro-oculography signal from the other electrode for a preset period (the ratio is predetermined. If it is smaller than the range or larger than the predetermined range), a warning indicating that the contact state of other electrodes is not good is output. Various methods may be used as the output method, such as display on a screen, voice output, and transmission using vibration or the like. As a result, the wearer of the electrode can be notified of the electrode having a poor contact state, and an opportunity to correct the contact state can be provided.

  The output unit 342 indicates that the comparison result indicates that the electro-oculography signal is highly reliable because correction is unnecessary when the fluctuations of the electro-oculography signals are approximately the same (when the ratio is within a predetermined range). Information or information indicating that the eyewear 100 is well balanced when worn may be output. On the other hand, when the comparison result shows that the fluctuations of the respective electro-oculography signals are different (when the ratio is out of the predetermined range), the output section 342 needs to correct the electro-oculography signals, and thus the reliability of the electro-oculography signals is low. It may output information indicating that or the fact that the worn eyewear 100 is out of balance. As a result, the reliability of the electro-oculography signal based on the contact state can be obtained, and this reliability can be added to the processing result relating to the electro-oculography signal, or the user can be informed of the wearing condition of the eyewear 100.

<Blink detection>
Next, detection of the blink by the detection unit 338 will be described. For example, the detection unit 338 obtains the vertical electro-oculography signal Vv from the acquired electro-oculography signal. The ocular potential signal Vv may be generated by subtracting the average of the potentials of the first electrode 152 and the second electrode 154 from the potential of the third electrode 156, or the potential of the first electrode 152 and the second electrode 154. It may be generated from the average.

  The detection unit 338 detects a blink when, for example, the time from the maximum value of the electro-oculography signal Vv to the next minimum value is within a predetermined time. The predetermined time is, for example, 0.5 sec. It should be noted that the above-described blink detection processing is merely an example, and another blink detection algorithm may be used.

<Detection of eye movement>
Next, detection of eye movement by the detection unit 338 will be described. For example, the detection unit 338 divides the acquired electro-oculography signal into a right electro-oculogram and a left electro-oculogram, and when the right electro-oculogram and the left electro-oculogram show a negative potential, the line of sight is changed. Detects that you are looking up. Further, the detection unit 338 detects that the right eye and the left electrogram show a positive potential, the line of sight is down, the right electrogram shows a negative potential, and the left electrogram shows a positive potential. Indicates that the line of sight is to the right, the right eye electrogram shows a positive potential, and the left eye electrogram shows a negative potential, the eye gaze is directed to the left.

The detecting unit 338, the potential of the first electrode 152 by using the eye potential signal Vh generated by subtracting the potential of the second electrode 154, the right line of sight in the case shown is a negative potential, positive When the potential of is indicated, it may be detected that the line of sight is directed to the left. Further, the detection unit 338 indicates a positive potential by using the electro-oculography signal Vv generated by subtracting the average of the potentials of the first electrode 152 and the second electrode 154 from the potential of the third electrode 156. In some cases, it may be detected that it is pointing up, and when a negative potential is shown, pointing down. Further, the detection unit 338 may use the average of the potential of the first electrode 152 and the potential of the second electrode 154 as the ocular potential signal Vv in the vertical direction. In this case, the detecting unit 338 detects that the signal Vv is upward when it is negative, and detects that the signal Vv is downward when it is positive. The above-described eye movement detection process is merely an example, and another eye movement detection algorithm may be used.

<Specific example of signal>
Next, a specific example of each bio-signal acquired from two bio-electrodes whose positions are symmetrical with respect to the detection target of the bio-signal will be described. FIG. 9 is a diagram showing an example of each biological signal when the contact state between each biological electrode and the skin is good. In the example shown in FIG. 9, each bioelectrode and the nose are in stable contact with each other and pressure is appropriately applied, so that the contact resistance of both is sufficiently small. Therefore, in each of the biomedical signals V1 and V2, there is almost no potential drop due to contact resistance, and the voltage has an appropriate voltage value according to the movement of the eye.

  FIG. 10: is a figure which shows the example of each biomedical signal when the contact state of one bioelectrode is good and the contact state of the other bioelectrode is bad. In the example shown in FIG. 10, one bioelectrode is not sufficiently in contact with the nose, so the contact resistance is large, and the other bioelectrode is in good contact, so the contact resistance is sufficiently small. Therefore, the signal V2 from one bioelectrode has a larger potential drop and the measured voltage is smaller than the signal V1 from the other bioelectrode due to the large contact resistance. That is, since the contact resistance increases due to the weak contact state, the voltage measured relatively decreases, so it is preferable to correct the dropped signal.

<Specific example of correction>
Next, a specific example of the correction using the electro-oculography signal will be described. FIG. 11 is a diagram showing two electro-oculography signals before correction. In the example shown in FIG. 11, the two electro-oculography signals are ideally signals whose phases are shifted by 180 degrees. 11A shows the electro-oculography signal S1 obtained from the electrode 152, for example, and FIG. 11B shows the electro-oculography signal S2 obtained from the electrode 154, for example. The example shown in FIG. 11 is an example of a signal when the wearer looks to the left and faces to the right. Further, in the example shown in FIG. 11, the contact state of the electrode 152 is stable and the contact state of the electrode 154 is unstable. In this case, the identifying unit 334 identifies the electro-oculography signal S1 as a signal acquired in a suitable state.

  In FIG. 11B, the fluctuation of the electro-oculography signal S2 is small due to the unstable contact state between the electrode 154 and the skin. Ideally, the electro-oculography signal S2 should be a signal S3 whose positive and negative signs are opposite to those of the electro-oculography signal S1.

  Therefore, the correction unit 336 corrects the electro-oculography signal S2 based on the absolute value of the electro-oculography signal S1 shown in FIG.

  FIG. 12 is a diagram showing two corrected electro-oculography signals. 12A shows the same signal as the electro-oculography signal S1 shown in FIG. 11A, and FIG. 12B shows the signal obtained by correcting the electro-oculography signal S2 shown in FIG. 11B.

  The electro-oculography signal S1 shown in FIG. 12A is the same as the electro-oculography signal S1 shown in FIG. 11A, but this is because the electro-oculography signal S1 is a signal identified by the identifying unit 334. This is because it is not corrected by the correction unit 336.

  The electro-oculography signal S3 shown in FIG. 12 (B) is a signal obtained by correcting the electro-oculography signal S2 shown in FIG. 11 (B) based on the electro-oculography signal S1. In the example illustrated in FIG. 12B, the correction unit 336 corrects the electro-oculography signal S2 so that the electro-oculography signal S1 has the same absolute value and the sign is opposite. As a result, the electro-oculography signal having a small fluctuation is corrected so as to match the signal acquired in a suitable state.

<Specific example of signal value>
13 to 15 show examples of values of the biological signal stored in the signal storage unit 322. Note that in the examples shown in FIGS. 13 to 15, the values of the biological signals sampled at the predetermined sampling rate are stored. The first line shown in FIGS. 13 to 15 indicates a number indicating the sampling order, the second line indicates a value sampled from the biological signal (Ch1), and the third line indicates, for example, the biological signal (Ch2). Indicates the value sampled from. In the examples shown in FIGS. 13 to 15, the biological signal Ch1 and the biological signal Ch2 are signals having no phase difference.

  FIG. 13 is a diagram showing the values of two biological signals before correction. In the example shown in FIG. 13, the predetermined period is, for example, a period in which the sampling number is 10. In this case, the identifying unit 334 identifies the Ch1 signal as the first biological signal because the variation of Ch1 (eg, 30-3 = 27) is larger than the variation of Ch2 (eg, 15-1 = 14). To do.

  FIG. 14 is a diagram showing the values of two biological signals after the correction process A. For example, the correction process A is a process of correcting the value of the biological signal Ch2 having a small fluctuation so as to be the same as the value of the preferable biological signal Ch1 having a large fluctuation. By performing the correction process A, the correction unit 336 can convert all the plurality of biological signals into signals acquired in a suitable state.

  FIG. 15 is a diagram showing the values of the two biological signals after the correction process B. For example, the correction process B is a process of multiplying the value of the biological signal Ch2 having a small variation by a coefficient d based on the ratio of the preferable biological signal Ch1 having a large variation and the biological signal Ch2 and performing the correction.

  In the example illustrated in FIG. 15, the correction unit 336 calculates, for example, the average value of the ratio between the biological signal Ch1 and the biological signal Ch2. Here, the average value of the ratio is ((4/2) + (7/3) + (15/7) +, ..., + (30/15), ...,) / 10 = 2 Suppose Next, the correction unit 336 sets the obtained average value in the coefficient d, and multiplies the biological signal Ch2 by the coefficient. As a result, the value of the biological signal Ch2 shown in FIG. 15 is twice the value of the biological signal Ch1 shown in FIG. Accordingly, the correction unit 336 performs the correction process B to make use of the profile of the detected original signal, and to make a signal that is not acquired in a suitable state similar to a signal acquired in a suitable state. It can be corrected to an intensity signal.

<Operation>
Next, the operation of the external device 300 in the embodiment will be described. FIG. 16 is a flowchart illustrating an example of the correction process according to the embodiment. In the flowchart shown in FIG. 16, the user wears the glasses 100 and the first electrode 152, the second electrode 154, and the third electrode 156 are in contact with the user's skin, and the external device 300 sends a signal. It starts when the operation mode (normal mode), which is the acquisition mode, is set. In the process shown in FIGS. 16 and 17, the biological signal will be described by taking an electro-oculogram signal as an example.

  In step S102 shown in FIG. 16, the acquisition unit 332 starts acquiring a plurality of electro-oculography signals from the glasses 100.

  In step S104, the detection unit 338 determines whether or not each of the acquired plurality of electro-oculography signals has changed within a predetermined period. For example, the detection unit 338 may detect the blink or the movement of the line of sight, and set the period in which the blink or the movement of the line of sight is detected as the above-described predetermined period.

  In step S106, the identifying unit 334 identifies the first electro-oculography signal having the largest variation within the predetermined period, from the acquired electro-oculography signals.

  In step S108, the correction unit 336 corrects another electro-oculography signal within the predetermined period based on the absolute value (or amplitude) of the first electro-oculography signal within the predetermined period. As a result, it becomes possible to correct a signal with a small fluctuation to an appropriate signal value.

  FIG. 17 is a flowchart illustrating an example of output processing according to the embodiment. The flowchart shown in FIG. 17 may be processed automatically or may be processed when the output mode is turned on.

  In step S202, the comparison unit 340 compares changes in each of the plurality of electro-oculography signals.

  In step S204, the output unit 342 determines whether the comparison result by the comparison unit 340 satisfies a predetermined output condition. If the output condition is satisfied (step S204-YES), the process proceeds to step S206, and if the output condition is not satisfied (step S204-NO), the process ends.

  In step S206, the output unit 342 performs output according to the satisfied output condition. For example, when the comparison result indicates that the variation of the electro-oculography signal from one electrode is larger than the variation of the electro-oculography signal from the other electrode during the preset period, the output unit 342 outputs the other electrode. A warning may be output indicating that the contact state of is not good. Further, the output unit 342 may output information indicating that the reliability of the electro-oculography signal is high because the comparison result does not require correction when the fluctuations of the electro-oculography signals are approximately the same. As a result, various outputs can be performed according to the comparison result, if necessary.

  As described above, according to the present embodiment, for example, when acquiring each biomedical signal from a plurality of electrodes that are theoretically provided at positions where similar biomedical signals can be acquired and contact resistance may be unstable, an appropriate biometric The signal can be acquired. In addition, the appropriate biomedical signal acquired in this embodiment can be suitably used in an application that processes the biomedical signal. As a result, since the biometric signal is appropriate, the processing accuracy of the application that processes the biometric signal can be improved.

  In this embodiment, the case where the eyewear is glasses has been described. However, the eyewear is not limited to this. The eyewear may be eye-related equipment, and may be face-wearing equipment or head-wearing equipment such as glasses, sunglasses, goggles and head mounted displays, and frames thereof.

  In this embodiment, the example in which the glasses 100 include the third electrode 156 has been described. However, the glasses 100 are not limited to this. The glasses 100 may not include the third electrode 156. In this case, the electrooculogram indicated by the potential of the first electrode 152 based on the reference electrode and the electrooculogram indicated by the potential of the second electrode 154 based on the reference electrode may be transmitted to the external device 300. Here, a ground electrode may be provided at the position of the third electrode 156 to serve as a reference electrode. Further, a ground electrode provided in the left modern may be used as a reference electrode, or an electrode separately provided at a position separated from the first electrode 152 and the second electrode 154 may be used as a reference electrode.

  In the present embodiment, the example in which the glasses 100 include the nose pad 140 integrated with the rim 122 has been described. However, the glasses 100 are not limited to this. The spectacles 100 may include a crimps attached to the rim 122 and a nose pad 140 attached to the crimps. In this case, the electrode provided on the surface of the nose pad 140 is electrically connected to the electric wire embedded in the frame via the clings.

  In this embodiment, an example in which the first electrode 152 and the second electrode 154 are provided below the center of the nose pad 140 has been described. However, it is not limited to this. The nose pad 140 may include an extending portion that extends downward, and the first electrode 152 and the second electrode 154 may be provided in the extending portion. This allows the first electrode 152 and the second electrode 154 to be contacted below the eye position even for a user whose nose pad is located directly beside the eye due to individual differences in the eye and nose positions. You can

  In this embodiment, the example in which the third electrode 156 is provided on the surface of the glabella portion 124 has been described. However, it is not limited to this. The eyebrow portion 124 may include an extending portion that extends upward, and the extending portion may be provided with the third electrode 156. Furthermore, a movable portion that vertically moves the extending portion may be provided between the extending portion and the glabella portion 124, and the position of the third electrode 156 may be vertically adjustable. As a result, even for a user whose contact position of the third electrode 156 is near the eye due to individual differences in the position of the eye, the contact position of the third electrode 156 can be separated from the eye by adjustment. . In addition, in the present embodiment, the position of each electrode is not limited to the position described above, and may be any position as long as the electro-oculography signal indicating the vertical and horizontal movements of the eye can be acquired.

  In the present embodiment, the ocular potential signal has been used as an example for description, but the processing may be performed using a difference signal of the ocular potential signal or an extreme value difference signal. The difference signal is a signal that represents the difference between a predetermined electro-oculography signal and the electro-oculography signal of this electro-oculography signal a predetermined time before. The predetermined time is, for example, 5 msec. By taking the difference between the signals, the noise resistance can be strengthened. Note that taking the difference between these signals is synonymous with performing differentiation.

  The extreme value difference signal is a signal representing the difference between adjacent extreme values (peak values) of the electro-oculography signal or the difference signal. By taking this extreme value difference signal, it is possible to appropriately determine the line-of-sight movement and the like without being affected by the signal level immediately before the eye moves.

  In the present embodiment, the biological signal can be applied as long as it quantifies biological phenomena such as pulse, heartbeat, electroencephalogram, respiration, and sweating with a sensor.

  In the present embodiment, as the example of the external device 300, a mobile communication terminal such as a mobile phone and a smartphone, which is separate from the processing device 200, has been described. However, it is not limited to this. The external device 300 may be a unit integrated with the processing device 200. In this case, the external device 300 is provided integrally with the eyewear.

  In addition, in this embodiment, a shielded cable may be used as an electric wire to prevent noise from entering.

  Further, in the present embodiment, the configuration using three electrodes is illustrated in FIG. 1, but a configuration using four or more electrodes may be used. In this case, the glasses have an upper electrode, a lower electrode, a left electrode, and a right electrode. For example, the upper electrode and the lower electrode are provided on the rim 122 shown in FIG. 1, the left electrode is provided on the left temple 130, and the right electrode is provided on the right temple 130, but they do not necessarily have to be at this position. There is no. It is assumed that these electrodes are in contact with part of the face. Further, the glasses may have two electrodes at the positions of the first electrode 152, the second electrode 154, and the third electrode 156.

  In the example of four electrodes, the vertical direction of the eye can be detected by the voltage difference between the upper electrode and the lower electrode, and the left-right direction of the eye can be detected by the voltage difference between the left electrode and the right electrode. .

  Further, although the example in which the external device 300 performs the correction process has been described in the present exemplary embodiment, a part or all of the correction process described above may be performed on the eyewear side. For example, the function of the control unit 330 described above can be executed by the processing unit 210 of the processing device 200. The identifying unit 334, the correcting unit 336, the detecting unit 338, and the comparing unit 340 may be provided in the processing device 200 instead of the external device 300. In this case, the processing device 200 executes the flowchart shown in FIG. Also, the comparison unit of the processing device 200. The step S202 shown in FIG. 17 is executed, and the comparison result is transmitted to the external device 300.

  Further, the detection result of the detection unit of the processing device 200 is transmitted to the external device 300 each time it is detected in real time, and when the detection result is aggregated every predetermined time (for example, one minute), the aggregated result is obtained. It may be transmitted to the external device 300.

  When the detection result is transmitted in real time, the output unit 342 of the external device 300 outputs the detection result (for example, whether or not the user blinks). As a result, detailed analysis can be performed at an early stage.

  When the counting result is transmitted every predetermined time, the output unit 342 of the external device 300 outputs a graph (for example, a bar graph, a line graph, or a scatter diagram) in which the counting result for each predetermined time is plotted. As a result, the data is collectively transmitted, so that the power consumption can be reduced.

  Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be added to the above embodiment. It is apparent from the description of the scope of claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

100 glasses 120 frame 124 eyebrow portion 140 nose pad 152 first electrode 154 second electrode 156 third electrode 200 processing device 300 external device 320 storage unit 330 control unit 332 acquisition unit 334 specifying unit 336 correcting unit 338 detecting unit 340 comparing unit 342 Output section

Claims (9)

An information processing method executed by a computer having a control unit,
The control unit is
The reference position of the detection target of the eye potential signal from a plurality of biological electrodes with symmetry acquires each eye potential signal,
Of the obtained plurality of eye potential signal, identifying the largest first eye potential signal varies within a predetermined period,
Correcting the other electro- oculography signal within the predetermined period based on the absolute value of the first electro- oculography signal within the predetermined period,
Comparing variations of each of the plurality of electro- oculogram signals,
Comparison result of the variation among the plurality of eye potential signal, when the average value of the ratio of the absolute value of any two eye potential signals at the same time includes whether it is within a predetermined range, said plurality of Outputting predetermined information including information on the reliability of the two electro- oculography signals based on the contact state of the bioelectrode,
Information processing method for executing. The correction is
The other eye potential signal, the same as the first eye potential signal, or the other eye potential signal, the same as the first eye potential signal obtained by inverting the positive and negative signs, according to claim 1 Information processing method. The correction is
The other electro- oculography signal is corrected using a coefficient based on a ratio between an absolute value of the first electro- oculography signal at a predetermined time and an absolute value of the other electro- oculography signal at the predetermined time. Information processing method described in. The identifying is
The information processing method according to any one of claims 1 to 3, wherein the specific process of the first electro- oculography signal is performed within a period in which a change in each electro- oculography signal is detected, which is set as the predetermined period. . The information processing method according to claim 1, wherein the information regarding the reliability is added to the correction result regarding the electro- oculography signal. When there are multiple detection targets,
Acquiring is
The electro- oculography signal is obtained from each of the plurality of bio-electrodes, the distances between each detection target and each bio-electrode corresponding to each detection target are substantially the same. Information processing method. From a plurality of bioelectrodes where the position based on the detection target of the electro-oculography signal is symmetric, an acquisition unit that acquires each electro-oculography signal,
Of the plurality of electro- oculography signals acquired by the acquisition unit, a specifying unit that specifies the first electro-oculography signal that has the largest variation within a predetermined period,
A correction unit that corrects another electro- oculography signal within the predetermined period based on the absolute value of the first electro- oculography signal within the predetermined period;
A comparison unit that compares the changes in each of the plurality of electro- oculography signals,
Comparison result of the variation among the plurality of eye potential signal, when the average value of the ratio of the absolute value of any two eye potential signals at the same time includes whether it is within a predetermined range, said plurality of An output unit that outputs predetermined information including information on the reliability of the two electro- oculography signals based on the contact state of the bioelectrode,
An information processing apparatus including. On the computer,
The reference position of the detection target of the eye potential signal from a plurality of biological electrodes with symmetry acquires each eye potential signal,
Of the obtained plurality of eye potential signal, identifying the largest first eye potential signal varies within a predetermined period,
Correcting the other electro- oculography signal within the predetermined period based on the absolute value of the first electro- oculography signal within the predetermined period,
Comparing variations of each of the plurality of electro- oculogram signals,
Comparison result of the variation among the plurality of eye potential signal, when the average value of the ratio of the absolute value of any two eye potential signals at the same time includes whether it is within a predetermined range, said plurality of Outputting predetermined information including information on the reliability of the two electro- oculography signals based on the contact state of the bioelectrode,
A program to execute. Frame and
A pair of nose pads provided on the frame,
Each electrode provided on the pair of nose pads,
An acquisition unit that acquires an electro-oculography signal from each of the electrodes,
Of the electro-oculography signals of each electrode acquired by the acquisition unit, a specifying unit that specifies the first electro-oculography signal that has the largest variation within a predetermined period,
A correction unit that corrects another electro-oculography signal within the predetermined period based on the absolute value of the first electro-oculography signal within the predetermined period;
A comparator for comparing the variation of each eye potential signal multiple,
When the comparison result of the variation among the plurality of eye potential signal, the average value of the ratio of the absolute value of any two eye potential signals at the same time includes whether it is within the predetermined range, the respective electrodes An output unit that outputs predetermined information including information regarding the reliability of the two electro- oculography signals based on the contact state of
Eyewear with.

Images