JP2015213734A - Program, information processing device, and eyewear - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly detect blinking.SOLUTION: A program makes a computer execute an acquisition step of acquiring an electrooculogram signal showing the movement of an eye in a vertical direction based on an eye potential detected by each electrode in contact with an eye periphery, a storage step of storing the maximum value and/or the minimum value of the electrooculogram signal of each prescribed period, or the maximum value and/or the minimum value of a differential signal between the electrooculogram signal and an electrooculogram signal before the prescribed period of the electrooculogram signal, a threshold calculation step of using the maximum value and/or the minimum value stored in the storage part to calculate a threshold, and a detection part of using the threshold to detect blinking from the electrooculogram signal or the differential signal.

Description

  The present invention relates to a program, an information processing apparatus, and eyewear.

  A sunglasses-type device for detecting and preventing sleep using an electrooculogram input electrode or an electroencephalogram input electrode attached to a frame of glasses is known (for example, see Patent Document 1).

JP 2008-264551 A

  However, when each electrode is brought into contact with the periphery of the eye and the blink is detected using the electrooculogram detected by each electrode, the magnitude of the electrooculogram varies depending on the fatigue state of the subject. There is a problem in that it is difficult to set a threshold value for determination and the blink cannot be detected appropriately.

  Therefore, an object of the present invention is to be able to detect blinks appropriately.

  The program according to an aspect of the present invention includes an acquisition step of acquiring an electrooculogram signal indicating a vertical movement of the eye based on an electrooculogram detected by each electrode in contact with the periphery of the eye, and the electrooculogram for each predetermined period. The maximum value and / or minimum value of the figure signal, or the maximum value and / or minimum value of the difference signal between the electrooculogram signal and the electrooculogram signal a predetermined time before the electrooculogram signal are stored in the storage unit. A storage step; a threshold calculation step for calculating a threshold value using the maximum value and / or minimum value stored in the storage unit; and a blink is detected from the electrooculogram signal or the difference signal using the threshold value. And a detecting step for performing the processing.

  The threshold value may be updated each time a maximum value and / or a minimum value is stored in the storage unit.

  In the storage step, a maximum value that is a maximum value and a minimum value that is a minimum value may be stored in the storage unit.

  Further, the threshold calculation step calculates a first threshold using the maximum value stored in the storage unit, calculates a second threshold using the minimum value stored in the storage unit, and the detection step includes The blink may be detected from the electrooculogram signal using the first threshold value and the second threshold value.

  The threshold calculation step includes a first calculation step of calculating an average value and a standard deviation for the maximum value and the minimum value stored in the storage unit, and the first calculation step based on the average value and the standard deviation of the maximum value. A second calculation step of calculating one threshold value and the second threshold value based on an average value and a standard deviation of the minimum values.

  In the second calculation step, a value obtained by adding a value obtained by multiplying a standard deviation of the maximum value by a coefficient to the average value of the maximum value is set as the first threshold value, and the minimum value is calculated from the average value of the minimum value. A value obtained by subtracting a value obtained by multiplying the standard deviation of the value by a coefficient may be used as the second threshold value.

  The detecting step includes a first time of the maximum value equal to or higher than the first threshold value and a second time of the minimum value equal to or lower than the second threshold value, the second time closest to the first time or later. If the difference from the time is within a predetermined time, blinks may be detected.

  The threshold calculation step is stored in the storage unit, a determination step for performing mode determination based on the intensity of the electrooculogram signal or the difference signal, a determination step for determining a coefficient according to the determined mode, and A first calculation step of calculating an average of each of the maximum value and the minimum value, a first threshold value obtained by multiplying the average of the maximum value by the coefficient, and a second threshold value obtained by multiplying the average of the minimum value by the coefficient. A second calculation step of calculating.

  In the first calculation step, the maximum value or the minimum value of the past electrooculogram signal or difference signal detected as the blink may be excluded from the average calculation process.

  An information processing apparatus according to another aspect of the present invention includes an acquisition unit that acquires an electrooculogram signal indicating a vertical movement of the eye based on an electrooculogram detected by each electrode that contacts the periphery of the eye, and a predetermined period. A maximum value and / or a minimum value of the electrooculogram signal for each time, or a maximum value and / or a minimum value of a difference signal between the electrooculogram signal and an electrooculogram signal a predetermined time before the electrooculogram signal. A storage unit that stores the threshold value, a threshold value calculation unit that calculates a threshold value using the maximum value and / or the minimum value stored in the storage unit, and an eyeblink from the electrooculogram signal or the difference signal using the threshold value And a detection unit for detecting.

  The eyewear according to another aspect of the present invention includes an eyebrow portion, a frame having a pair of nose pads, a first electrode and a second electrode provided on each surface of the pair of nose pads, and an eyebrow portion. A third electrode provided on the surface; an acquisition unit for acquiring an electrooculogram signal indicating a vertical movement of the eye based on an electrooculogram detected by each electrode; and a maximum value of the electrooculogram signal for each predetermined period And / or a minimum value, or a storage unit that stores a maximum value and / or a minimum value of a difference signal between the electrooculogram signal and an electrooculogram signal a predetermined time before the electrooculogram signal, and the storage unit A threshold value calculation unit that calculates a threshold value using the stored maximum value and / or minimum value; and a detection unit that detects blinks from the electrooculogram signal or the difference signal using the threshold value.

  According to the present invention, blinks can be detected appropriately.

It is a figure which shows roughly an example of the glasses in Example 1. FIG. 1 is a block diagram illustrating an example of a processing apparatus according to a first embodiment. It is a figure which shows roughly the contact position of the electrode with respect to a user. FIG. 3 is a diagram illustrating an example of a configuration of an amplification unit in the first embodiment. It is a figure for demonstrating the reason for providing a buffer amplifier. FIG. 6 is a diagram illustrating another example of the configuration of the amplifying unit in the first embodiment. 1 is a block diagram illustrating an example of a configuration of an external device according to a first embodiment. It is a figure which shows an example of the electrooculogram signal which shows the motion of the perpendicular | vertical direction of eyes. It is a figure which shows an example of the electrooculogram signal at the time of blinking continuously. 6 is a flowchart illustrating an example of blink detection processing according to the first embodiment. 6 is a flowchart illustrating an example of a threshold value calculation process according to the first embodiment. It is a figure which shows roughly the other example of the glasses in Example 1. FIG. FIG. 10 is a block diagram illustrating an example of a configuration of an external device according to a second embodiment. It is a figure for demonstrating the noise of an electrooculogram signal. It is a figure for demonstrating the blink detection in the case of using the threshold value based on a small coefficient. It is a figure for demonstrating the blink detection in the case of using the threshold value based on a big coefficient. It is a figure which shows an example of the threshold value based on the coefficient according to the mode in Example 2. FIG. 10 is a flowchart illustrating an example of blink detection processing according to the second embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and technical applications that are not explicitly described below. That is, the present invention can be implemented with various modifications without departing from the spirit of the present invention. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. The drawings are schematic and do not necessarily match actual dimensions and ratios. In some cases, the dimensional relationships and ratios may be different between the drawings.

[Example 1]
FIG. 1 is a diagram schematically illustrating an example of glasses 100 according to the first embodiment. The glasses 100 include a lens 110 and a frame 120. Glasses 100 and frame 120 are examples of eyewear.

  The frame 120 supports the pair of lenses 110. The frame 120 includes a rim 122, an eyebrow portion (for example, a bridge) 124, an armor 126, a hinge 128, a temple 130, a modern 132, a pair of nose pads 140, a first electrode 152, and a second electrode. 154, a third electrode 156, a ground electrode 158, an electric wire (not shown), the processing device 200, and an amplification unit 250. Depending on the type of glasses 100, there may be no bridge portion of the frame by using a single lens. In this case, the portion between the eyebrows of the single lens is defined as the portion between the eyebrows.

  The pair of nose pads 140 includes a right nose pad 142 and a left nose pad 144. The rim 122, the armor 126, the hinge 128, the temple 130, and the modern 132 are provided in a pair on the left and right.

  The rim 122 holds the lens 110. The armor 126 is provided outside the rim 122 and holds the temple 130 rotatably with a hinge 128. The temple 130 presses the upper part of the user's ear to pinch this part. The modern 132 is provided at the tip of the temple 130. The modern 132 contacts the upper part of the user's ear. The modern 132 is not necessarily provided in the glasses 100.

  The first electrode 152 and the second electrode 154 are provided on the respective surfaces of the pair of nose pads 140 and detect the electrooculogram. For example, the first electrode 152 is provided on the right nose pad 142, and the second electrode 154 is provided on the left nose pad 144.

  The first electrode 152 detects the electrooculogram of the user's right eye. The second electrode 154 detects the electrooculogram of the user's left eye. As described above, the electrode for detecting the electrooculogram is provided on the surface of the nose pad that inevitably contacts the skin of the user. Thereby, the burden given to a user's skin can be reduced compared with making a pair of electrodes contact the circumference | surroundings of a user's eye.

  The third electrode 156 is provided on the surface of the eyebrow portion 124 and detects an electrooculogram. The ground electrode 158 is provided on the surface of the modern 132. When the glasses 100 do not have the modern 132, the ground electrode 158 is provided at the tip of the temple 130. In the first embodiment, the ground electrode 158 is provided on the surface of the left modern 132. The potential detected by the first electrode 152, the second electrode 154, and the third electrode 156 may be based on the potential detected by the ground electrode 158.

  The processing apparatus 200 may be provided in the temple 130, for example. Thus, the design when the glasses 100 are viewed from the front is not impaired. The installation position of the processing apparatus 200 is not necessarily the temple 130, but may be positioned in consideration of the balance when the glasses 100 are worn. The processing device 200 is connected to the amplifying unit 250 via an electric wire. Note that the processing device 200 and the amplifying unit 250 may be connected via wireless.

  The amplifying unit 250 is provided in the vicinity of the first electrode 152, the second electrode 154, and the third electrode 156, and is connected to each electrode to be amplified via an electric wire. The amplifying unit 250 acquires an electrooculogram signal indicating the electrooculogram detected by each electrode. For example, the amplification unit 250 amplifies an electrooculogram signal indicating an electrooculogram detected by the first electrode 152, the second electrode 154, and the third electrode 156.

  In addition, if the amplification unit 250 includes a processing unit that processes an electrooculogram signal, the amplification unit 250 may perform addition / subtraction processing on each electrooculogram signal before amplification or after amplification. For example, the amplifying unit 250 may obtain a reference electrooculogram signal indicating the potential of the first electrode 152 with respect to the third electrode 156. The amplifying unit 250 may obtain a reference electrooculogram signal indicating the potential of the second electrode 154 with respect to the third electrode 156. The signal amplified or processed by the amplification unit 250 is output to the processing device 200.

  The external device 300 is an information processing device having a communication function. For example, the external device 300 is a mobile communication terminal such as a mobile phone and a smartphone possessed by the user. The external device 300 executes processing based on the electrooculogram signal received from the transmission unit 220. For example, when the external device 300 detects from the received electrooculogram signal that the number of blinks of the user has increased, the external device 300 issues a warning for preventing a doze. Details of the external device 300 will be described later.

<Configuration of processing device>
FIG. 2 is a block diagram illustrating an example of the processing apparatus 200 according to the first embodiment. As illustrated in FIG. 2, the processing device 200 includes a processing unit 210, a transmission unit 220, and a power supply unit 230. The first electrode 152, the second electrode 154, and the third electrode 156 are connected to the processing unit 210 via, for example, the amplification unit 250.

  The processing unit 210 acquires and processes the amplified electrooculogram signal from the amplification unit 250. For example, the processing unit 210 may process a reference electrooculogram signal indicating the potential of the first electrode 152 with respect to the third electrode 156. In addition, although the reference | standard electrocardiogram signal attached | subjected "reference | standard" for convenience of explanation, it is contained in the electrooculogram signal. Further, the processing unit 210 may process a reference electrooculogram signal indicating the potential of the second electrode 154 with respect to the third electrode 156.

  At this time, the processing unit 210 may perform processing so as to obtain an electrooculogram signal indicating the vertical movement of the eye based on the electrooculogram detected from each electrode in the right eye and the left eye.

  In addition, if the acquired electrooculogram signal is not digitized, the processing unit 210 performs the digitization process or acquires an electrooculogram signal amplified from each electrode. Add or subtract signals. The processing unit 210 may transmit the electrooculogram signal acquired from the amplification unit 250 to the transmission unit 220 as it is.

  The transmission unit 220 transmits the electrooculogram signal processed by the processing unit 210 to the external device 300. For example, the transmission unit 220 transmits an electrooculogram signal to the external device 300 by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wired communication. The power supply unit 230 supplies power to the processing unit 210, the transmission unit 220, and the amplification unit 250.

  FIG. 3 is a diagram schematically showing the contact position of the electrode with respect to the user. The first contact position 452 represents the contact position of the first electrode 152. The second contact position 454 represents the contact position of the second electrode 154. The third contact position 456 represents the contact position of the third electrode 156. A horizontal center line 460 represents a horizontal center line connecting the center of the right eye 402 and the center of the left eye 404. The vertical center line 462 represents a center line orthogonal to the horizontal center line 460 at the center of the right eye 402 and the left eye 404.

  It is desirable that the first contact position 452 and the second contact position 454 are located below the horizontal center line 460. Further, it is desirable that the first contact position 452 and the second contact position 454 are arranged so that the line connecting the centers of the first contact position 452 and the second contact position 454 is parallel to the horizontal center line 460.

  Further, the first contact position 452 and the second contact position 454 are desirably arranged so that the distance from the first contact position 452 to the right eye 402 and the distance between the second contact position 454 and the left eye 404 are equal. . Further, it is desirable that the first contact position 452 and the second contact position 454 are separated from each other by a certain distance or more.

  The third contact position 456 is preferably located on the vertical center line 462. The third contact position 456 is preferably located above the horizontal center line 460 and away from the first contact position 452 and the second contact position 454. In addition, for example, the distance between the third contact position 456 and the right eye 402 is separated from the distance between the right eye 402 and the first contact position 452, and the distance from the left eye 404 is the second contact with the left eye 404. The distance from the position 454 may be greater than the distance.

  The eyeball is positively charged on the corneal side and negatively charged on the retinal side. Therefore, when the line of sight moves upward, the potential of the first electrode 152 with respect to the third electrode 156 and the potential of the second electrode 154 with respect to the third electrode 156 become negative. When the line of sight moves downward, the potential of the first electrode 152 with respect to the third electrode 156 and the potential of the second electrode 154 with respect to the third electrode 156 become positive.

  When the line of sight moves to the right, the potential of the first electrode 152 with respect to the third electrode 156 becomes negative, and the potential of the second electrode 154 with respect to the third electrode 156 becomes positive. When the line of sight moves to the left, the potential of the first electrode 152 with respect to the third electrode 156 becomes positive, and the potential of the second electrode 154 with respect to the third electrode 156 becomes negative.

  By detecting the potential of the first electrode 152 with respect to the third electrode 156 and the potential of the second electrode 154 with respect to the third electrode 156, the influence of noise can be suitably reduced. In order to make the third contact position 456 as far as possible from the first contact position 452 and the second contact position 454, the inter-brow portion 124 may be disposed at or near the upper end of the rim 122. Further, the third electrode 156 may be provided above the center of the eyebrow portion 124. In this case, it is desirable to adopt the eyebrow portion 124 having a wide vertical width as the arrangement position of the third electrode 156.

  Instead of detecting the potential of the first electrode 152 based on the third electrode 156, the processing unit 210 detects the third electrode based on the reference electrode from the potential of the first electrode 152 based on the reference electrode. The potential of 156 may be reduced. Similarly, instead of detecting the potential of the second electrode 154 with respect to the third electrode 156, the processing unit 210 detects the potential of the second electrode 154 with respect to the reference electrode as a reference. The potential of the three electrodes 156 may be reduced.

  A ground electrode 158 may be used as the reference electrode. Further, a reference electrode may be separately provided in the glasses 100 at a position away from the first electrode 152, the second electrode 154, and the third electrode 156. For example, the reference electrode may be provided on the modern 132 on the right side. Further, the reference electrode may be provided at a portion of the right temple 130 that is in contact with the user's skin.

  The process of subtracting the potential of the third electrode 156 from the potential of the first electrode 152 relative to the reference electrode and the process of subtracting the potential of the third electrode 156 from the potential of the second electrode 154 relative to the reference electrode are as follows: The processing unit 210 may execute, or the amplification unit 250 or the external device 300 may execute. In this case, the signal indicating the potential to be processed is amplified by the amplification unit 250.

<Configuration of amplification unit>
Next, the configuration of the amplification unit 250 will be described. FIG. 4 is a diagram illustrating an example of the configuration of the amplification unit 250 according to the first embodiment. As illustrated in FIG. 4, the amplification unit 250 includes a first amplifier 260 and a second amplifier 270. The first amplifier 260 is an amplifier that is positioned in front of the second amplifier 270 and functions as a buffer amplifier. Hereinafter, the first amplifier 260 is also referred to as a buffer amplifier 260. The second amplifier 270 is an amplifier that functions as a main amplifier. Hereinafter, the second amplifier 270 is also referred to as a main amplifier 270. The signal amplified by the main amplifier 270 is output to the processing device 200 by wire or wireless.

  As for the installation position of the amplification part 250, it is desirable that it is the part 124 between eyebrows. Note that the amplification unit 250 may be provided so as to be embedded in the eyebrow portion 124. As described above, it is desirable that the electrodes be separated as much as possible. However, since the installation positions of the electrodes depend on the shape of the frame 120, there is a limit even if they are separated.

  For this reason, the potential difference between the electrodes may not be sufficiently large, and if noise is mixed in an electrooculogram signal indicating a small potential detected at each electrode, a sufficiently accurate potential can be detected. Will become difficult.

  Therefore, in the first embodiment, the amplifying unit 250 is in the vicinity of the first electrode 152, the second electrode 154, and the third electrode 156 for the purpose of amplifying the detected electrooculogram signal before noise is mixed therein. Is provided. For example, the amplifying unit 250 is preferably provided in a portion between the eyebrows 124 that is close to each electrode and has a relatively large space in the frame 120. Thereby, while the electrooculogram signal detected by each electrode passes an electric wire, the risk that noise mixes and reduces the accuracy of the electrooculogram signal can be reduced.

  Next, the reason why the buffer amplifier 260 is provided at the position preceding the main amplifier 270 will be described with reference to FIG. FIG. 5 is a diagram for explaining the reason why the buffer amplifier 260 is provided. The example shown in FIG. 5 uses the third electrode 156, but the same applies to the first electrode 152 and the second electrode 154.

Since the third electrode 156 touches human skin when wearing the glasses 100, it may be considered that a resistance R ₀ exists between the third electrode 156 and the ground. At this time, the resistance R ₀ is, for example, several hundred kΩ. The main amplifier 270 has an internal resistance R ₁ . At this time, when a normal amplifier is used as the main amplifier 270, the internal resistance R ₁ is several tens kΩ to several hundreds kΩ.

Here, although ideally is that does not flow a current to the main amplifier 270, the internal resistance R ₁ is smaller than the resistance R _0, the current flows to the main amplifier 270 side. Then, the voltage Vi of the electrode and the voltage Vx of the main amplifier 270 are divided and observed. Therefore, a buffer amplifier 260 is provided at a position before the main amplifier 270 so that no current flows into the main amplifier 270 side.

  FIG. 6 is a diagram illustrating another example of the configuration of the amplifying unit in the first embodiment. The amplifying unit shown in FIG. 6 is denoted by reference numeral 250A. The amplification unit 250A includes a buffer amplifier 260, a main amplifier 270, an A / D conversion unit 280, and a wireless communication unit 290. Since the buffer amplifier 260 and the main amplifier 270 have the same functions as those shown in FIG. 4, the A / D conversion unit 280 and the wireless communication unit 290 will be mainly described below.

  The A / D converter 280 converts the signal amplified by the main amplifier 270 from analog to digital. The A / D conversion unit 280 outputs the digitally converted signal to the wireless communication unit 290.

  The wireless communication unit 290 transmits the digital signal converted by the A / D conversion unit 280 to the processing device 200 using wireless communication. Therefore, the wireless communication unit 290 functions as a transmission unit. The wireless communication unit 290 uses wireless communication such as Bluetooth (registered trademark) and wireless LAN. The wireless communication unit 290 may directly transmit a digital signal to the external device 300.

  In the first embodiment, an example is shown in which one buffer amplifier 260 and one main amplifier 270 are provided. In this case, however, the order of the electrooculogram signals from the electrodes may be determined and amplified. Further, a buffer amplifier 260 and a main amplifier 270 may be provided for each electrode.

<Configuration of external device>
Next, the configuration of the external device 300 will be described. FIG. 7 is a block diagram illustrating an example of the configuration of the external device 300 according to the first embodiment. As illustrated in FIG. 7, the external device 300 includes a communication unit 310, a storage unit 320, and a control unit 330.

  The communication unit 310 receives an electrooculogram signal by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wired communication. Here, the received electrooculogram signal is an electrooculogram signal indicating the vertical movement of the eye. The communication unit 310 outputs the received electrooculogram signal to the control unit 330.

  The control unit 330 is a CPU (Central Processing Unit), for example, and controls each unit and performs various arithmetic processes. In the example illustrated in FIG. 7, the control unit 330 includes an acquisition unit 340, a threshold calculation unit 350, and a detection unit 360.

  The acquisition unit 340 acquires an electrooculogram signal indicating the vertical movement of the eye based on the electrooculogram detected by each electrode in contact with the periphery of the eye. The vertical direction here refers to the direction from the top to the bottom of the head with respect to the face. For example, the acquisition unit 340 acquires an electrooculogram signal indicating the vertical movement of the eye received by the communication unit 310.

  The control unit 330 stores the maximum value and / or minimum value of the electrooculogram signal in the storage unit 320 for each predetermined period. The predetermined period is, for example, 200 msec, but is not limited thereto. Further, the predetermined period may be temporally varied by allowing overlap by using a time window.

  In each predetermined period, the control unit 330 stores the maximum value of the maximum value and the minimum value of the minimum value in the storage unit 320 when the maximum value is the maximum value and when the minimum value is the minimum value. Good. Thereby, since a peak can be memorize | stored now, the precision of the threshold value mentioned later can be raised. This is based on the finding that blinks are detected from one set of peaks. Note that a lower limit may be provided for the absolute value of the maximum value and the minimum value to be stored. Thereby, since the maximum value and the minimum value that affect the threshold are not stored, the maximum value and the minimum value whose absolute values are too small are not stored, so that the accuracy of the threshold described later can be further improved. The control unit 330 may detect a local maximum value and a local minimum value for the electrooculogram signal, and store the local maximum value and the local minimum value in the storage unit 320.

  The storage unit 320 is, for example, a RAM (Random Access Memory), and stores the maximum value (or maximum value) and / or minimum value (or minimum value) of the electrooculogram signal for each predetermined period. For example, the storage unit 320 includes a FIFO buffer for maximum value and a FIFO buffer for minimum value. When the storage capacity of the FIFO buffer is full due to the maximum or minimum value data, the oldest data is erased and the latest data is stored, whereby the data stored in the storage area is updated. .

  In addition, the storage unit 320 stores a program that causes a computer to execute blink detection processing described later. This program may be installed in the external device 300 via the Internet or a recording medium such as an SD card, or may be preinstalled. In addition, the storage unit that stores the program may be different from the storage unit 320.

  The threshold calculation unit 350 calculates the threshold using the maximum value and / or the minimum value stored in the storage unit 320. For example, the threshold calculation unit 350 may calculate the absolute value of the threshold from the average value of the absolute values of the maximum value or the minimum value, for example, in order to simplify the process.

  The threshold calculation unit 350 may calculate the first threshold using the maximum value stored in the storage unit 320 and may calculate the second threshold using the minimum value stored in the storage unit 320. . Here, the first threshold value is used to determine that the eye has moved in the vertical direction, and the second threshold value is used to determine that the eye has moved in the vertical direction. Accordingly, threshold values can be set for each of the upward eye movement and the downward eye movement, so that appropriate threshold determination can be performed.

  The threshold calculation unit 350 includes a first calculation unit 352 and a second calculation unit 354. The first calculator 352 calculates an average value and a standard deviation of the maximum values stored in the maximum value FIFO buffer. The first calculation unit 352 calculates the average value and standard deviation of the minimum values stored in the FIFO buffer for the minimum value.

  The second calculator 354 calculates a first threshold based on the average value and standard deviation of the maximum values, and calculates a second threshold value based on the average value and standard deviation of the minimum values. Thereby, for example, the threshold value can be set using an electrooculogram signal indicating the latest state of the subject. Further, the first threshold value and the second threshold value can be changed in accordance with the intensity of a signal indicating the latest state of the subject.

  For example, the second calculation unit 354 sets a value obtained by adding a value obtained by multiplying the standard value of the maximum value by a coefficient to the average value of the maximum value as the first threshold value. The second calculation unit 354 sets a value obtained by subtracting a value obtained by multiplying the standard deviation of the minimum value by a coefficient from the average value of the minimum values as the second threshold value. Thereby, an appropriate threshold value can be set.

  As described above, the threshold value calculation unit 350 updates the first threshold value and the second threshold value every time the maximum value and / or the minimum value is stored in the storage unit 320. As a result, the threshold calculation unit 350 can set the threshold based on the past electrooculogram signal, so that even if the electrooculogram signal becomes weak due to sleepiness and slowing of eye movement. Since the threshold can be set according to the weakened signal, the blink can be detected appropriately. The threshold calculation unit 350 outputs the calculated first threshold and second threshold to the detection unit 360. Note that the threshold calculation unit 350 may calculate the threshold using the maximum value and / or the minimum value stored in the storage unit 320. For example, the threshold value calculation unit 350 may use an average of a predetermined number of local maximum values stored in the storage unit 320 as a first threshold value, or may set an average of a predetermined number of local minimum values stored in the storage unit 320 as a second threshold value. May be. Further, the threshold value calculation unit 350 may calculate the first threshold value and the second threshold value using the standard deviations of the local maximum value and the local minimum value stored in the storage unit 320.

  The detection unit 360 detects blinks from the electrooculogram signal using the first threshold value and the second threshold value calculated by the second calculation unit 354. For example, the detection unit 360 determines the difference between the first maximum time stored in the storage unit 320 that is equal to or greater than the first threshold and the second minimum time stored in the storage unit 320 that is equal to or less than the second threshold. If is within a predetermined time, blink is detected. Here, the second time is the latest time after the first time. The predetermined time is, for example, 500 msec, but is not limited thereto.

  Note that the detection unit 360 detects blinks using the electrocardiogram signals of the right eye and the left eye, and detects the final blink when the blinks are detected in both eyes at a timing within a predetermined range. May be. The detection unit 360 may detect blinks using the average of both electrooculogram signals on the assumption that the right eye and the left eye move in the same manner. Next, a blink detection algorithm will be described using specific electrooculogram signals.

<Example of blink detection algorithm>
FIG. 8 is a diagram illustrating an example of an electrooculogram signal indicating the vertical movement of the eye. The electrooculogram signal S1 shown in FIG. 8 is an electrooculogram signal indicating the vertical movement of one eye. Here, the blink detection algorithm will be described with reference to FIG.

(1) Obtaining maximum and minimum values for each predetermined period The control unit 330 obtains maximum and minimum values for each predetermined period T1 (eg, 500 msec) of the electrooculogram signal S1.

(2) When the maximum value is a maximum value, it is stored in the first FIFO buffer. When the maximum value obtained in (1) is a maximum value, the control unit 330 stores it in the first FIFO buffer (storage unit 320).

(3) When the minimum value is a minimum value, the controller 330 stores the value in the second FIFO buffer. When the minimum value obtained in (1) is the minimum value, the control unit 330 stores the minimum value in a second FIFO buffer in a different area from the first FIFO buffer. Save (storage unit 320). The order of (2) and (3) is not limited. Hereinafter, when the first FIFO buffer and the second FIFO buffer are combined, they are simply referred to as buffers. In FIG. 8, black points on the electrooculogram signal S1 represent values detected as maximum values or minimum values in each period. Note that the maximum value and the minimum value can be obtained using differentiation, for example, using a difference signal of the electrooculogram signal S1.

(4) When the buffer capacity data is stored, the average value and the standard deviation are calculated. The first calculation unit 352 calculates the average value (a1) and the standard deviation (b1) of the maximum values stored in the first FIFO buffer. Is calculated. The first calculation unit 352 calculates the average value (a2) and standard deviation (b2) of the maximum values stored in the second FIFO buffer.

(5) Calculating the threshold The second calculation unit 354 calculates the first threshold and the second threshold using the average value and the standard deviation calculated in (4). The first threshold value (Th1) and the second threshold value (Th2) are calculated by the following equations.
Th1 = a1 + E × b1 Formula (1)
Th2 = a2-E × b2 Formula (2)
Here, the coefficient E is set to 2, for example.
In addition, you may set a lower limit to the absolute value of a 1st threshold value (Th1) and a 2nd threshold value (Th2). As a result, it is possible to prevent erroneous detection of a slight vertical movement of the eye as blinking.

  In FIG. 8, the example of the first threshold value is represented by the symbol TH1, and the example of the second threshold value is represented by the symbol TH2. As shown in FIG. 8, each threshold value varies with the passage of time of the electrooculogram signal, and the threshold value is appropriately changed following the intensity (magnitude) of the electrooculogram signal. . The first threshold value TH1 and the second threshold value TH2 shown in FIG. 8 are merely conceptually showing threshold value fluctuations.

(6) Specify the maximum value greater than or equal to the first threshold and the minimum value less than or equal to the second threshold The detection unit 360 identifies the maximum value greater than or equal to the first threshold calculated in (5). Here, the maximum value to be subjected to the threshold determination is a maximum value that is stored in the first FIFO buffer and has not yet been subjected to the threshold determination.

  Moreover, the detection part 360 specifies the minimum value below the 2nd threshold value calculated by (5). Here, the minimum value to be subjected to the threshold determination is a minimum value that is stored in the second FIFO buffer and has not yet been subjected to the threshold determination. In FIG. 8, the specified maximum and minimum values are represented by black dots surrounded by squares.

(7) Detecting blinks The detection unit 360 includes, for each specified maximum value, the first time of the maximum value and the second time of the specified minimum value, and the latest time after the first time If the difference from the second time is within a predetermined time, the vertical movement of the eye is detected as a blink.

  In FIG. 8, for example, if the difference between the specified maximum value time t11 and the minimum value time t21 specified most recently after t11 is within a predetermined time, the eye movement is detected as a blink. . The predetermined time is, for example, 500 msec.

  The blink detection algorithm has been described above, but this algorithm can appropriately detect blinks even in successive blinks, as will be described next. In particular, experiments have shown that blinking is performed continuously when sleepiness is desired.

<Continuous blink>
FIG. 9 is a diagram illustrating an example of an electrooculogram signal when blinking continuously. The electrooculogram signal S2 shown in FIG. 9 is an electrooculogram signal indicating the vertical movement of one eye. Reference signs T1, TH1, and TH2 shown in FIG. 9 have the same meaning as the same reference signs shown in FIG.

  As shown in FIG. 9, when blinking continuously, the electrooculogram signal swings to the plus side before it largely swings to the minus side. In the example shown in FIG. 9, even if it is a minimum value, an absolute value smaller than a lower limit value (not shown) (a value represented by a triangle surrounding the black circle shown in FIG. 9) is stored in the second FIFO buffer. Not used for calculation of threshold value. This is because when these values are used, the threshold value becomes unnecessarily small, and the blink may be erroneously detected.

  Here, as a general idea, when an eye blink is detected simply by a pair of a local minimum value that comes after a local maximum value in time series with respect to an electrooculogram signal, the first threshold value TH1 and the second threshold value TH2 There are only a pair exceeding the maximum value of t33 and a minimum value of t41. That is, the blink is detected only once from the electrooculogram signal shown in FIG.

  On the other hand, according to the blink detection algorithm described above, blinks are detected by comparing the time difference with the nearest minimum value after the maximum value for each maximum value greater than or equal to the first threshold value with a predetermined time. That is, the minimum value for taking the difference does not necessarily need to be the minimum value that comes after the maximum value.

  Accordingly, in the example shown in FIG. 9, the difference between the first maximum value time t31 and the latest minimum value time t41, and the difference between the second maximum value time t32 and the latest minimum value time t41. If the difference between the third maximum value time t33 and the latest minimum value time t41 is less than or equal to a predetermined time, these are all detected as blinks. As a result, the above-described blink detection algorithm can detect blinks appropriately by reducing detection omissions even when blinks are continuously performed at high speed. Although the method is the reverse of the above method, the blink may be detected using the maximum value immediately before each minimum value on the basis of the minimum value.

<Blink detection processing>
FIG. 10 is a flowchart illustrating an example of blink detection processing according to the first embodiment. The flowchart shown in FIG. 10 shows a state in which the user wears the glasses 100, and the first electrode 152, the second electrode 154, the third electrode 156, and the ground electrode 158 are in contact with the user's skin. The process starts when 300 is set to an operation mode that is a mode for executing blink detection processing.

  In step S102, the acquisition unit 340 acquires an electrooculogram signal indicating the vertical movement of the eye.

  In step S104, the control unit 330 determines whether a predetermined period has elapsed. If the predetermined period has elapsed (step S104-YES), the process proceeds to step S106. If the predetermined period has not elapsed (step S104-NO), the process returns to step S102.

  In step S <b> 106, the control unit 330 stores the maximum value and / or the minimum value in the predetermined period in the storage unit 320.

  In step S108, the threshold value calculation unit 350 calculates a threshold value (first threshold value and second threshold value) using the maximum value and / or the minimum value stored in the storage unit 320.

  In step S110, the detection unit 360 specifies the first time of the maximum value that is equal to or greater than the first threshold, or the second time of the maximum value that is equal to or less than the second threshold. At this time, the detection unit 360 specifies the latest minimum time after the specified first time as the second time.

  In step S112, the detection unit 360 determines whether the second time-first time is smaller than a predetermined time. If this condition is satisfied (step S112—YES), the process proceeds to step S114. If this condition is not satisfied (step S112—NO), the process returns to step S102.

  In step S114, the detection unit 360 detects a movement of the eye between the first time and the second time as a blink.

  Through the above processing, the external apparatus 300 can detect blinks using an appropriate threshold value.

  Next, details of the threshold value calculation processing in step S106 and step S108 will be described. FIG. 11 is a flowchart illustrating an example of threshold calculation processing according to the first embodiment. In step S202 illustrated in FIG. 11, the control unit 330 determines whether the maximum value within the predetermined period is a maximum value or whether the minimum value within the predetermined period is a minimum value. If any of these conditions are satisfied (step S202—YES), the process proceeds to step S204, and if none of these conditions is satisfied (step S202—NO), the process proceeds to step S110.

  In step S <b> 204, the control unit 330 stores the maximum value that is the maximum value or the minimum value that is the minimum value in the storage unit (buffer portion) 320.

  In step S <b> 206, the first calculation unit 352 calculates the average value and standard deviation of the maximum values stored in the storage unit 320. In addition, the first calculation unit 352 calculates the average value and standard deviation of the minimum values stored in the storage unit 320.

  In step S208, the second calculation unit 354 calculates the first threshold value using the average value and standard deviation of the maximum values, and calculates the second threshold value using the average value and standard deviation of the minimum values.

  With the above processing, the external apparatus 300 can calculate an appropriate threshold value according to the intensity of the electrooculogram signal indicating the most recent state of the subject.

  In the present embodiment, the case where the eyewear is glasses has been described. However, eyewear is not limited to this. The eyewear may be any device related to the eye, and may be a face wearing device or a head wearing device such as glasses, sunglasses, goggles and a head mounted display and their frames.

  In this embodiment, the example in which the glasses 100 include the third electrode 156 has been described. However, the glasses 100 are not limited to this. The glasses 100 may not include the third electrode 156. In this case, an electrooculogram indicated by the potential of the first electrode 152 relative to the reference electrode and an electrooculogram indicated by the potential of the second electrode 154 relative to the reference electrode may be transmitted to the external device 300. Here, the ground electrode 158 may be provided at the position of the third electrode 156 to serve as a reference electrode. Further, the ground electrode 158 provided in the left modern may be used as a reference electrode, or an electrode provided separately from the first electrode 152 and the second electrode 154 may be used as a reference electrode.

  In the present embodiment, the example in which the glasses 100 include the nose pad 140 integrated with the rim 122 has been described. However, the glasses 100 are not limited to this. The glasses 100 may include a klings provided on the rim 122 and a nose pad 140 attached to the krings. In this case, the electrode provided on the surface of the nose pad 140 is electrically connected to the electric wire embedded in the frame via the krings.

  In the present embodiment, the first electrode 152 and the second electrode 154 have been described as examples provided below the center of the nose pad 140. However, it is not limited to this. The nose pad 140 may include an extending portion that extends downward, and the first electrode 152 and the second electrode 154 may be provided in the extending portion. This allows the first electrode 152 and the second electrode 154 to be in contact below the eye position even for a user whose nose pad is located directly beside the eye due to individual differences in eye and nose positions. Can do.

  In the present embodiment, the example in which the third electrode 156 is provided on the surface of the eyebrow portion 124 has been described. However, it is not limited to this. The eyebrow portion 124 may include an extending portion that extends upward, and the third electrode 156 may be provided in the extending portion. Furthermore, a movable part that moves the extending part up and down between the extending part and the eyebrow part 124 may be provided so that the position of the third electrode 156 can be adjusted up and down. As a result, even if the user makes the contact position of the third electrode 156 near the eye due to individual differences in the eye position, the contact position of the third electrode 156 can be separated from the eye by adjustment. . Further, in the present embodiment, the position of each electrode is not limited to the position described above, and it is only necessary to be disposed at a position where an electrooculogram signal indicating the vertical movement of the eye can be acquired.

  In the present embodiment, as an example of the external device 300, a mobile communication terminal such as a mobile phone and a smartphone that is separate from the processing device 200 has been described. However, it is not limited to this. The external device 300 may be a unit integrated with the processing device 200. In this case, the external device 300 is provided integrally with the eyewear.

  In the present embodiment, a shield cable may be used as the electric wire to prevent noise from being mixed.

  Further, in this embodiment, the configuration using three electrodes is illustrated in FIG. 1, but a configuration using four or more electrodes may be used. FIG. 12 is a diagram schematically illustrating another example of the glasses in the embodiment. The glasses 600 shown in FIG. 12 are different from the glasses 100 shown in FIG. 1 in that an electrooculogram signal is acquired using four electrodes. Hereinafter, the differences from the glasses 100 shown in FIG. 1 will be described.

  12 includes an upper electrode 652, a lower electrode 654, a left electrode 656, and a right electrode 658. In the example shown in FIG. 12, the upper electrode 652 and the lower electrode 654 are provided on the rim 122, the left electrode 656 is provided on the left temple 130, and the right electrode 658 is provided on the right temple 130. It does not have to be in this position. Note that these electrodes are in contact with a part of the face.

  In the example shown in FIG. 12, the vertical direction of the eye can be detected based on the voltage difference between the upper electrode 652 and the lower electrode 654, and the horizontal direction of the eye is detected based on the voltage difference between the left electrode 656 and the right electrode 658. can do.

[Example 2]
Next, the eyewear in Example 2 will be described. The outline of the eyeglasses as an example of eyewear in the second embodiment is the same as the eyeglasses 100 shown in FIG. Further, the configuration of the processing apparatus in the second embodiment is the same as the configuration shown in FIG.

  In the external device according to the second embodiment, as described in the first embodiment, the case where the maximum value and the minimum value are obtained using the difference between the electrooculogram signals will be described as an example. Furthermore, the external device according to the second embodiment determines the noise mode, calculates a threshold value using the determined mode and the obtained extreme value, and performs blink detection using the threshold value. Hereinafter, determination of the noise mode and threshold calculation in the second embodiment will be described in detail.

<Configuration of external device>
Next, the configuration of the external device 300A according to the second embodiment will be described. FIG. 13 is a block diagram illustrating an example of the configuration of the external device 300A according to the second embodiment. As illustrated in FIG. 13, the external device 300 </ b> A includes a communication unit 310, a storage unit 700, and a control unit 710.

  Similar to the first embodiment, the communication unit 310 receives an electrooculogram signal by wireless communication such as Bluetooth (registered trademark) and wireless LAN, or wired communication. Here, the received electrooculogram signal is an electrooculogram signal indicating the vertical movement of the eye. The communication unit 310 outputs the received electrooculogram signal to the control unit 710.

  The control unit 710 is, for example, a CPU (Central Processing Unit), and controls each unit and performs various arithmetic processes. In the example illustrated in FIG. 13, the control unit 710 includes an acquisition unit 720, a difference calculation unit 730, a threshold calculation unit 740, and a detection unit 750.

  The acquisition unit 720 acquires an electrooculogram signal indicating the vertical movement of the eye based on the electrooculogram detected by each electrode that contacts the periphery of the eye. For example, the acquisition unit 720 acquires an electrooculogram signal indicating the vertical movement of the eye received by the communication unit 310.

  The difference calculation unit 730 calculates a difference signal between the electrooculogram signal acquired by the acquisition unit 720 and the electrooculogram signal a predetermined time before the electrooculogram signal. The predetermined time is, for example, 5 msec. By taking the signal difference, noise tolerance can be increased. Note that taking the difference between these signals is synonymous with performing differentiation.

  The control unit 710 stores the maximum value and / or minimum value of the difference signal in the storage unit 700 for each predetermined period. Further, the predetermined period may be temporally varied by allowing overlap by using a time window. In the second embodiment, an example in which a maximum value and a minimum value are used as the maximum value and the minimum value of a difference signal and a time window is used as a predetermined period will be described.

  The control unit 710 may store the minimum value and the maximum value of the difference signal newly included in the time window in the storage unit 700 as the time window moves. Note that a lower limit may be provided for the absolute value of the maximum value and the minimum value to be stored. Thereby, since the maximum value and the minimum value that affect the threshold are not stored, the maximum value and the minimum value whose absolute values are too small are not stored, so that the accuracy of the threshold described later can be further improved. Further, the control unit 710 may detect a local maximum value and a local minimum value with respect to the difference signal of the electrooculogram signal, and store the local maximum value and the local minimum value in the storage unit 700.

  The storage unit 700 is, for example, a RAM (Random Access Memory), and stores a maximum value and / or a minimum value of the difference signal for each predetermined period. For example, the storage unit 700 includes a maximum value FIFO buffer and a minimum value FIFO buffer. The FIFO buffer can hold, for example, 10 pieces of data. When 10 pieces of maximum or minimum value data are stored, the oldest data is erased and the latest data is stored in the storage area. The stored data is updated.

  The storage unit 700 also stores a program that causes a computer to execute blink detection processing described later. This program may be installed in the external device 300 via the Internet or a recording medium such as an SD card, or may be preinstalled. In addition, the storage unit that stores the program may be different from the storage unit 700.

  The threshold calculation unit 740 calculates a threshold using the maximum value and / or the minimum value stored in the storage unit 700. For example, the threshold value calculation unit 740 may calculate the absolute value of the threshold value from the average value of the absolute value of the local maximum value or the local minimum value in order to simplify the process.

  Further, the threshold value calculation unit 740 may calculate the first threshold value using the local maximum value stored in the storage unit 700, and may calculate the second threshold value using the local minimum value stored in the storage unit 700. . For example, as described in the first embodiment, the threshold calculation unit 740 calculates the first threshold using the standard deviation or average of the maximum value or the second threshold using the standard deviation or average of the minimum value. Or may be calculated.

  The threshold calculation unit 740 includes a first calculation unit 742, a second calculation unit 744, a determination unit 746, and a determination unit 748.

  The determination unit 746 acquires a difference signal from the difference calculation unit 730 and determines a mode based on the intensity (amplitude) of the difference signal. Here, the differential signal mode includes, for example, a low noise mode and a high noise mode. The determination unit 746 determines the mode according to the signal strength of the difference signal. For example, if the absolute value or average value of the signal strength is less than 10, it determines that the mode is the low noise mode, and determines the absolute value or average of the signal strength. If the value is 10 or more, it is determined that the mode is the high noise mode.

  In addition, since the mode does not change frequently, the determination unit 746 switches to the mode only when the same mode continues for a predetermined number of times or for a predetermined time, or switches the mode by providing hysteresis based on the history of the past mode. Good. Further, the determination unit 746 may determine three or more modes instead of determining two modes from the difference signal.

  The determination unit 748 determines the coefficient σ according to the determined mode. For example, the determination unit 748 sets the coefficient σ to σ1 in the low noise mode, and sets the coefficient σ to σ2 (<σ1) in the high noise mode. It is preferable that the number of coefficients σ is prepared as many as the number of modes.

  The first calculation unit 742 calculates the average value of the maximum values stored in the maximum value FIFO buffer. The first calculation unit 742 calculates the average value of the minimum values stored in the minimum value FIFO buffer.

  The second calculation unit 744 calculates a first threshold value based on the average value of the local maximum value and the coefficient σ determined by the determination unit 748, and based on the average value of the local minimum value and the coefficient σ determined by the determination unit 748 A second threshold value is calculated.

  For example, the second calculation unit 744 sets a value obtained by multiplying the average value of the maximum values by the determined coefficient as the first threshold value. The second calculation unit 744 sets a value obtained by multiplying the average value of the local minimum values by the determined coefficient as the second threshold value. Thereby, the threshold value according to the mode of the difference signal of the electrooculogram signal which shows a subject's immediate state, for example can be set. The reason for setting the threshold according to the mode will be described with reference to FIGS.

  Note that the second calculation unit 744 may delete the maximum value and the minimum value of the difference signal detected as blinking in the detection unit 750 described later from the FIFO buffer. As a result, since the signal intensity is strong, the differential signal detected as blink can be excluded from mode determination for determining signal intensity other than blink, and appropriate threshold determination based on an appropriate mode is performed. be able to.

  Further, the threshold value calculation unit 740 updates the first threshold value and the second threshold value each time the maximum value and / or the minimum value is stored in the storage unit 700. Thereby, the threshold value calculation unit 740 can determine the mode based on the difference signal of the electrooculogram signal and set the threshold value according to the determined mode. Therefore, the threshold value calculation unit 740 sets a threshold value suitable for the surrounding environment. Therefore, the blink can be detected appropriately. The threshold calculation unit 740 outputs the calculated first threshold and second threshold to the detection unit 750.

  The detection unit 750 detects blinks from the difference signal of the electrooculogram signal using the first threshold value and the second threshold value calculated by the second calculation unit 744. For example, the detection unit 750 calculates the difference between the first time of the maximum value stored in the storage unit 700 that is equal to or greater than the first threshold and the second time of the minimum value that is stored in the storage unit 700 that is equal to or less than the second threshold. If is within a predetermined time, blink is detected. Here, the second time is the latest time after the first time. The predetermined time is, for example, 500 msec, but is not limited thereto.

  The detection unit 750 detects blinks using the difference signals of the right eye and left eye, and detects a final blink when the blinks are detected in both eyes at a timing within a predetermined range. Also good. The detection unit 750 may detect blinks using an average of both difference signals on the assumption that the right eye and the left eye move in the same manner.

<Problems caused by constant values used for threshold calculation>
Here, with reference to FIGS. 14 to 16, problems due to a constant coefficient used for threshold calculation will be described.

  FIG. 14 is a diagram for explaining noise of an electrooculogram signal. FIG. 14A is an electrooculogram signal in the vertical direction when noise is small (signal strength is small). FIG. 14B is an electrooculogram signal in the vertical direction when noise is large (signal strength is large). The vertical axis | shaft shown in FIG. 14 shows electroocular intensity. In the example shown in FIG. 14, for example, the measurement value of an electrooculogram signal × 1.5 (V) ÷ 2048 is used as an example of the electrooculogram intensity.

  In FIG. 15 and FIG. 16, the blink detection when the coefficient σ that is a constant value is used without performing the mode determination of the difference signal in the second embodiment will be described. FIG. 15 is a diagram for explaining blink detection when a threshold based on a small coefficient σ is used. FIG. 15A shows the first threshold value and the second threshold value for the difference spectrum when the noise is small. As shown in FIG. 15A, since the noise is small, the peak value of the difference signal is small and the coefficient σ is also small, so the absolute values of the first threshold value TH21 and the second threshold value TH22 are small. Since the absolute values of the first threshold value TH21 and the second threshold value TH22 are small, the peaks P11, P12, and P13 are not extreme values at the time of blinking. Misdetected.

  On the other hand, FIG. 15B shows the first threshold value and the second threshold value for the difference spectrum when the noise is large. As shown in FIG. 15B, since the noise is large, the peak value of the difference signal is relatively large, but the first threshold TH21 and the second threshold TH22 are detected to be able to detect blinks because the coefficient σ is small. It becomes an appropriate value. Therefore, in the example shown in FIG. 15B, blinks can be detected appropriately. As described above, when the coefficient σ is set to a small value, it is possible to appropriately detect blinks when the noise is large, but erroneous detection of blinks occurs when the noise is small.

  FIG. 16 is a diagram for explaining blink detection when a threshold value based on a large coefficient σ is used. FIG. 16A shows the first threshold for the difference spectrum when the noise is small. As shown in FIG. 16A, since the noise is small, the peak value of the differential signal is relatively small, but the first threshold value TH31 is an appropriate value that can detect blinks because the coefficient σ is large. . Therefore, in the example shown in FIG. 16A, it is possible to appropriately detect blinks.

  On the other hand, FIG. 16B shows the first threshold value for the difference spectrum when the noise is large. As shown in FIG. 16B, since the noise is large, the peak value of the differential signal is high and the coefficient σ is also large, so that the absolute value of the first threshold value TH31 is large. Therefore, since the absolute value of the first threshold value TH31 is large, even if it is an extreme value at the time of blinking, the extreme value falls below the threshold value due to the threshold determination, and the blink is not detected. As described above, when the coefficient σ is set to a small value, blinks can be detected appropriately when the noise is small, but undetected blinks appear when the noise is large.

  Therefore, in the second embodiment, to solve the problem described with reference to FIGS. 15 and 16, as described above, the mode based on the intensity of the difference signal is determined, and the coefficient is determined according to the determined mode. By setting the threshold based on the determined coefficient, it is possible to prevent both erroneous detection of blinks and non-detection of blinks.

<Mode Determination and Threshold in Example 2>
FIG. 17 is a diagram illustrating an example of a threshold based on a coefficient according to a mode in the second embodiment. FIG. 17A shows the first threshold value and the second threshold value for the differential signal in the low noise mode. In the example illustrated in FIG. 17A, the determination unit 746 determines that the low noise mode is set, and the determination unit 748 determines the coefficient σ to be σ1. Therefore, although the peak value of the difference signal is relatively small, the first threshold value TH41 and the second threshold value TH42 are set to appropriate values because the coefficient σ is large. Therefore, in the example shown in FIG. 17A, erroneous detection of blinks can be prevented and blinks can be detected appropriately.

  FIG. 17B shows the first threshold value and the second threshold value for the differential signal in the high noise mode. In the example illustrated in FIG. 17B, the determination unit 746 determines that the noise mode is high, and the determination unit 748 determines the coefficient σ to be σ2 (<σ1). Therefore, although the peak value of the difference signal is high, the coefficient σ is small, so the first threshold value TH41 and the second threshold value TH42 are set to appropriate values. Therefore, in the example shown in FIG. 17B, undetected blinks can be prevented and blinks can be detected appropriately.

  As in the example illustrated in FIG. 17, in the second embodiment, a mode is determined using a difference signal of an electrooculogram signal, and a more appropriate threshold can be set by using a coefficient corresponding to the mode. As a result, erroneous detection and undetection of blinks can be prevented.

<Blink detection processing>
FIG. 18 is a flowchart illustrating an example of blink detection processing according to the second embodiment. The flowchart shown in FIG. 18 shows a state in which the user wears the glasses 100 and the first electrode 152, the second electrode 154, the third electrode 156, and the ground electrode 158 are in contact with the user's skin, and the external device The process starts when 300 is set to an operation mode that is a mode for executing blink detection processing.

  In step S302, the acquisition unit 720 acquires an electrooculogram signal indicating the vertical movement of the eye.

  In step S304, the difference calculation unit 730 calculates a difference signal between the acquired electrooculogram signal and the electrooculogram signal a predetermined time ago.

  In step S306, the determination unit 746 performs mode determination based on the difference signal. Here, any one of a plurality of preset modes may be determined.

  In step S308, the determination unit 748 determines a coefficient according to the determined mode. The coefficients only need to be associated with each mode in advance.

  In step S310, the threshold calculation unit 740 uses a value obtained by multiplying the average of the maximum values stored in the storage unit 700 by a coefficient as a first threshold, and a value obtained by multiplying the average of the minimum values stored in the storage unit 700 by a coefficient. Is the second threshold.

  The processes in steps S312, S314, and S316 are the same as S110, S112, and S114 shown in FIG.

  With the above processing, the external device 300A can perform mode determination of the electrooculogram signal and detect blinks using an appropriate threshold value for each mode. Further, the modification in the first embodiment can be similarly applied to the second embodiment. Moreover, the mode determination process in Example 2 and the process which makes a coefficient variable are applicable also in Example 1 by replacing the difference signal in Example 2 with the electrooculogram signal.

  As mentioned above, although this invention was demonstrated using the Example, the technical scope of this invention is not limited to the range as described in the said Example. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above-described embodiments. It is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

100 eyeglasses 120 frame 124 eyebrow portion 140 nose pad 152 first electrode 154 second electrode 156 third electrode 158 ground electrode 200 processing device 300 external device 320 storage unit 330 control unit 340, 720 acquisition unit 350, 740 threshold calculation unit 360, 750 Detection unit 730 Difference calculation unit

Claims (11)

An acquisition step of acquiring an electrooculogram signal indicating a vertical movement of the eye based on an electrooculogram detected by each electrode in contact with the periphery of the eye;
The maximum value and / or minimum value of the electrooculogram signal for each predetermined period, or the maximum value and / or minimum value of the difference signal between the electrooculogram signal and the electrooculogram signal a predetermined time before the electrooculogram signal A storage step of storing the value in the storage unit;
A threshold value calculating step for calculating a threshold value using the maximum value and / or the minimum value stored in the storage unit;
A detection step of detecting blinks from the electrooculogram signal or the difference signal using the threshold;
A program that causes a computer to execute.   The program according to claim 1, wherein the threshold is updated each time a maximum value and / or a minimum value is stored in the storage unit. The storing step includes
The program according to claim 1 or 2, wherein a maximum value that is a maximum value and a minimum value that is a minimum value are stored in the storage unit. The threshold calculation step includes:
Calculating a first threshold value using a maximum value stored in the storage unit, calculating a second threshold value using a minimum value stored in the storage unit;
The detecting step includes
The program according to any one of claims 1 to 3, wherein a blink is detected from the electrooculogram signal or the difference signal using the first threshold value and the second threshold value. The threshold calculation step includes:
A first calculation step of calculating an average value and a standard deviation for the maximum value and the minimum value stored in the storage unit;
A second calculation step of calculating the first threshold value based on the average value and standard deviation of the maximum value and the second threshold value based on the average value and standard deviation of the minimum value;
The program according to claim 4, including: The second calculating step includes
A value obtained by adding a value obtained by multiplying a standard deviation of the maximum value by a coefficient to the average value of the maximum value is set as the first threshold, and the standard deviation of the minimum value is multiplied by a coefficient from the average value of the minimum value. The program according to claim 5, wherein a value obtained by subtracting a value is used as the second threshold value. The detecting step includes
The difference between the first time of the maximum value greater than or equal to the first threshold and the second time of the minimum value less than or equal to the second threshold and closest to the first time is a predetermined time. The program according to any one of claims 4 to 6, wherein the program detects blinks within the range. The threshold calculation step includes:
A determination step of performing mode determination based on the intensity of the electrooculogram signal or the difference signal;
A determining step for determining a coefficient according to the determined mode;
A first calculation step of calculating an average of each of the maximum value and the minimum value stored in the storage unit;
A second calculation step of calculating a first threshold value based on the average of the maximum value and the coefficient, and a second threshold value based on the average of the minimum value and the coefficient;
The program according to claim 1, comprising: The first calculation step includes
The program according to claim 8, wherein a maximum value or a minimum value of a past electrooculogram signal or difference signal detected as the blink is excluded from the average calculation process. An acquisition unit for acquiring an electrooculogram signal indicating a vertical movement of the eye based on an electrooculogram detected by each electrode in contact with the periphery of the eye;
The maximum value and / or minimum value of the electrooculogram signal for each predetermined period, or the maximum value and / or minimum value of the difference signal between the electrooculogram signal and the electrooculogram signal a predetermined time before the electrooculogram signal A storage unit for storing values;
A threshold value calculation unit that calculates a threshold value using the maximum value and / or the minimum value stored in the storage unit;
Using the threshold value, a detection unit that detects blinks from the electrooculogram signal or the difference signal;
Information processing apparatus provided. Between the eyebrows,
A frame having a pair of nose pads;
A first electrode and a second electrode provided on the surface of each of the pair of nose pads;
A third electrode provided on the surface of the portion between the eyebrows;
An acquisition unit that acquires an electrooculogram signal indicating a vertical movement of the eye based on an electrooculogram detected by each electrode;
The maximum value and / or minimum value of the electrooculogram signal for each predetermined period, or the maximum value and / or minimum value of the difference signal between the electrooculogram signal and the electrooculogram signal a predetermined time before the electrooculogram signal A storage unit for storing values;
A threshold value calculation unit that calculates a threshold value using the maximum value and / or the minimum value stored in the storage unit;
Using the threshold value, a detection unit that detects blinks from the electrooculogram signal or the difference signal;
Eyewear to prepare.