WO2002025615A1

WO2002025615A1 - Alerting device

Info

Publication number: WO2002025615A1
Application number: PCT/GB2001/004172
Authority: WO
Inventors: Martin John Minter-Kemp; David Armitage
Original assignee: Eyeswide Ltd.
Priority date: 2000-09-20
Filing date: 2001-09-19
Publication date: 2002-03-28
Also published as: GB0023172D0; AU2001287888A1; GB0309129D0; GB2390425A; GB2390425B

Abstract

The invention provides a wearable alerting device for monitoring the eyelid movements of a user wearing the device. The device comprises a wearable frame (12), at least one emitter (24) fixed relative to the frame for emitting radiation towards the wearer's eye, at least one photo detector (26, 28) positioned with respect to the emitter for receiving scattered radiation reflected from the eyelid of the wearer when the eyelid is closed or partially closed, and an alerting means (36, 89) for providing an alert signal in response to the detector detecting scattered radiation reflected from the closed or partially eyelid. The frame is preferably in the form of eyeware such as a frame for a pair of spectacles or sunglasses with the emitter(s) and detectors disposed in the frame around the eyepiece lenses. A microprocessor is provided for distinguishing between normal blinking and the onset of sleep.

Description

ALERTING DEVICE

This invention relates to a wearable alerting device for alerting a vehicle driver, train driver, machine operator or watch keeper or the like in response to detected physical indications of personal tiredness.

Each year in the United Kingdom thousands of people are killed or injured in vehicle accidents on the road. In particular, around 250 people are killed or injured on motorway hard shoulders. Research has shown that the majority of accidents on motorway hard shoulders involves stationary vehicles being hit by other vehicles. Most accidents are caused by vehicles veering off the main carriageway due to lack of driver concentration, usually because the driver has fallen asleep at the wheel. The more serious accidents, as indicated by fatalities occur on downhill bends and there is a particular problem in the United Kingdom with right hand bends where vehicles are more likely to wander onto the hard shoulder if the driver of the vehicle has fallen asleep. In countries where vehicles drive on the right hand side of the road, down hill left hand bends present similar problems. This type of accident is more common on motorways and dual carriageways due to the monotonous nature of the road and the relatively high vehicle speeds.

Although regulations exist to restrict the number of hours drivers of commercial vehicles may drive within a given period or at any one time without a rest break, there are no regulations which restrict hours for drivers of non-commercial vehicles. There is a requirement therefore to reduce the incidence of road accidents due to driver tiredness. There is a similar requirement to reduce accidents in the work place due to tiredness of machine operatives and the like including, pilots marine watchkeepers, train drivers and security personnel, for example.

According to an aspect the present invention there is a wearable device for monitoring eyelid movements of a user wearing the said device; comprising: a wearable frame; at least one detector for receiving scattered radiation reflected from the closed or partially closed eyelid of the wearer; and, alerting means for providing an alert signal in response to the said detector detecting radiation reflected from the eyelid.

According to another aspect of the invention there is provided a wearable device for monitoring eyelid movements of a user wearing the said device; comprising: a wearable frame; an emitter fixed relative to the said frame for emitting radiation towards the wearer's eye; a detector positioned with respect to the said emitter for receiving scattered radiation reflected from the closed or partially closed eyelid of the wearer; and, alerting means for providing an alert signal in response to the said detector detecting radiation reflected from the eyelid.

This aspect of the invention is based in part on the observation that the onset of tiredness is associated with an increase in the rate of blinking followed by increased periods of eyelid closure before full closure and sleep. By ignoring normal rates of blinking it is possible for the detector to detect the onset of tiredness of the wearer by detecting part of the scattered radiation reflected from the closed or partially closed eyelid of the wearer. This may be due to closure of the eyelid or significant changes in the rate of blinking of the wearer. In this way, by continually directing non harmful radiation from the emitter towards the wearers eye, it is possible to continuously monitor the state of alertness of the wearer. The alerting means readily enables the wearer to be alerted to the onset of tiredness so that the wearer may take appropriate action as required. For instance, a driver of a vehicle wearing such a device may be encouraged to take a break and rest before continuing with their journey.

Preferably, the said detector is positioned with respect to the said emitter to receive scattered radiation reflected from the closed or partially closed eyelid only. This is advantageous since it can avoid the problem of the detector receiving specular reflections of the emitted light from the surface of the wearer's eyeball. The invention contemplates embodiments where the specular reflection from the surface of the eyeball is monitored together with the scattered reflection from the closed or partially closed eyelid to distinguish between the eye being open or closed. It is preferable however that the detector is positioned so that the detector receives the scattered radiation from the closed or partially closed eyelid and not the specular reflection from the surface of the eyeball. In this way it is possible to overcome problems associated with the eyeball not being 100 % spherical. The cornea of the eyeball has a different curvature to the remainder of the eyeball surface. In this respect the eyeball surface can present a different curvature to the emitter when the eye is looking in different directions so that the reflection angle of the emitted light changes when the eyeball moves within the eye socket. For instance, the cornea may present a relatively small angle of incidence for radiation emitted from the emitter when the wearer is looking straight ahead but present a greater angle of incidence when the wearer is looking downwards. Since the angle of the reflected radiation with respect to the plane normal to the incident point on the surface of the eyeball is the same as the angle of the emitted radiation with respect to the normal plane, the angle between the emitted and reflected beams will increase as the curvature at the incident point at the eyeball decreases. In this way the detector may detect radiation reflected from the surface of the eye ball for only limited ranges of movement of the eyeball within the eye socket. This problem is readily over come by positioning the detector so that it can only detect the scattered radiation that is reflected from the closed or partially closed eyelid.

In preferred embodiments, the said detector is positioned with respect to the said emitter so that radiation from the said emitter is reflected on a path from the surface of the wearer's eyeball divergent from the path between the detector and the said eyeball surface, independently of the incident position of the emitted radiation on the said eyeball. This can enhance the reliability of the device in that light reflected from the surface of the wearer's eyeball is reflected on a path divergent from the path between the detector and the eyeball, so that detection of non-scattered radiation can be readily avoided. Preferably, the said detector is positioned adjacent to the said emitter so that the detector receives reflected radiation from the surface of the closed or partially closed eyelid along a path substantially co-incident with the path of the emitted radiation. This further enhances the above mentioned advantages since the detector will only detect scattered reflections of emitted radiation if the reflected surface is substantially normal to the direction of the emitted radiation towards the eyeball. The position of the detector with respect to the emitter is analogous to an optical filter, in that when the detector is positioned closely adjacent to the emitter it will only detect scattered reflected radiation from the skin of the wearers eyelid.

In preferred embodiments the emitter comprises a light emitting diode (LED). This enables low cost light weight components to be used. LEDs are particularly suitable as they are highly efficient and therefore conserve battery life.

In preferred embodiments, the radiation is infrared radiation, in this way the radiation will be invisible to the wearer and therefore will not distract the wearer when wearing the wearable device. Infrared LEDs are considered to be relatively safe depending on the emission wavelength, source intensity and angle that the radiation subtends at the eye.

In another embodiment, the radiation is visible light. An embodiment of this type is envisaged for use by optometrists and opticians and the like for adjusting the relative positions of the detector and emitters with respect to the frame for a particular user. In preferred embodiments the alerting means is an audible or vibration alarm. Additionally or alternatively the alerting means may be a visible alarm. This readily enables the wearer to be alerted to the onset of tiredness and ensure that the wearer does not fall asleep.

Preferably the radiation is pulsed radiation. In this way it is possible to reduce the amount of radiation incident on the eyeball. This also readily enables battery life to be conserved. The radiation may be in a coded sequence of pulses so that spurious indications of scattered radiation do not cause an alert signal to be generated.

Preferably, the device further comprises processing means for processing output signals from the said detector over a pre-determined period of time to determine whether the detected radiation is representative of closure or blinking of the wearer's eyelid. In this way it is possible to distinguish between normal blinking, increased blinking or prolonged closure of the wearers eyelid.

In preferred embodiments, the device further comprises signal transmission means for ttansmitting a signal representative of the wearer's eyelid state, that is closed or open, to a data recorder or vehicle engine control system. In one particular embodiment it may be desirable to monitor the tiredness pattern of a wearer, say a vehicle driver, so that tiredness data may be analysed to determine when it may be appropriate for individuals to take a break from their usual activity, and /or rest between periods of activity. In one embodiment, the data recorder may be a modified tachograph recorder of the type used to record vehicle parameters and^* driver activity patterns in commercial vehicles. In another embodiment the transmitted signal may be used as an input to the engine control system of the vehicle for reducing the speed of the vehicle in response to the detected onset of sleep, for example the input signal may disengage the vehicle's cruise control system and/or the fuel supply to the vehicle's engine.

Preferably, the transmission link is a wireless link. In this way movement of the wearer is unrestricted since the signal is transmitted by wireless communication means to the data recorder or the vehicle engine control system. In preferred embodiments the wireless link is a low power infrared data link. This type of data link is appropriate for low data rates and short range data communications.

Preferably the angular orientation of the said emitter and/or detector with respect to each another and/or with respect to the said frame is adjustable.

In preferred embodiments the emitter and/or detector is mounted in a rotatable spherical bearing so that the orientation can be adjusted to suit the physical characteristics of the wearer's face.

In preferred embodiments, the frame comprises a spectacle frame. In this way, it is possible to incorporate the device according to the above aspects of the invention, either in a conventional spectacle frame prescription lenses, sunglasses or eye protection glasses and the like. In preferred embodiments, either the said detector or the said emitter or both are mounted centrally in the region of the nosepiece of the spectacle frame. In this way, the position of the emitter and/or detector is less obtrusive to the wearer compared with say side mounted positions.

Various embodiments of the invention will now be more particularly described by way of example only, with reference to accompanying drawings, in which:

Figure 1 is a schematic view of a wearable device incorporating an embodiment of the present invention in a spectacle frame; Figure 2 is a circuit diagram for an embodiment of the present invention;

Figure 3 is a schematic representation showing the position of an emitter and detector with respect to the eyeball of the wearer of the device shown in Figure 1 ; and,

Figure 4 is a schematic view similar to that of Figure 3 with the eyeball closed by the eyelids of the wearer. Figure 5 is a flow chart for appropriate software for implementing an embodiment of the invention; and

Figure 6 is a circuit diagram comprising a microprocessor for implementing an embodiment of the invention.

Referring to Figure 1 , a pair of spectacles 10, which may comprise optical correction spectacles, sunglasses, eye protectors or the like, comprises a structural frame 12 which includes a pair of arms 14, a cross member 16 on which is mounted a lens 18 having a generally U shaped nose bridge support 20 for resting the eyewear item on the nose of a wearer. The arms 14 are hinged at 22 in conventional manner to the cross member 16.

The eyewear item comprises a infrared signal emitter 24 and a pair of infrared optical detectors 26, 28 mounted on the nose bridge 20 and having their respective optical axis positioned in a direction corresponding to that of the eye ball of the wearer.

The emitter 24 and detectors 26 and 28 constitute part of an electrical circuit as shown schematically in Figure 2. The emitter 24 comprises an infrared LED having a relative narrow half angle of 28° and a foot print of 2mm². Infrared LEDs of this type are readily available from a number of manufacturers including Agilent Technologies, a division of Hewlett Packard. The detectors 26 and 28 are also narrow angle infrared photo detectors having a relatively narrow half angle of 28°. The infrared LED emitter 24 is connected to a mark space generator 30 in the form of a standard signal generator for driving the infrared emitter 24 with pulses having a mark/space ratio of 10/1. In the illustrated embodiment the LED on period is approximately 1msec. The output from the infrared detector 26 is connected to a transimpedance amplifier 32, the output of which is connected to the output of the reference infrared detector 28 and the inverting input of a Schmitt Trigger 34. The anode of infrared detector 26 and the cathode of infrared detector 28 are connected to the summing input of transimpedance amplifier 32. The cathode of infrared detector 26 and the anode of infrared detector 28 are connected to signal ground. This arrangement amplifies the difference in signals between the infrared detectors 26 and 28, in effect nulling background light. The output of transimpedance amplifier 32 is connected to the inverting input of a Schmitt trigger 34. The non-inverting input of the Schmitt Trigger is connected to a reference voltage between +/-5 volts. The output of the Schmitt Trigger drives a audible alarm 36 which maybe a Piezo electric effect sounder 36. The Circuit of Figure 2 produces a click in the sounder 36 when a blink is detected and a continuous train of clicks if a closed eye is detected. Although not shown in Figure 2, the output of the mark space generator 30 and transimpedance amplifier 32 may also be connected to a microprocessor which is programd to distinguish between normal blinking and increased rates of blinking and eye closure when the emitted radiation is detected by detectors 26 and 28. In preferred embodiments the components of the circuit shown in Figure 2 are incorporated within the frame 12 of the eye piece item 10 together with the microprocessor and a power source in the form of a single cell or multiple cell battery. In addition, the frame 12 may include micro switches in the hinges 22 which automatically switch the circuit of Figure 2 on and off when the arms 14 are opened and closed with respect to the member 16.

Referring now to Figures 3 and 4, the emitter 24 and detector 26 are positioned closely adjacent to each other on the nose bridge 20 so that radiation emitted from the emitter towards the eye ball 38 of the wearer is reflected from the surface of the eyeball in a direction 40 along a path which is divergent from the radiation path between the detector and the point of incidence 42 on surface of the eye ball 38. In this way, the detector cannot detect the specular reflected radiation from the surface of the eye ball whatever the curvature of the eye ball is at the points of incidence 42, that is to say, the detector is positioned in a blind spot with respect to the specular reflection from the smooth surface of the eyeball regardless of the position and the orientation of the eye ball in the eye ball socket.

Referring now to Figure 4, when the wearer blinks or closes their eyelid 44 over the surface of the eye ball the emitted radiation from the emitter is scattered in a plurality of directions 46 due to the characteristics of the skin of the eyelid such that a portion of the scattered reflected light is directed to the detector 26. By monitoring the frequency that the detector detects closure or part closure of the eye lid as in Figure 4, the microprocessor can determine whether the eye lid closure is due to normal blinking, the onset of tiredness due to an increase in the rate of blinking, and/or whether the eye lid remains closed for a predetermined period indicating closure of the eye lid due to tiredness. If increased blinking or closure due to tiredness is determined the microprocessor activates the alarm 36 in order to ensure that the wearer is fully awake. After such an alert the alarm circuit must be reset by a switch (not shown).

In other embodiments, the circuit shown in Figure 2 may be connected to a communications interface comprising an infrared wireless communications link to a receiver element in a vehicle aircraft or train, for example. Data from the circuit of Figure 2 can then be transmitted to a receiver where it may be recorded in a vehicle data recorder such as a tachograph or maybe used as an input signal to the vehicle's fuel control system so that the vehicle's cruise control system and/or fuel supply may be de-activated in response to the driver of the vehicle falling asleep.

In further embodiments, a plurality of infra red LEDs ancl photodetectors are arranged around the rim of each lens, for example four LEDs and eight pairs of photodetectors mounted, in pairs, around the lens holders of the frame. In operation, one of the LEDs is switched on for 1 millisecond every 10 milliseconds and the light reflected from the eyeball and eyelids is monitored by each of the photo detector pairs, one of which monitors the background light and the other monitors infrared light scattered from the eyelid.

There are a number of suitable algorithms for the detection of the closed eye condition and the invention contemplates spectacles which are programd with the optimum detection criteria for the particular wearer. Appropriate software determines the best combination of LED and photodetectors for each of the wearer's eyes. Once the optimum LED and photodetectors have been selected an appropriate algorithm for the LED and photodetectors is downloaded to the microprocessor in the frame. It is envisaged that this activity will be undertaken by suitably qualified personnel such as opticians and the like so that each wearable alerting device of the present invention is optimised according to the physical characteristics of the wearer.

In preferred embodiments, a microprocessor integrated in the frame is programd to distinguish between normal blinking, increased rates of blinking, and eye closure when the emitted radiation is detected by the photo detectors. In one embodiment the processor is programd in accordance with the steps shown in the flow chart of Figure 5. Briefly stated, when the device is switched on in Step 51 the system is initialised in Step 52 and the left eye photo detectors are scanned in Step 53 and the data obtained from these detectors is stored. In Step 54 an infrared LED (selected for the wearer) is switched on for the left eye. The system then waits 300 microseconds at Step 55 and each of the left eye photo detectors are read and the data obtained from the detectors is stored in a suitable data storage means in Step 56. The left hand LED is switched off in Step 57. The stored data is then analysed to determine whether a closed eye condition exists in Step 58. If a closed eye condition is detected a counter parameter SUB-BLINK is incremented in Step 59. The parameter SUB-BLINK holds the number of consecutive program cycles in which a closed eye condition is detected. The program cycle is repeated every 10 milliseconds. If only one or two consecutive cycles detect a closed eye condition, the program interprets this as noise which is ignored. In Step 60 me value of the parameter SUB-BLINK is compared with the value of the predetermined parameter ALARM VALUE which is the number of consecutive program cycles, indicative of a closed eye condition, that need to be detected to be certain that the wearer has fallen asleep. This value is determined for the wearer prior to being downloaded to the microprocessor. If the comparison in Step 60 is positive an alarm signal is generated in Step 61. If the comparison in Step 60 is negative the value of the parameter SUB-BLINK is compared with the predetermined value of the parameter BLINKVAL which is the number of consecutive cycles needed to detect a closed eye condition for the program to be certain that a blink has occurred. In a similar way to the value of the parameter ALARM VALUE the value of the parameter BLINK VALUE is determined when the device is set up for the particular wearer. If the comparison in Step 62 is positive, that is to say the value of the parameter SUB-BLINK is greater than the value of the parameter BLINKVAL, the parameter BLINK is incremented and the parameter TIMER #1 is reset. The parameter BLINK is a measurement of the current blink rate and is equal to the number of times that a blink has been detected minus the number

of times a normal blink would have occurred in the relevant time period. The parameter TIMER #1 counts the time between each blink. In Step 64 the value of the parameter BLINK is compared with the predetermined value of the parameter MAX BLINK which is the number of blinks expected in the relevant time period if the wearer is tired. If the blink rate of the wearer is determined to be greater than the predetermined value set by the parameter MAXBLINK in Step 64, a sleep warning alarm is activated in Step 65 to provide the wearer with an indication that they are experiencing the onset of sleep. If the comparison in Step 58 is negative, in that the photo detectors do not detect a closed eye condition, parameter SUB-BLINK is decremented in Step 66 and the program proceeds to Step 67 where the value of the parameter TIMER #1 is compared with the predetermined parameter TIMEVAL which is the time interval between the wearers normal, non-tired, blinks as determined prior to use for the particular wearer. The program also proceeds to Step 67 if the comparison in Step 62 is negative. If the comparison of Step 67 is positive, that is to say the value of the parameter TIMER #1 is greater than the value of the parameter TIME VALVE, the program proceeds to Step 68 where the parameter BLINK is decremented and the parameter TIMER #1 is reset. If the comparison in Step 67 is negative the parameter TIMER #1 is incremented in Step 69.

Following Step 61, 65, 68 or 69 the program proceeds to Step 70 where the program determines whether a user button is switched on. The user button is a miniature press switch mounted on a side of the frame to provide a means for the user to cancel the alarm and to switch from the main program mode to, for example a test program mode for checking that the device is operating correctly. If the user button is on, the program proceeds to Step 71 where the alarm is cleared and the parameter TIMER #2 measures the time that the user button is held on. In Step 72 the program determines if the user button has been held on for longer than 3 seconds and switches to the user mode if this is the case since it determines that the user has requested a change of operating mode, for example the device can be set to enter a test mode whereby the user can check that the device is correctly detecting a closed eye condition. If the user button is not switched on the program determines in Step 73 whether communications with the infra red communications serial port is switched on for downloading the data relating to the wearer's state of tiredness to a data recording device or other means as previously described. If the data is not to be transmitted in Step 73 the program repeats the procedure for the right eye in Step 74 and then returns to Step 53 after waiting for a period of 9 milliseconds in Step 75.

There now follows a description of a suitable circuit board with suitable circuitry for implementing the device of the present invention. The circuit board is described with reference to Figure 6. Ten digital output ports of a PIC micro controller (IC3) 80drive, via 100R resistors 81, infra red LEDs (D17 to D27) 82. These LEDs are mounted on the spectacles frame 5 distributed around each lens holder. Sixteen infra red photodiodes (Dl to D16) 83 are also mounted, in pairs, around the lens holders of the spectacles frame and monitor scattered infra red light. In each pair of photodiodes, one monitors the background light whilst the other monitors infra red light scattered from the eyelid. The output from each pair of photodiodes is amplified by one of the transimpedance amplifiers 1C1 & IC2, 84 and 85 respectively to provide input signals to the A/D ports 86 of the micro controller. The serial input/output of the micro controller is converted by the Infra red Data Association compliant Encoder Decoder (IrDA EnDec IC) (IC4) 87 which interfaces to the Infra red Data Assocation (IrDA) transceiver (IC5) 88. This transceiver is external to the PCB, and is mounted on the bridge of the spectacles frame. There are 4 other digital input/output ports on the micro controller connected to external devices, including a miniature sounder SPK1 89. This sounder provides an audible warning to the wearer of the spectacles that he is in danger of falling asleep. Alternatively, the sounder SPK1 can be a miniature vibrator, to provide non-audible warnings. A 10 MHz crystal 90 provides a clock signal for the micro controller and a 3.6864 MHz crystal 91 drives the IrDA EnDec 87. The PIC microcontroller's three program pins and its master reset, together with the +3v and GND lines, interface to a connector (CN1) 92 on the PCB to provide an interface for a set up computer system. This allows the spectacles to be set up for individual wearers.

Although the invention has been described with reference to the embodiments shown in the accompanying drawings it is to be understood that the invention is not limited to those precise embodiments and that various changes and modifications may be effected without the exercise of further inventive skill and effort. For example, the invention also envisages embodiments where the infra red source, or radiation emitter(s), is/are separate from the wearable device, for instance mounted on the dashboard of a vehicle or any other location where the emitted radiation can be detected when scattered from the wearer's eyelid.

Claims

1. A wearable device for monitoring eyelid movements of a user wearing the said device; comprising: a wearable frame; at least one detector for receiving scattered radiation emitted towards andreflected from the closed or partially closed eyelid of the wearer; and, alerting means for providing an alert signal in response to the said detector detecting radiation reflected from the eyelid.

2. A wearable device as claimed in Claim 1 further comprising at least one emitter fixed relative to the said frame for emitting radiation towards the wearer's eye.

3. A device as claimed in Claim 2 wherein the said detector is positioned with respect to the said emitter to receive scattered radiation reflected from the closed or partially closed eyelid only.

4. A device as claimed in Claim 3 wherein the said detector is positioned with respect to the said emitter so that radiation from the said emitter is reflected on a path from the surface of the wearer's eyeball divergent from the path between the emitter and the said eyeball surface, independently of the incident position of the emitted radiation on the said eyeball.

5. A device as claimed in any one of Claims 2 to 4 wherein the said detector is positioned adjacent to the said emitter so that the detector receives reflected radiation from the surface of the closed or partially closed eyelid along a path substantially coincident with the path of the emitted radiation.

6. A device as claimed in any one of Claims 2 to 5 wherein the said emitter comprises a Light Emitting Diode (LED).

7. A device as claimed in any preceding claim wherein the said radiation is Infra- red radiation.

8. A device as claimed in any one of Claims 1 to 6 wherein the said radiation is visible light.

9. A device as claimed in any preceding claim wherein the said alerting means is an audible and/or visible and or vibrating alarm.

10. A device as claimed in any preceding claim wherein the said radiation is pulsed radiation.

11. A device as claimed in any preceding claim further comprising processing means for processing output signals from the said detector over a pre-determined period of time to determine whether the detected radiation is representative of closure or blinking of the wearer's eyelid.

12. A device as claimed in any preceding claim further comprising signal transmission means for transmitting a signal representative of the wearer's eyelid position to a data recorder or vehicle engine control system.

13. A device as claimed in any preceding Claim 12 wherein the transmission link is a wireless link.

14. A device as claimed in any one of Claims 2 to 5 wherein the angular orientation of the said emitter and/or detector with respect to each another and/or with respect to the said frame is adjustable.

14. A device as claimed in any preceding claim wherein the frame comprises a spectacle frame.

15. A device as claimed in Claim 14 wherein either the said detector or the said emitter or both are mounted centrally in the region of the nosepiece of the spectacle frame.

16. A device substantially as hereinbefore described and/or with reference to the accompanying drawings.