JP3057996B2 - Eye position detection device for vehicle driver - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting an eyeball position of a vehicle driver.

[0002]

2. Description of the Related Art Conventionally, there has been known an eyeball position detecting device of a vehicle driver for use in a device for detecting dozing or looking away from a face image of the vehicle driver.
No. 0-166807 and JP-A-60-168005. In order to accurately detect the position and condition of the driver's eyes, these control the light amount of the light emitting means or control the light amount incident on the image pickup device. The aim is to always get clear images.

[0003]

However, in such a conventional apparatus for detecting the position of the eyeball of a vehicle driver by image processing, the purpose of the present invention is to correct the grayscale image information of the entire input image. Since the control of the light amount of the light emitting unit and the control of the light amount incident on the image pickup device are performed as described above, for example, when natural light having a light amount much larger than the light emitting unit enters a part of the input image, However, there is a problem in that a complicated image processing may not be performed.

In the detection of the position of the driver's eyeball, when the image information is binarized, the eyes, eyebrows, hair, etc. are converted to black pixels, forehead,
It is assumed that the cheek, nose, chin, and the like are processed so as to be converted into white pixels. There are various processing methods such as a differential histogram method and a discriminant analysis method for binarizing the image. However, in these methods, a threshold value for binarization is set by performing a mathematical calculation on the grayscale information of all the input pixels. If natural light with a large amount of light is mixed into the input image, the eyes will be converted to white pixels, or the shadows that can be cast on the face will be converted to black pixels, and the true black pixels of the eyes will be converted. Sometimes became indistinguishable.

[0005] Therefore, there is a problem that the position and state of the driver's eyeball cannot always be accurately grasped in an ever-changing illuminance environment such as in a vehicle cabin. The present invention
It has been made by focusing on such conventional problems, and is not affected by the light environment such as extraneous light, so that a vehicle driver who can accurately detect the position, state, and the like of the driver's eyeballs. It is an object of the present invention to provide an eyeball position detecting device.

[0006]

SUMMARY OF THE INVENTION For this reason, the present invention has been described with reference to FIG.
As shown in FIG. 1, an eyeball position detecting device for a vehicle driver including a face image input unit 1 for inputting an image including a driver's face, and an eyeball position detecting unit 4 for processing input image data to detect an eyeball position. A density information measuring means 2 for measuring the density information in a local area in the image frame where the influence of external light is low, and a binarization threshold setting for setting a binarization threshold based on the density information The eyeball position detecting means 4 is configured to perform image processing using the binarization threshold set by the binarization threshold setting means 3.

[0007]

The local area density information measuring means 2 measures the density information of an area having a small influence of the light environment such as external light as the local area. Then, the binarization threshold setting means 3 sets a binarization threshold for binarizing image data based on the density information. As a result, even if the light environment in the vehicle interior changes significantly due to external light, the eyeball position detecting means 4 can reliably identify the eyeball and other parts without being affected by the light environment. It is detected with high accuracy.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings. FIG. 2 shows the configuration of a first embodiment in which the present invention is applied to a driver's arousal level warning device. T for photographing the face of the driver 21
The V camera 11 is installed in front of the driver 21.
The input image data captured by the TV camera 11 is A / D
The data is converted into a digital quantity by the converter 13 and the
Is held. A light environment measurement circuit 15 for measuring a light environment state based on the state of the local region of the input image data is provided.
A binarization threshold setting circuit 16 for setting a binarization threshold is connected.

An eyeball position detecting circuit 17 for binarizing the image data and detecting the position of the eyeball using the threshold value set by the binarizing threshold value setting circuit 16 is provided. The open / closed eye detection circuit 18 for detecting the degree of eye open / closed degree is connected. An awakening degree determination circuit 19 determines an awakening degree based on an output signal of the open / closed eye detection circuit 18, and an alarm device 20 is provided which issues an alarm when receiving a signal indicating a reduced state of the awakening degree. In order to photograph the face of the driver 21 without dazzling the driver 21 at night or in a tunnel, an infrared lamp 12 is installed in front of the driver 21 to illuminate the face.

Next, the control operation will be described with reference to the flowchart of FIG. First, in step 101, the TV
The face of the driver 21 is photographed by the camera 11, and in step 102, the input image data for one frame is A / D
The image signal is converted into a digital signal by the converter
Is stored in Step 101 constitutes the face image input means of the invention.

In step 103, the light environment measuring circuit 15
In FIG. 4, an average density value is obtained by calculating an average density gradation as density information of a local area limited to an upper right portion of an image frame as shown as a local area A in FIG. As the local region A, an upper portion and a central portion of the passenger compartment that are hardly affected by extraneous light from a window are selected. In the present embodiment, the local region A is set to an upper half portion and a right half portion assuming a right-hand drive vehicle. The left-hand drive vehicle is located in the upper half and the left half is the local area. Therefore, if the average density value in this local area is obtained, the light environment state where the left eye of the driver 21 is placed can be objectively grasped, and the influence of daytime external light can be suppressed as much as possible. When the image data is binarized in the step, at least the left eye of the driver 21 can be easily extracted as a black pixel. Step 10
Reference numeral 3 constitutes the local area density information measuring means of the present invention.

In step 104, the binarization threshold setting circuit 16 receives the average density value in the local area A and sets it as a binarization threshold. Step 105
Then, as preprocessing of image data, binarization processing using the above-described binarization threshold value and noise removal processing are performed, and binary image data suitable for subsequent processing is prepared.

In step 106, the eyeball position detecting circuit 1
In step 7, the face width is set.
At 07, an eye existence area is set. In these two steps, first, the image is scanned in an area where the center of the face is considered to be located in the image frame, and the face width portion is extracted from the state of the number of continuous white pixels. Then, scanning is further performed within the face width to search for two black pixel regions corresponding to the eyeball portion and the eyebrows above the eyeball portion, thereby obtaining the eye existence region. Step 104 constitutes the binarization threshold value setting means of the invention, and steps 105 to 107 constitute the eyeball position detecting means.

Next, at step 108, the state of the degree of opening of the eyes is detected by the open / closed eye detection circuit 18, and at step 109, the eyes are determined based on the degree of opening of the eyes. Then, in step 110, the arousal level determination circuit 19 determines the arousal level of the driver based on the occurrence pattern of the open / closed eyes.

Here, when it is determined that the awakening degree is in a reduced state, the routine proceeds to step 111, where an alarm is issued by the alarm device 20. Thereafter, the process returns to step 101 to continue monitoring the eye condition. Also, when the determination in step 110 does not indicate that the arousal level has been reduced, the process returns to step 101 similarly. Details of the setting of the face width, the setting of the eye existence area, the detection of the degree of opening of the eyes, the judgment of the open / closed eyes, the judgment of the degree of arousal, and the like after step 106 are detailed in Japanese Patent Application No. 5-39562.

This embodiment is configured as described above, and the light environment measuring circuit determines a portion having little influence of external light as a local region, finds an average density value of the local region in the face image data, and calculates the average density value. The binarization threshold setting circuit sets the binarization threshold based on this, so that even in the case of a face image taken in a vehicle interior where external light enters from the front or side, the eyeball position and eye condition can be accurately detected. It has the effect that it can be done.

FIG. 5 shows a second embodiment in which local areas are switched in accordance with daytime and nighttime changes to set a binarization threshold value. Taking a right-hand drive car as an example,
In the daytime, the local region A in the upper half and the right half of the image data as shown in FIG. 4 is less affected by external light, and is suitable for extracting at least the left eye of the driver.
The light received by the driver is only the light from the infrared lamp illuminating the face. At this time, if the same local region A as in the daytime is used, the area of the portion of the local region that is exposed to light changes due to the movement of the driver's face. As a result,
Even if the light environment state due to illumination has not changed, the density information in the local area may change and be affected by the binarization threshold. This second embodiment addresses this.

In FIG. 5, a day / night determination circuit 22 is connected to a light environment measurement circuit 25 connected to the image memory 14. The day / night determination circuit 22 includes, for example, an illuminance sensor and determines whether it is daytime or nighttime based on whether or not the output value of the illuminance sensor is equal to or more than a predetermined value (= day). The light environment measurement circuit 25 switches the local area from which the density information indicating the light environment state should be obtained based on the determination result received from the day / night determination circuit 22. The density information of the switched local area, that is, the average density value is obtained and output to the binarization threshold value setting circuit 16. Other configurations are the same as those of the first embodiment.

The operation flow of the above configuration will be described with reference to the flowchart of FIG. 1 in step 102
When the digital input image for the frame is stored in the image memory 14, in step 201, the day / night determination circuit 2
According to 2, it is determined whether it is daytime or nighttime. If the determination result is daytime, the process proceeds to step 103, where the light environment measurement circuit 25 determines the upper half and the right half of the image frame based on the image data stored in the image memory 14 as shown in FIG. The calculation of the average density gradation of the local area A covering the portion is performed. When the result of the determination is nighttime, the routine proceeds to step 203, where the infrared lamp 12 is turned on to illuminate the driver's face.

Then, in step 204, the light environment measurement circuit 25 calculates the average density gradation of the local area B covering the lower half of the image frame as shown in FIG. The local region B is a region corresponding to the chin to the neck of the face, and is a region where the change in brightness is small even when the driver's face moves. In step 104, step 1
03 or the calculation result of step 204 is received from the light environment measurement circuit 25, and the binarized threshold value setting circuit 16
A binarization threshold is set based on the average density value of any of the local areas corresponding to daytime and nighttime. Other steps are the same as those in the flowchart of FIG.

According to this embodiment, the local area for acquiring the density information which is the basis of the binarization threshold value for image processing is switched between daytime and nighttime. Since the lower half is used, even when the driver's face moves under irradiation by the infrared lamp, it is possible to always obtain stable binary image data without being affected by the brightness.

In this embodiment, the binarization threshold value is set for each cycle of the flow even at night. However, since the light environment condition is stable at night, the threshold value is set after the night is determined. After the density information measurement of the local area B has been performed for a predetermined time, the face width and the eye existence area can be set accurately even if the binarization threshold is fixed. Also,
The day / night determination circuit 22 has an example in which an illuminance sensor is provided. In addition, for example, a light switch sensor of a vehicle light such as a headlight or a density information output circuit of image data is used, and a light switch is turned on (lighting = Night) or off, and the density information of the image data is greater than or equal to a predetermined value (= day)
Whether or not it is daytime or nighttime may be determined based on whether or not.

[0023]

As described above, according to the present invention, in an eyeball position detecting apparatus for a vehicle driver which detects an eyeball position by performing image processing on a face image, an area which is less affected by light environment such as external light is defined as a local area. Since the density information of the area is measured, and the binarization threshold value for binarizing the image data is set based on the density information, even if the light environment in the vehicle interior changes significantly due to external light, There is an effect that the eyeball part and other parts are reliably identified without being affected by the image processing, and the eyeball position of the driver is accurately detected. Also, by switching the local area between daytime and nighttime,
Even in the face image at night or in a tunnel, it is possible to accurately and stably detect the eyeball position and the state of the eyes.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a first exemplary embodiment of the present invention.

FIG. 3 is a flowchart illustrating a flow of an operation in the first embodiment.

FIG. 4 is a diagram illustrating a local region.

FIG. 5 is a block diagram showing a configuration of a second embodiment.

FIG. 6 is a flowchart showing a flow of an operation in the second embodiment.

FIG. 7 is a diagram illustrating a local region at night.

[Explanation of symbols]

 Reference Signs List 1 face image input means 2 local area density information measuring means 3 binarization threshold setting means 4 eyeball position detecting means 11 TV camera 12 infrared lamp 13 A / D converter 14 image memory 15, 25 light environment measuring circuit 16 2 Threshold value setting circuit 17 Eye position detection circuit 18 Open / closed eye detection circuit 19 Arousal level determination circuit 20 Alarm device 21 Driver 22 Day / Night determination circuit A, B Local area

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ identification symbol FI G06T 5/00 G06F 15/62 380 G08G 1/16 15/68 320Z (58) Investigated field (Int.Cl. ⁷ , DB name) ) G01B 11/00-11/30 G06T 1/00-7/00 G01N 21/17-21/61 G02B 27/00 G08G 1/16

Claims (4)

(57) [Claims] 1. An eyeball position detecting device for a vehicle driver, comprising: a face image input unit for inputting an image including a driver's face; and an eyeball position detecting unit for performing image processing on input image data to detect an eyeball position. A local area density information measuring unit for measuring density information in a local area where the influence of external light is low in the image frame; and a binarization threshold setting unit for setting a binarization threshold based on the density information. The eyeball position detecting means sets the two-dimensional value set by the binarization threshold setting means.
An eyeball position detecting apparatus for a vehicle driver, wherein the apparatus is configured to perform image processing using a binarization threshold. 2. A vehicle driver according to claim 1, wherein said local area density information measuring means sets the local area in an upper portion of an image frame and on a center side of a vehicle compartment. Eyeball position detecting device. 3. The local area density information measuring means includes an upper portion of the image frame during the daytime and a central portion of the vehicle interior as the local region, and a lower portion of the image frame at night as the local region. The eyeball position detecting device for a vehicle driver according to claim 1, wherein: 4. The binarization threshold value setting means for setting an average value of density gradations of the local area as the binarization threshold value. An eyeball position detecting device for a vehicle driver according to claim 1.