JPH0944676A - Face detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To exactly detect the feature point of face image by performing correlative arithmetic by scanning the face image while using a template corresponding to the feature point of face, deciding the candidate points of feature point when they exceeds a prescribed threshold value, and providing the candidate point having position relation similar to the position relation of feature point out of the combination of those candidate points. SOLUTION: The image of the face of a reagent is picked up, and the face image is provided by an image pickup means M1. Next, the correlative arithmetic is performed by a correlative arithmetic means M2 while using plural templates prepared in advance corresponding to the respective feature points of face. Next, a position relation comparing means M3 detects the feature point of face image by finding the combination of candidate points having the position relation similar to the position relation of plural feature points on face prepared in advance out of the combination of candidate points of feature points. Thus, since the plural candidate points are permitted for the plural feature points, the feature points are not missed and because of the combination of candidate points having the position relation similar to the position relation of respective feature points, erroneous detection can be prevented.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face detecting apparatus, and more particularly to a face detecting apparatus for detecting feature points of a face from a face image of a subject obtained by imaging.

[0002]

2. Description of the Related Art Conventionally, in order to detect the driving state of a vehicle driver, it has been proposed to image the driver's face and monitor the direction of the face or the direction of the line of sight. For example, in Japanese Patent Laid-Open No. 6-243367, a driver's face is imaged, and four points at the inner and outer ends of both eyes are extracted as facial feature points from the face image, and the face is symmetrical with respect to the center line. Based on this, the face direction is calculated by determining the direction of the isosceles trapezoid formed by the extracted four points.

[0003]

When extracting feature points of a face image by the conventional technique, a template of face feature points, that is, an eye template and a mouth template is used, and the template is sequentially scanned on the face image. It is conceivable to perform a correlation calculation between the image and the template and extract the eye and mouth feature points when the degree of correlation exceeds a predetermined threshold value.

However, when the driver changes, the face orientation changes, the distance from the camera to the driver changes, or the outside light changes, the degree of correlation between the face image and the template changes. Therefore, if the threshold value of the degree of correlation is increased, the feature points cannot be extracted. If the threshold value of the degree of correlation is lowered in order to eliminate this, there is a problem that eyebrows, noses, etc. in a face image may be mistakenly extracted as eyes.

The present invention has been made in view of the above points,
Among the combinations of candidate points of each feature point obtained by the correlation calculation, by finding the combination of candidate points having a similar positional relationship to the positional relationship of the facial feature points, the facial feature points are not overlooked and an error occurs. It is an object of the present invention to provide a face detection device that can detect a face with high accuracy.

[0006]

According to a first aspect of the present invention, as shown in FIG. 1, an image pickup means M1 for picking up a face of a subject to obtain a face image, and a plurality of feature points of a face prepared in advance. Correlation calculation means M2 that scans the face image using a plurality of templates respectively corresponding to the above to perform a correlation calculation, and makes a candidate point of each feature point when the degree of correlation exceeds a predetermined threshold,
Among the combinations of the candidate points of each of the feature points, the feature points in the face image are found by finding a combination of candidate points having a similar positional relationship to the positional relationship of the plurality of feature points of the face prepared in advance. It has a positional relationship comparing means M3 for detecting.

Therefore, a plurality of candidate points are allowed for each of the plurality of feature points, so that the feature points are not overlooked, and the face image can be detected without error from the combination of the candidate points having the similar positional relationship to the positional relationship between the characteristic points. It is possible to detect the feature points of. Claim 2
The invention according to claim 1 is the face detecting apparatus according to claim 1, wherein the correlation calculating means changes the threshold value of the degree of correlation for each of a plurality of feature points.

For this reason, among several feature points of the face,
An optimum threshold value can be set for each feature point according to the size of the image contrast and the individual difference, and the feature point extraction accuracy can be improved.

[0009]

2 shows a block diagram of the present invention. In the figure, a camera 10 as an image pickup means M1 for picking up an image of a driver's face is attached to a predetermined position of a vehicle, and an image signal of the driver's face picked up by the camera 10 is digitally converted by a D / A converter 11. The image is converted and stored in the image memory 12. In addition, in the ROM 13, as shown in FIGS.
The eye, nose, and mouth templates shown in (C) and the positional relationship between the eyes, nose, and mouth shown in FIG.

The image processing circuit 14 has a normalization section and a correlation calculation section. After performing the grayscale normalization processing of the face image stored in the image memory 12, the image processing circuit 14 uses the template loaded from the ROM 13 to perform the above-mentioned processing. The face image is scanned and the correlation operation is performed to extract candidate points for each of the eyes, nose, and mouth. From the candidate points of each part, those satisfying the positional relationship of the eyes, nose, and mouth are extracted, Determine the positions of the nose and mouth.

4 and 5 are flow charts of the eye detection processing executed by the image processing circuit. This processing is executed when the ignition is turned on and when eye tracking fails. In the figure, in step S10, the left eye template of 32 × 16 pixels, the nose template of 32 × 16 pixels, and the mouth template of 64 × 16 pixels shown in FIGS. 3A, 3B, and 3C are read from the ROM 13 in step S10. Load each one. Next, in step S12, the ROM 13 is read as shown in FIG.
The reference coordinates (Xn, Yn) and (Xm, Ym) of the nose and mouth, respectively, whose origin is (0, 0), are loaded.

Next, in step S14, as shown in FIG.
A face image of 12 × 512 pixels is input, and the image shown in FIG.
(B), (C) The eye template, the nose template, and the mouth template shown in each of FIG. 8 scan the face image to perform the correlation calculation, and the correlation distribution of the eyes shown in FIG. 7, the correlation distribution of the nose shown in FIG. The respective mouth correlation distributions shown in 9 are calculated. Here, the correlation distribution γ (x, at the coordinates (x, y) on the face image
y) is calculated by the following formula.

[0013]

[Equation 1]

However, I (x, y) is a density value of coordinates (x, y) on the face image, and 0 <x <ωx, 0 <y <ω
Each of y, ωx, and ωy is the number of pixels in the horizontal and vertical directions of the template. Further, T (x ', y') is the density value of the coordinates (x ', y') on the template block. In each of FIG. 7, FIG. 8 and FIG. 9, a portion having a small correlation distribution value (high correlation degree) is displayed dark and a portion having a large correlation distribution value (low correlation degree) is displayed brightly. That is, the darker the correlation, the higher the correlation.

Next, in step S18, FIG. 7, FIG. 8 and FIG.
Candidate points are extracted from the correlation distribution of each eye, nose, and mouth, and numbering is performed for each eye, nose, and mouth. Here, the candidate points are points that satisfy all of the following expressions (2) to (6). γ (x, y) <THR (2) γ (x-1, y) -γ (x, y)> THR '(3) γ (x + 1, y) -γ (x, y)>THR' … (4) γ (x, y−1) −γ (x, y)> THR ′… (5) γ (x, y + 1) −γ (x, y)> THR ′… (6) where THR is A correlation distribution threshold value, THR ′, is a correlation distribution difference threshold value. That is, the correlation distribution value is less than the threshold value THR, and the difference between the correlation distribution values for adjacent pixels is the threshold value TH.
Points exceeding R ', that is, points that are markedly dark in FIGS. 7, 8, and 9 are candidate points for the eyes, nose, and mouth.

The threshold value of the correlation distribution for eyes and nose is set to 1
When it is set to 0000, the threshold value of the difference is set to about 200. Further, since the image contrast of the mouth is lower than that of the eyes and the nose, and the individual difference is large, the threshold value of the correlation distribution for the mouth is set to 5000 and the threshold value of the difference is set to 100. Further, since the nose does not open and close like the eyes and the mouth, the threshold value of the correlation distribution for the nose may be set higher than the threshold value of the eyes.

As a result, eye candidate points e1 to e10 indicated by arrows in FIG. 7 and nose candidate points n1 to n1 indicated by arrows in FIG.
0, mouth candidate points m1 to m9 indicated by arrows in FIG. 9 are extracted. The above steps S16 and S18 are the correlation calculation means M2.
Corresponding to. In this way, it is possible to set the optimum threshold value for each feature point according to the size of the image contrast and the individual difference between the eyes, nose, and mouth, which are some features of the face, and the extraction accuracy of the feature points can be improved. Can be improved.

Thereafter, in step S20, the number of eye discovery candidates n
s is reset to 0, the eye candidate counter ie is reset to 0 in step S22, the nose candidate counter in is reset to 0 in step S24, and the mouth candidate counter im is reset to 0 in step S26.

In the next step S28, one or more eye candidate points, one or more nose candidate points, or one or more eye, nose, or mouth candidate points are selected for all candidate points. A combination is made, and the eye candidate points for each combination are shown in FIG.
(D) The error En of the nose candidate point with respect to the reference coordinate (Xn, Yn) of the nose and the error E of the candidate point of the mouth with respect to the reference coordinate (Xm, Ym) of the nose when the positions of the eyes are overlapped.
m is calculated by the equations (7) to (9). However, since the distances between the eye candidate points and the nose and mouth candidate points differ depending on the distance from the camera to the face, the distance in the X direction between the eye candidate points and the nose candidate points is shown in FIG. It is normalized so as to match the distance between the eyes and the nose in the X direction, and the coefficient g is used for this purpose.

G = Xn / [xn (in) -xe (ie)] (7) En = [{Xn-g. (Xn (in) -xe (ie))} ² + {Yn-g. ( yn (in) -ye (ie)} ² ] ^1/2 (8) Em = [{Xm-g · (xm (im) -xe (ie))} ² + {Ym-g · (ym (im (im ) -Ye (ie)} ² ] ^1/2 (9) where xe (ie), yn (ie) are x, y coordinates on the face image of the candidate point of the ith eye, xn (in), yn (i
n) is the x, y coordinates of the candidate point of the inth nose, xm (i
m) and ym (im) are the x and y coordinates of the im-th mouth candidate point.

Next, in step S30, it is determined whether the error En is less than a predetermined threshold value THE and the error Em is less than the threshold value THE. If both En and Em are less than the threshold value THE, it is considered as a strong eye candidate point, and step S32 in FIG.
X and y coordinates of eye candidate point ei and error En + Em
Is stored, the number of eye discovery candidates ns is incremented by 1 in step S34, and the process proceeds to step S36.

In step S30, En ≧ THE or E
When m ≧ THE, the combination of the candidate points is shown in FIG.
The process proceeds to step S36 in order to change the candidate point because it does not match the positional relationship of (D). In step S36, the mouth candidate counter im is incremented by 1, and in step S3
In step 8, it is determined whether or not the value of im is less than the total number Nm of mouth candidate points. If im <Nm, the process proceeds to step S28 to continue the process. If im ≧ Nm, the nose candidate counter in is incremented by 1 in step S40, and it is determined in step S42 whether or not the value of in is less than the total number Nn of nose candidate points. If in <Nn, the process proceeds to step S26. Then, the mouth candidate counter im is reset to zero.

If in ≧ Nn, step S44
Then, the eye candidate counter ie is incremented by 1, and in step S46 it is determined whether or not the value of ie is less than the total number Ne of eye candidate points. If less than ie <Ne, the process proceeds to step S24 and the nose candidate counter in is set to 0. Reset, and if ie ≧ Ne, the process proceeds to step S48. This causes step S
At step 48, steps S28 and S30 are executed for all combinations of the eye, nose, and mouth candidates.

In step S48, the number ns of eye discovery candidates is 0.
It is determined whether or not. If ns = 0, there is no eye, nose, or mouth candidate point that satisfies the positional relationship of FIG. 3D, that is, there is no strong eye candidate point, and therefore it is determined that the eye cannot be detected in step S14. Then, the face image is input again and the above processing is repeated. If ns> 1, the process proceeds to step S50, the eye candidate point having the smallest error En + Em among the promising candidates represented by the number ns of eye discovery candidates is recognized as an eye, and in step S52, the coordinates of this eye candidate point are recognized. Is passed to the next tracking process, eyes are detected, and the process is terminated. The above steps S20 to S50 correspond to the positional relationship comparing means M3.

Since a plurality of candidate points are allowed for each of the eyes, nose, and mouth in this way, feature points are not overlooked, and a combination of candidate points having a positional relationship similar to the positional relationship of each characteristic point is found. Eyes, nose, and mouth, which are the feature points of the face image, can be detected without error.

[0026]

As described above, according to the invention as set forth in claim 1, the image pickup means for picking up the face of the subject to obtain the face image, and the plurality of templates prepared in advance respectively corresponding to the plurality of feature points of the face. Of the combination of the correlation calculation means for scanning the face image using the above and performing the correlation calculation, and setting the correlation point as the candidate point of each feature point when the degree of correlation exceeds a predetermined threshold, , Because it has a positional relationship comparison means for detecting a characteristic point in the face image by finding a combination of candidate points having a similar positional relationship to the positional relationship of the plurality of characteristic points of the face prepared in advance, Since multiple candidate points are allowed for each of the multiple feature points, feature points are not overlooked, and feature points of the face image are detected without error from the combination of candidate points with a positional relationship similar to the positional relationship of each feature point. You can do it.

[0027] The invention described in claim 2 is the same as that in claim 1.
In the face detection device described above, the correlation calculating means changes the threshold value of the degree of correlation for each of a plurality of feature points, and therefore, among several feature points of the face, depending on the magnitude of the image contrast or the magnitude of the individual difference. Therefore, an optimum threshold value can be set for each feature point, the extraction accuracy of feature points can be improved, and it is extremely useful in practice.

[Brief description of drawings]

FIG. 1 is a principle diagram of the present invention.

FIG. 2 is a block diagram of the present invention.

FIG. 3 is a diagram showing a positional relationship between a template and a nose and mouth.

FIG. 4 is a flowchart of eye detection processing.

FIG. 5 is a flowchart of eye detection processing.

FIG. 6 is a diagram showing a face image.

FIG. 7 is a diagram showing an eye correlation distribution.

FIG. 8 is a diagram showing a correlation distribution of the nose.

FIG. 9 is a diagram showing a mouth correlation distribution.

[Explanation of symbols]

 10 camera 11 D / A converter 12 image memory 13 ROM 14 image processing circuit M1 image pickup means M2 correlation calculation means M3 positional relationship comparison means

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 5, 1995

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Brief description of drawings

[Correction method] Change

[Correction contents]

[Brief description of drawings]

FIG. 1 is a principle diagram of the present invention.

FIG. 2 is a block diagram of the present invention.

FIG. 3 is a diagram showing a positional relationship between a template and a nose and mouth.

FIG. 4 is a flowchart of eye detection processing.

FIG. 5 is a flowchart of eye detection processing.

FIG. 6 is a photograph showing a face image when displayed on a display .

FIG. 7 is a photograph showing the correlation distribution of eyes when displayed on a display .

FIG. 8 is a photograph showing the correlation distribution of the nose when it is displayed on the display .

FIG. 9 is a photograph showing the correlation distribution of the mouth when displayed on the display .

[Explanation of reference numerals] 10 camera 11 D / A converter 12 image memory 13 ROM 14 image processing circuit M1 image pickup means M2 correlation calculation means M3 positional relationship comparison means

Claims (2)

[Claims] 1. An image pickup means for picking up a face of a subject to obtain a face image, and a correlation calculation by scanning the face image using a plurality of templates respectively corresponding to a plurality of feature points of a face prepared in advance. And a plurality of features of the face prepared in advance among the combinations of the correlation calculation means that determines the candidate points of the respective feature points when the degree of correlation exceeds a predetermined threshold value, and the candidate points of the respective feature points. And a positional relationship comparing means for detecting a characteristic point in the face image by finding a combination of candidate points having a positional relationship similar to that of the point. 2. The face detecting apparatus according to claim 1, wherein the correlation calculating means makes the threshold value of the degree of correlation different for each of a plurality of feature points.