JP4812743B2 - Face recognition device, face recognition method, face recognition program, and recording medium recording the program - Google Patents

Description

  The present invention relates to a face recognition method in a case where a face area in an input image is detected and the shooting direction is known to be constant in advance, and a face recognition device, a face recognition method, a face recognition program, and a program thereof are provided. The present invention relates to a recorded recording medium.

  Conventionally, there are the following methods for identifying a person of a face from an input image obtained by photographing the face.

  Method: A process of estimating the face direction from a face pattern detected from an image is provided, and parameters are corrected based on the estimation result so that matching with the front face is easy (see Non-Patent Document 1).

In addition, there exists a thing of a nonpatent literature 2-6 as a technique relevant to this invention.
Yoshihisa Ijiri, Yukiko Yanagawa, Sakai ≡, Masato Kawade, “Face Recognition Using Face Orientation Estimation Results”, 10th Symposium on Image Recognition and Understanding (MIRU2007), pp. 1247-12522,2007. Image Processing Standard Textbook, Association for Promotion of Image Information Education, pp. 159-162, 1997. Image Processing Standard Textbook, Association for Promotion of Image Information Education, pp. 178-179, 1997. Image Processing Standard Textbook, Association for Promotion of Image Information Education, pp. 253-254, 1997. Computer Vision, New Technology Communications, pp. 212-214, 1999. IEICE Transactions D Vol. J89-D No. 8pp. 1840-1847 "Pose Estimation Method for 3D Objects Using Support Vector Regression" Shingo Ando, Yoshinori Kusachi, Akira Suzuki, Kenichi Arakawa.

  The above method is a processing method of face area detection, face direction estimation, parameter correction from the estimation result, and recognition. The direction is known in advance before recognition, but the direction information is a parameter for the front direction. It is only used for correction. However, when the face is the target, the feature amount corresponding to the face direction is selected such that the image facing the front has many edges in the y direction and the image facing the diagonal direction includes many edges in the diagonal direction. The selection can be recognized more effectively.

  SUMMARY OF THE INVENTION The present invention solves the above-described problems, and its purpose is to record a face recognition device, a face recognition method, a face recognition program, and a program capable of accurately recognizing even if there is a change in face orientation. It is to provide a recording medium.

  In the present invention, when information on the position of the face region and the shooting direction of the face is known in advance, feature amount extraction according to the face direction (shooting direction) is performed thereafter, and similarity calculation using the feature amount is performed. And performing recognition based on the calculated similarity. By using features according to the face direction (shooting direction), it is possible to accurately recognize even if the face direction varies.

That is, the face recognition device according to claim 1 includes a dictionary face image input unit that inputs a face image for a dictionary in a certain shooting direction together with shooting direction information, and a shooting direction input by the dictionary face image input unit. Based on the information, the variance covariance using the x-direction and y-direction differential values of each pixel for a plurality of dictionary face images in the target shooting direction and using them as explanatory variables The first principal component of the matrix is calculated, and using the differential value and the first principal component, (differential value in x direction) × (x component of first principal component vector) + (differential value in y direction) × (first A face direction feature amount extraction unit that calculates a feature amount of y component of one principal component vector and extracts it as a feature amount based on a shooting direction, an image input unit that inputs an image to be recognized, and the input From the captured image based on the shooting direction Face area detection means for detecting a face area, feature quantity extraction means for extracting a feature quantity from the face area detected based on the shooting direction, and face direction feature quantity extraction means extracted from the face image for dictionary. A similarity degree between the feature quantity extracted from the face area in the input image by the feature quantity extraction means, and a similarity calculation means for recognizing a face based on the similarity, and the similarity And an output means for outputting a recognition result based thereon.

The face recognition method according to claim 3 is a face recognition method in an environment where the shooting direction of a face is constant, and the dictionary face image input means inputs a face image for a dictionary in a certain shooting direction together with shooting direction information. A dictionary face image input step and a face direction feature quantity extraction unit for a plurality of dictionary face images in the target shooting direction based on the shooting direction information input in the dictionary face image input step. Then, the differential value in the x direction and the y direction in each pixel is calculated, the first principal component of the variance-covariance matrix using these as explanatory variables is calculated, and the differential value and the first principal component are used ( A feature value of (differential value in x direction) × (x component of the first principal component vector) + (differential value in y direction) × (y component of the first principal component vector) is calculated and is based on the shooting direction. Face direction features to be extracted as feature quantities An amount extraction step; an image input step in which the image input means inputs an image to be recognized; and a face area detection step in which the face area detection means detects a face area based on the shooting direction from the input image; The feature amount extraction means extracts a feature amount from the face area detected based on the shooting direction, and the similarity calculation means uses the face direction feature amount extraction step to extract the feature amount from the dictionary face image. A similarity calculating step of calculating a similarity between the extracted feature quantity and the feature quantity extracted from the face region in the input image by the feature quantity extraction step, and recognizing a face based on the similarity; and output means Is provided with an output step for outputting a recognition result based on the similarity.

According to the above configuration, since the feature amount corresponding to the face direction (shooting direction) can be extracted and face recognition can be performed based on the feature amount, the recognition accuracy is improved.
Further, it is possible to select a feature amount suitable for the face direction (shooting direction).

The face recognition device according to claim 2 , wherein the face area detecting unit scans a template created from a face image photographed from a predetermined direction on the input image, and the similarity between the input image and the template is detected. A degree is calculated, and an area having the similarity equal to or higher than a threshold is detected as a face area.

Further, in the face recognition method according to claim 4 , in the face area detection step, a template created from a face image photographed from a predetermined direction is scanned on the input image, and the similarity between the input image and the template is determined. A degree is calculated, and an area having the similarity equal to or higher than a threshold is detected as a face area.

  According to the above configuration, it is possible to select a feature amount suitable for the face direction (shooting direction).

A face recognition program according to claim 5 is a face recognition program for causing a computer to function as each means according to claim 1 .

A recording medium according to claim 6 is a computer-readable recording medium on which a face recognition program for causing a computer to function as each means according to claim 1 is recorded.

(1) According to the inventions described in claims 1 to 6 , the face recognition accuracy is improved by using a feature amount suitable for the face direction (shooting direction). For example, in the case of the front face, the shadow often becomes vertically long beside the nose, but by using the feature amount of the y-direction derivative specialized for the front face, it is not affected by such frontal fluctuation. It is possible to reduce recognition errors such as recognition.
(2) It is also possible to select a feature quantity suitable for the face direction (shooting direction).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. FIG. 1 is a block diagram of an embodiment of the face recognition apparatus of the present invention.

  The present embodiment includes a dictionary feature quantity extraction device 101, an input image feature quantity extraction device 104, and a recognition device 108, and the feature quantity calculated by the input image feature quantity extraction device 104 and the dictionary feature quantity extraction device 101. The recognition device 108 is configured to collate the calculated feature values of the dictionary images of the registered persons. The technique of the present invention is characterized in that the feature quantity to be calculated is determined based on the face shooting direction information.

  In FIG. 1, reference numeral 102 in the dictionary feature quantity extraction apparatus 101 is a dictionary face image input apparatus as the dictionary face image input means of the present invention, and 103 is a face direction feature quantity as the face direction feature quantity extraction means of the present invention. It is an extraction device.

  Reference numeral 105 in the input image feature quantity extraction device 104 is an image input device as image input means of the present invention, 106 is a face area detection device as face area detection means of the present invention, and 107 is a feature of the present invention. It is a feature quantity extraction device as quantity extraction means.

  109 in the recognition device 108 is a similarity calculation device as the similarity calculation means of the present invention, and 110 is an output device as the output means of the present invention.

  The dictionary face image input device 102, face direction feature amount extraction device 103, image input device 105, face area detection device 106, feature amount extraction device 107, similarity calculation device 109, and output device 110 are configured by, for example, a computer. The

  Here, a face image acquisition scene when the shooting direction information is constant will be described with reference to FIG. In FIG. 2, in the case where a camera 201 for photographing is installed on the upper part of a home door or an office entrance 202, in most cases, a face image photographed from above is input. A recognition method in the case where the orientation of a face input in advance is known will be described.

  First, the dictionary feature quantity extraction apparatus 101 of FIG. 1 will be described.

  The dictionary face image input device 102 sets the direction of the face to be input according to the usage scene, and acquires the face image taken by the camera or the like from that direction as the dictionary face image together with the shooting direction information.

  The face direction feature quantity extraction device 103 extracts the feature quantity of the acquired dictionary face image. That is, a logarithmic filter is applied to the dictionary face image, smoothing processing is performed, and differential features based on the shooting direction information are extracted.

  FIG. 3 shows an example of dictionary face image and feature extraction. In FIG. 3, reference numeral 301 denotes an example of an image obtained by photographing a registered person from the front. It is assumed that the camera 201 in FIG. 2 is installed in the middle of the entrance 202.

  First, the logarithmic filter will be described. A logarithmic table represented by the following formula (1) is created.

  In Expression (1), I represents a pixel value from 0 to 255. A value corresponding to each pixel value in the input image is determined with reference to the created logarithmic table.

  Next, the smoothing filter will be described. Smoothing is performed by applying a 3 * 3 pixel smoothing filter or Gaussian filter as described in Non-Patent Document 2. FIG. 3 shows an example of an image 302 obtained by performing a smoothing process by applying a logarithmic filter to the image 301. It can be seen that by applying a logarithmic filter, it is strong against illumination fluctuation.

  Next, a method for extracting a differential feature in a direction corresponding to the shooting direction information will be described.

  First, a method for calculating in which direction the differential features should be extracted according to the shooting direction will be described. For each pixel, the differential values in the x and y directions are calculated by the Prewitt edge detection operator or the Sobel edge detection operator described in Non-Patent Document 3.

  Reference numerals 303 and 304 in FIG. 3 show examples of images representing differential values of the image 302 in the x and y directions. The differential covariance values in the x direction and y direction are used as explanatory variables, and the variance covariance matrix is obtained by observing the number of data (number of dictionary face images having the same shooting direction information) × (number of pixels per sheet), The first principal component is calculated. In this case, since the differential values in the x direction and the y direction are used as explanatory variables, the variance-covariance matrix is 2 × 2.

  As the observation data, the entire dictionary face images of all persons registered as described above may be used, or characteristic dictionary face images and regions may be selected. The calculated differential value in the vector direction of the first principal component is used as a feature amount corresponding to the shooting direction information. That is, a directional derivative having a large variance is selected.

  For example, in a usage scene in which front face images are often input, a variance covariance matrix is calculated from differential values in the x and y directions of a plurality of registered front face images, and the first principal component is calculated. calculate. When the vector of the first principal component is calculated as (0, 1), the differential value in the y direction is used as the feature value of the front face. That is, the feature indicated by 304 in FIG. 3 is a feature amount suitable for the front face image. Further, when the first principal component is calculated as (1 / √2, 1 / √2) with respect to the face image taken from the upper right direction, x-direction differential value × 1 / √2 + y-direction differential value × 1. / √2 may be used as a feature amount. Although the method based on principal component analysis has been described, the edge direction in each pixel may be calculated, a histogram thereof may be created, and the largest number of angles may be selected.

  Next, each device constituting the input image feature quantity extraction device 104 will be described. The image input device 105 inputs an image shot with a digital video or the like in the usage scene described with reference to FIG.

  Next, the face area detection device 106 will be described.

  Any method may be used to detect the face area. Here, a case will be described in which appearance-based matching is performed to detect the face area.

  First, an average face image is created using face images of a plurality of persons photographed from a direction that seems to be input most according to the usage scene, and is set as a template 401 shown in FIG. 4, for example. Then, the template 401 is scanned on the input image 402 shown in FIG. 4, the similarity is calculated, and an area where the similarity is greater than or equal to a threshold is detected as the face area 403.

  The template shown in 401 of FIG. 4 is an example. Depending on the usage scene, templates taken from various directions such as those taken from the upper direction, those taken from the lower direction, those taken from the upper right are prepared. do it. Although the face image used as a template is an average face image of a plurality of faces, a representative face may be selected and used as a template, and an image suitable for a target to be detected is used. In addition, the pixel value as it is may be used as a template, or a characteristic extracted for a target such as a differential filter may be used as a template as described in Non-Patent Document 3. At this time, the same calculation is performed on the input image to extract similar features, and then the template is scanned to calculate the similarity.

  As described in Non-Patent Document 4, the similarity calculation method includes a difference, a normalized correlation value, and the like. Any of these may be used, but the selection is made according to the characteristics of the input image. When extracting a region whose similarity is greater than or equal to the threshold, when performing template matching while arbitrarily changing the size of the input image, the same part may be extracted multiple times. Although a plurality of regions may be extracted, clustering may be performed based on the center coordinates of the regions and the extracted regions may be combined. Since only the face area in the shooting direction used as a template is detected, the shooting direction of the input face area is known in advance.

  Next, the feature amount extraction device 107 performs logarithmic filter processing and smoothing processing on the face area in the input image calculated by the face region detection device 106 in the same manner as the feature amount extraction in the face direction feature amount extraction device 103. Direction differentiation based on the shooting direction is calculated as a feature amount.

  Next, each device constituting the recognition device 108 will be described.

  The similarity calculation device 109 divides the face area and the dictionary face image in the input image into rectangular regions, and extracts the feature amount extracted by the face direction feature amount extraction device 103 of the corresponding rectangular area and the feature amount extraction device 107. Similarities with the calculated feature amounts are calculated.

  For example, when the feature quantity corresponding to the face orientation detected by the face area detection device 106 is 501 in FIG. 5, that is, a front face image is detected, and the feature quantity corresponding to the front is a differential value in the y direction. In this case, each rectangular area is divided as indicated by 502 in FIG. 5, and the similarity is calculated for each rectangular area. As described in Non-Patent Document 4, there are a difference, a normalized correlation value, and the like, and any of them may be used, but here, a correlation value C of the following equation (2) is calculated.

  In Expression (2), the feature value of the rectangular area of the dictionary face image is jisho (i, j), and the feature value of the rectangular area in the face area in the input image is input (i, j). Although an example of dividing into rectangular areas has been described, any shape may be used.

  The person of the dictionary face image showing the largest sum of correlation values C in each rectangular area is taken as the identification result. As described above, the sum of the correlation values C may be simply taken, or the correlation values C of the rectangular areas may be sorted, and the median value may be compared as the similarity with the dictionary face image. When the similarity is equal to or less than the threshold value, it is determined that the person is not a registered person, and the identification result may be “not applicable”.

  When calculating the similarity for each rectangular area, average movement or affine transformation is performed for each rectangular area in order to deal with the positional shift of the facial part due to changes in facial expression and slight variations in the orientation, etc. You may deform | transform so that it may become high.

  The output device 110 outputs the identification result from the similarity calculation device 109.

  The face recognition method of the present invention is a face recognition method in an environment in which the shooting direction of a face is constant, a dictionary face image input step of inputting a face image for a dictionary in a certain shooting direction together with shooting direction information, and the dictionary face A face direction feature amount extraction step for extracting a feature amount from a dictionary face image based on the shooting direction information input in the image input step, an image input step for inputting an image to be recognized, and the input image A face area detecting step for detecting a face area based on a shooting direction from the camera, a feature amount extracting step for extracting a feature quantity from the face area detected based on the shooting direction, and a face direction feature quantity extracting step. The similarity between the feature amount extracted from the face image and the feature amount extracted from the face region in the input image by the feature amount extraction step is calculated, and the similarity Recognizing similarity calculation step face based, and an output step of outputting a recognition result based on the similarity.

  Specifically, for example, the processing of each device in FIG. 1 is executed in the following procedure.

That is,
(Step S1)
The dictionary face image input device 102 sets the direction of the face to be input according to the usage scene, and acquires a face image photographed with a camera or the like from that direction as a dictionary face image together with photographing direction information.
(Step S2)
The face direction feature quantity extraction device 103 applies a logarithmic filter to the acquired dictionary face image, performs a smoothing process, and further extracts differential features based on the shooting direction information. At that time, with respect to a plurality of dictionary face images in the target shooting direction, the differential values of the x direction and the y direction at each pixel are calculated, and the variance-covariance matrix of which these are used as explanatory variables An edge feature based on the direction of the first principal component is calculated and used as a feature amount based on the shooting direction.
(Step S3)
The image input device 105 inputs an image taken with a digital video or the like in the usage scene described with reference to FIG.
(Step S4)
The face area detection device 106 detects a face area based on the shooting direction from the input image. For example, a template created from a face image taken from a predetermined direction is scanned on the input image to calculate the similarity between the input image and the template, and an area where the similarity is equal to or greater than a threshold is set as a face area. To detect.
(Step S5)
The feature amount extraction device 107 performs logarithmic filter processing on the face area in the input image detected in step S4, as in the feature amount extraction in step S2 (processing performed by the face direction feature amount extraction device 103). Then, smoothing processing is performed, and a directional differential based on the shooting direction is calculated as a feature amount.
(Step S6)
The similarity calculation device 109 divides the face area and the dictionary face image in the input image into rectangular regions, and the feature amount and feature amount extracted by the face direction feature amount extraction device 103 (step S2) of the corresponding rectangular region. Similarities with the feature quantities extracted by the device 107 (step S5) are calculated, and faces are identified based on the similarities.
(Step S7)
The output device 110 outputs the identification result obtained by the similarity calculation device 109 (step S6).

  By performing the feature amount extraction based on the shooting direction according to the above embodiment example, the matching can be performed using the feature amount suitable for each use scene, and the recognition can be performed with high accuracy.

  Further, a part or all of the functions of each means in the face recognition apparatus of the present embodiment can be configured by a computer program, and the program can be executed using the computer to realize the present invention. It goes without saying that the procedure in the face recognition method can be constituted by a computer program, and the program can be executed by the computer.

  It is also possible to supply a recording medium recording a software program for realizing the functions of the above-described embodiments to a system or apparatus, and read and execute the program stored in the recording medium by the CPU (MPU) of the system or apparatus. ,realizable. In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the recording medium on which the program is recorded, for example, a CD-ROM, a DVD-ROM, a CD-R, a CD-RW. , MO and HDD constitute the present invention.

The block diagram which shows the embodiment of the face recognition apparatus of this invention. Explanatory drawing of the imaging | photography direction in the example of embodiment of this invention. Explanatory drawing of the feature-value extraction in the embodiment of this invention. Explanatory drawing of the method of the face area detection means in the embodiment of this invention. Explanatory drawing of the image division | segmentation in the similarity calculation means in the embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Dictionary feature amount extraction apparatus, 102 ... Dictionary face image input device, 103 ... Face direction feature amount extraction device, 104 ... Input image feature amount extraction device, 105 ... Image input device, 106 ... Face area detection device, 107 ... Feature Quantity extraction device, 108 ... recognition device, 109 ... similarity calculation device, 110 ... output device.

Claims (6)

In a face recognition device in an environment where the shooting direction of the face is constant,
A dictionary face image input means for inputting a face image for a dictionary in a certain shooting direction together with shooting direction information;
Based on the photographing direction information input by the dictionary face image input means , the differential values in the x direction and y direction at each pixel are obtained for a plurality of dictionary face images in the target photographing direction. And calculating a first principal component of a variance-covariance matrix using them as explanatory variables, and using the differential value and the first principal component (differential value in the x direction) × (x component of the first principal component vector) ) + (Differential value in the y direction) × (y component of the first principal component vector) is calculated, and a facial direction feature amount extraction means for extracting the feature amount as a feature amount based on the shooting direction ;
An image input means for inputting an image to be recognized;
Face area detecting means for detecting a face area based on a shooting direction from the input image;
Feature amount extraction means for extracting a feature amount from the face area detected based on the shooting direction;
The similarity between the feature quantity extracted from the face image for dictionary by the face direction feature quantity extraction means and the feature quantity extracted from the face area in the input image by the feature quantity extraction means is calculated, and the similarity is Similarity calculating means for recognizing a face based on
An output means for outputting a recognition result based on the degree of similarity. The face area detecting means includes
A template created from a face image photographed from a predetermined direction is scanned on the input image to calculate the similarity between the input image and the template, and an area having the similarity equal to or greater than a threshold is detected as a face area. The face recognition apparatus according to claim 1 . In the face recognition method in an environment where the shooting direction of the face is constant,
A dictionary face image input means for inputting a dictionary face image in a certain shooting direction together with shooting direction information;
The face direction feature quantity extraction unit performs x in each pixel on a plurality of dictionary face images in the target shooting direction based on the shooting direction information input in the dictionary face image input step. The first principal component of the variance-covariance matrix with the direction and y-direction differential values as the explanatory variables is calculated, and the differential value and the first principal component are used to calculate (differential value in the x direction) × ( A facial direction feature that calculates a feature quantity of x component of the first principal component vector) + (differential value in y direction) × (y component of the first principal component vector) and extracts it as a feature quantity based on the shooting direction A quantity extraction step;
An image input step in which the image input means inputs an image to be recognized;
A face area detecting unit for detecting a face area based on a shooting direction from the input image;
A feature amount extracting means for extracting a feature amount from the face area detected based on the shooting direction;
The similarity calculation means calculates the similarity between the feature quantity extracted from the face image for dictionary by the face direction feature quantity extraction step and the feature quantity extracted from the face area in the input image by the feature quantity extraction step. A similarity calculating step for recognizing a face based on the similarity,
An output means, comprising: an output step for outputting a recognition result based on the similarity. The face area detecting step includes
A template created from a face image photographed from a predetermined direction is scanned on the input image to calculate the similarity between the input image and the template, and an area having the similarity equal to or greater than a threshold is detected as a face area. The face recognition method according to claim 3 . A face recognition program for causing a computer to function as each means according to claim 1 . A computer-readable recording medium on which a face recognition program for causing a computer to function as each means according to claim 1 is recorded.