JP2009003866A - Face authentication device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face authentication device whose collation accuracy is enhanced even if an image of a face part photographed by a monitoring camera. <P>SOLUTION: The face authentication device is provided with: a registered image acquisition part 5 which acquires the image of the face part as a registered image; an acquisition condition setting part 6 which sets the photographing direction of the face part of the registered image as a registered image acquisition condition; a registration database 2 which stores the registered image acquisition condition and the registered image by associating them with each other; a collated image acquisition part 7 which acquires the face part image of a person to be collated as a collated image; an acquisition condition estimation part 8 which sets the photographing direction of the face part of the collated image as a collated image acquisition condition; a registered image selection part 9 which selects the registered image associated with the registered image acquisition condition corresponding to the collated image acquisition condition from the registration database 2; and a collation processing part 10 which collates the registered image with the collated image by calculating similarity between the selected registered image and the collated image to output collation results. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face authentication apparatus that performs a matching process using a face image.

  A conventional face authentication apparatus includes a database in which face images of a plurality of persons are registered. When the face image to be verified is input, the input face image is verified with the face image in the database. Here, the face image registered in the database is referred to as a registered image, and the face image to be verified is referred to as a verification image.

  By the way, in a monitoring system including a monitoring camera, it is useful to include the face authentication device as described above in order to enhance the monitoring function. In this case, the face authentication device is configured to perform a matching process using the video of the surveillance camera.

A conventional face authentication device that uses video from a surveillance camera is used, for example, to find an illegal person in a store. In this case, the face image of the illegal person supplemented in the past is accumulated in the database as a registered image. And the collation image image | photographed with the surveillance camera of the store is collated with the registration image of a database (for example, refer patent document 1). Further, for example, when a face authentication device is provided in a residential facility, a face image of an authorized person such as a resident is stored in a database as a registered image and collated with a video of a monitoring camera.
JP 2005-196716 A

  However, in the conventional face authentication device, when using the video of the surveillance camera, as described below, it is not easy to obtain a face image suitable for collation, and there is a problem in that the authentication rate decreases. there were.

  In other words, as a registered image for face authentication, a face image taken from the front like a so-called ID photo is ideal. An ideal registered image can be obtained from illegal persons or unauthorized users who have been supplemented in the past as in the prior art. However, it is not common to obtain such an ideal registered image. Rather, a fraudulent act usually occurs, the fraudulent act is confirmed after the fact, and then a fraudulent person is identified from the accumulated surveillance video. Therefore, it should be considered that it is more common to use a face image obtained from a monitoring video as a registered image. In this case, since it is difficult for the surveillance camera to photograph a person's face from the front, a face image having a direction different from the front is obtained as a registered image. On the other hand, since the collation image is also acquired from the video of the surveillance camera, the face image has a different orientation from the front. Then, as a result of the orientation of the registered image and the collation image being different, the collation accuracy is lowered.

  As described above, conventionally, when the video of the surveillance camera is used, an ideal face image cannot be obtained, and the matching accuracy is lowered. On the other hand, it is desired that stable collation accuracy can be obtained even when video from a surveillance camera is used.

  The present invention has been made to solve the conventional problems, and is a face authentication device capable of improving collation accuracy even when a face image photographed by a surveillance camera is used as a collation registered image. Is to provide.

  The face authentication apparatus according to the present invention detects a face from an image, detects a face image of the registered image as a registered image, detects a shooting direction of the face of the registered image, and detects the detected shooting direction An acquisition condition setting unit that sets a registered image acquisition condition, a registration database that stores a plurality of registered image acquisition conditions and registered images in association with each other, a face of a person to be verified, and a face image A collation image acquisition unit to be acquired, an acquisition condition estimation unit to estimate the photographing direction of the face of the collation image and set the estimated photographing direction as a collation image acquisition condition, and a registered image acquisition corresponding to the collation image acquisition condition A registered image selection unit that selects a registered image associated with a condition from the registration database, and a collation that collates the registered image and the collated image by calculating the similarity between the selected registered image and the collated image And a processing section.

  According to this configuration, a plurality of registered images are acquired from the image, the shooting direction of the face portion of each registered image is associated with the corresponding registered image as the registered image acquisition condition, and the registered image is acquired based on the registered image acquisition condition. The image of the face taken by the surveillance camera as a registered image for verification by reducing the effect of the difference in the shooting direction between the registered image and the verification image on the verification accuracy. Even so, the collation accuracy can be improved.

  In the face authentication device according to the present invention, the registration database stores the weighting corresponding to the photographing direction of the face portion of the registered image in association with the registered image together with the registered image acquisition condition, and a plurality of collations based on the plurality of collation images. When the result is obtained, the collation processing unit adjusts a plurality of collation results based on the weights associated with the registered image acquisition conditions corresponding to the respective collation image acquisition conditions, and the face in the photographing direction of the face portion The weighting is set smaller as the relative angle with respect to the front part direction increases.

  According to this configuration, processing based on the difference in collation accuracy depending on the photographing direction of the face is performed. Then, considering that the collation accuracy decreases as the shooting direction deviates from the front direction, weighting is performed according to the shooting direction of the face portion of the registered image, and the matching result is adjusted based on the weighting. Accuracy can be improved.

  The face authentication apparatus according to the present invention generates a composite registration image by combining a registration image in a missing shooting direction based on a registration image stored in the registration database when there is a shooting direction missing in the registration database. The registration database stores the image capturing direction of the face of the composite registration image in association with the composite registration image as a registration image acquisition condition.

  According to this configuration, even if a registration image is missing from the registration database, a composite registration image can be generated by combining the registration images, and verification processing can be performed based on a larger number of images. Can be improved.

  In the face authentication apparatus according to the present invention, the registration database stores weights corresponding to the registered image and the composite registered image in association with the registered image together with the registered image acquisition condition, and a plurality of matching results are obtained based on the plurality of matching images. When obtained, the collation processing unit adjusts a plurality of collation results based on the weights associated with the registered image acquisition conditions corresponding to the respective collation image acquisition conditions, and compares them with the weights of the registered images for synthesis. The registered image weight is set small.

  According to this configuration, processing based on the difference in collation accuracy depending on the photographing direction of the face is performed. In consideration of the fact that the collation accuracy decreases as the shooting direction deviates from the front direction, weighting is performed according to the registered image and the composite registered image, and the collation result is adjusted based on the weighting. It can be improved.

  The present invention provides a face authentication device having an effect that a high verification rate can be stably obtained even when a face image captured by a monitoring camera is used as a registered image for verification. It is.

(First embodiment)
Hereinafter, a face authentication device according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a face authentication apparatus according to a first embodiment of the present invention.

  In FIG. 1, the face authentication apparatus 100 includes a registration unit 1, a registration database 2, a verification unit 3, and an image output unit 4. The registration unit 1 includes a registered image acquisition unit 5 and an acquisition condition setting unit 6. The collation unit 3 includes a collation image acquisition unit 7, an acquisition condition estimation unit 8, a registered image selection unit 9, a collation processing unit 10, and a collation result accumulation unit 11. The verification processing unit 10 includes a verification result determination unit 30.

  The image output unit 4 outputs an image captured by the monitoring camera or an image stored in the recording device to the registration unit 1 and the verification unit 3.

  The registered image acquisition unit 5 detects the face of the person to be monitored from the image output by the image output unit 4 and acquires the image of the face as a registered image.

  The acquisition condition setting unit 6 detects the shooting direction of the face portion of the registered image acquired by the registered image acquisition unit 5 and sets the detected shooting direction as the registered image acquisition condition. In this case, the acquisition condition setting unit 6 detects the photographing direction of the face with respect to the front direction of the face of the person to be monitored in three-dimensional coordinates based on the image information of the face that is a registered image. For example, “J.Xiao, S. Baker, I. Matthews, and T. Kanade,“ Real-time combined 2D + 3D active appearance models ”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume II, pp.535-542, 2004. ”. Further, the photographing direction may be detected based on a template that approximates the face by matching with a face template at various angles. Alternatively, the shooting direction may be detected from the relative positions of the eyes and nose of the face. Further, the movement direction of the face part may be predicted by tracking images of the face part that is continuous in time, and the shooting direction may be detected according to the predicted movement direction. Alternatively, when the face can be expected to face a specific direction such as a passage, the shooting direction may be detected based on the installation conditions of the monitoring camera and the spatial position of the face. Further, the shooting direction may be confirmed visually by an operator, the shooting direction may be input, and the input information may be detected.

  The registration database 2 stores the registered image acquisition condition set by the acquisition condition setting unit 6 and the registered image acquired by the registered image acquisition unit 5 in association with each other. In the above example, the registered image acquisition condition is relative angle information of the shooting direction with respect to the front direction of the face of the monitoring subject, and this information is associated with the registered image and stored in the registration database 2.

  As described above, the registration unit 1 stores the registration image in the registration database 2 based on the image output by the image output unit 4. The registration unit 1 detects the face portion of the same person to be monitored a plurality of times based on the image output from the image output unit 4, and stores the plurality of images of the face portion in the registration database 2 as registered images. Also good. In this case, a plurality of different registered image acquisition conditions corresponding to the plurality of registered images are obtained by the movement of the person to be monitored, etc., and the plurality of registered image acquisition conditions are associated with the plurality of registered images, and the registration database 2 Stored in

  Next, the configuration of the matching unit 3 will be described. The collation image acquisition unit 7 determines the face part of the collation target person from the image output by the image output unit 4 in order to collate the collation target person photographed by the monitoring camera with the registered image stored in the registration database 2. The face image of the person to be collated is detected as a collation image.

  The acquisition condition estimation unit 8 estimates the shooting direction of the face portion of the collation image, and sets the estimated shooting direction as the collation image acquisition condition. In this case, the acquisition condition estimation unit 8 detects the photographing direction of the face portion of the person to be collated in the three-dimensional coordinates based on the image information of the face portion of the collation image. As a method for detecting the shooting direction, the shooting direction detection method of the acquisition condition setting unit 6 described above may be used.

  The registered image selection unit 9 inputs the collation image acquisition condition set by the acquisition condition estimation unit 8, and registers the registration image associated with the registration image acquisition condition corresponding to the input collation image acquisition condition from the registration database 2. select. In the above example, the image acquisition condition of the registered image and the collation image is relative angle information of the shooting direction with respect to the front direction of the face. Therefore, the registered image selection unit 9 selects a registered image acquisition condition having a relative angle that matches or approximates the relative angle of the collation image acquisition condition from the registration database 2 and registers the registered image associated with the selected registered image acquisition condition. Is read from the registration database 2.

  The collation processing unit 10 collates the registered image with the collation image by calculating the similarity between the selected registered image and the collation image, and outputs a collation result. For collation of the face image, for example, a pattern recognition method described in Japanese Patent Application Laid-Open No. 2001-11868 can be applied. The verification result storage unit 11 stores the verification result calculated by the verification processing unit 10.

As described above, the collation unit 3 collates the collation image with the registered image based on the image output by the image output unit 4 and outputs a collation result. In this case, a tracking process may be performed by the collation unit 3, and a plurality of collation images may be obtained as the same collation subject moves, and a plurality of collation results may be obtained from the plurality of collation images. The plurality of verification results are stored in the verification result storage unit 11. Then, the verification result determination unit 30 of the verification processing unit 10 reads a plurality of verification results stored in the verification result storage unit 11 and determines the final verification result. For example, the matching result determination unit 30 determines a matching result having the highest similarity among the plurality of matching results as a final matching result. The final verification result is output from the verification processing unit 10.
Next, operation | movement of the face authentication apparatus 100 concerning this Embodiment is demonstrated using a flowchart.

  First, registration processing by the registration unit 1 in FIG. 1 will be described using the flowchart in FIG. The registered image acquisition unit 5 detects a face from the digital image output by the image output unit 4 and acquires the image of the face as a registered image (S100). For example, as shown in FIG. 3, the face 22 of the person to be monitored 21 is detected from the digital image taken by the monitoring camera 20, and the image of the face 22 is acquired as a registered image by the registered image acquisition unit 5. .

  The acquisition condition setting unit 6 detects the shooting direction with respect to the front direction of the face of the monitoring subject in three-dimensional coordinates, and sets the horizontal angle and the vertical angle of the shooting direction with respect to the front direction of the face as registered image acquisition conditions (S101). . For example, as shown in FIG. 3, the shooting direction D1 of the face 22 of the monitoring subject 21 is detected by the acquisition condition setting unit 6, and the relative angle of the shooting direction D1 with respect to the face front direction D2 is set. And as shown in FIG. 4, the relative angle information of the horizontal angle of the imaging | photography direction with respect to the front part direction of a face, and a vertical angle is set as registration image acquisition conditions. For example, the area A1 in FIG. 4 has a horizontal angle and a vertical angle in the shooting direction with respect to the front direction of the face in the vicinity of 0 °. Therefore, the registered image having the registered image acquisition condition of the area A1 is taken from the front direction of the face. It corresponds to a face image. The registered image having the registration image acquisition condition of the area A2 corresponds to a face image taken from the right side of the face, and the registered image having the registration image acquisition condition of the area A3 is from the left side of the face. This corresponds to the photographed face image. Further, the registered image having the registered image acquisition condition of the region A4 corresponds to a face image captured from the top of the head.

  The registration database 2 stores the relative angle information as shown in FIG. 4 in association with the registered image as a registered image acquisition condition (S102).

  It is determined whether or not the image of the face of the monitoring subject is further output from the image output unit 4 to the registration unit 1 (S103). When a further image is output, the process returns to step S100, and through steps S101 and S103, the registration database 2 stores a plurality of registered image acquisition conditions and registered images in association with each other. For example, when a plurality of registered images are obtained as the monitoring subject moves, the plurality of registered images are stored in the registration database 2 in association with a plurality of registered image acquisition conditions (relative angle information). The

  Note that it is not necessary to continuously repeat the registration process in steps S100 to S103, and the process in steps S100 to S103 may be repeated when a predetermined registered image acquisition condition is obtained. For example, when a registered image acquisition condition corresponding to the area F1 shown in FIG. 4 is set in advance and a registered image corresponding to the set registered image acquisition condition is obtained, the registration unit 1 registers the registered image in the registration database 2. May be stored. As a result, for example, only registered images for obtaining high collation accuracy such as a face front image can be stored in the registration database 2, and the data amount of unnecessary registered images can be reduced.

  Moreover, the registration image regarding each of a some monitoring subject may be stored in the registration database 2, and these registration images may be collated with the collation image by the collation part 3. FIG.

  If there is no other image to be registered in step S103, the registration unit 1 ends the registration process.

  Next, collation processing by the collation unit 3 in FIG. 1 will be described using the flowchart in FIG. First, the collation image acquisition unit 7 detects the face part of the collation target person from the image in which the collation target person is photographed, and acquires an image of the face part of the collation target person as a collation image (S200).

  The acquisition condition estimation unit 8 estimates the shooting direction of the face of the person to be checked in the three-dimensional coordinates based on the image information of the face of the check image, and determines the horizontal angle and the vertical angle of the shooting direction with respect to the front direction of the face. The verification image acquisition condition is set (S201). In this case, a shooting direction detection method similar to that of the acquisition condition setting unit 6 may be used.

  Next, a registered image acquisition condition having a relative angle that matches or approximates the relative angle of the collation image acquisition condition set by the acquisition condition setting unit 8 is selected from the registration database 2 and associated with the selected registered image acquisition condition. The registered image is read from the registration database 2 (S202).

  The collation processing unit 10 collates the registered image with the collation image by calculating the similarity between the selected registered image and the collation image as a score, and outputs a collation result (S203). In this case, the higher the similarity between the registered image and the collation image, the higher the score. The collation result accumulation unit 11 stores the collation result calculated by the collation processing unit 10 (S204).

  Next, the collation unit 3 determines whether the person to be collated is being tracked (S205). When the collation image acquisition unit 7 further acquires a collation target person's face image as a collation image, it is determined that the collation target person is being tracked, and the process returns to step S200 and goes through steps S201 to S204. Thus, the matching processing unit 10 matches a plurality of matching images with the registered image, and a plurality of matching results are stored in the matching result storage unit 11.

  In step S205, when the verification target person tracking process is completed, such as when the verification target person is out of the shooting range of the surveillance camera, the verification result determination unit 30 stores the verification result stored in the verification result storage unit 11. Based on the result, a final matching result is determined (S206). For example, the matching result determination unit 30 determines the highest score as the final matching result.

  In addition, collation results regarding each of the plurality of collation subjects are accumulated in the collation result accumulation unit 11, the collation result determination unit 30 determines the highest score as the final collation result, and the collation processing unit 10 The information on the person to be verified corresponding to the result of the verification may be output to a display unit (not shown) such as a display.

  As described above, according to the present embodiment, a plurality of registered images are acquired from an image, the shooting direction of the face portion of each registered image is used as a registered image acquisition condition, and associated with the corresponding registered image. By collating the registered image and the collation image based on the acquisition conditions, the effect of the difference in the shooting direction between the registration image and the collation image on the collation accuracy is reduced, and the face image photographed by the surveillance camera is reduced. Even when a registered image for verification is used, verification accuracy can be improved.

  In the present embodiment, the face direction limit is set to ± 40 ° in FIG. 4, but a larger angle may be set as the face direction limit depending on the face direction estimation performance.

(Second Embodiment)
As shown in FIG. 6, a face authentication apparatus 200 according to the second embodiment of the present invention includes a registration unit 1 including a shooting direction weighting setting unit 12, and a collation result adjustment unit included in the collation processing unit 10 of the collation unit 3. 13 is different from the face authentication apparatus 100 according to the first embodiment of FIG. In addition, about the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and since it bears the same function and operation | movement, description is abbreviate | omitted.

  The shooting direction weight setting unit 12 sets a weight according to the shooting direction of the registered image, and stores the set weight in the registration database 2 in association with the registered image together with the registered image acquisition condition.

  The collation result adjustment unit 13 adjusts the collation result based on the weighting set by the shooting direction weighting setting unit 12 when a plurality of collation results are obtained. For example, the matching result adjustment unit 13 performs weighted averaging of a plurality of scores calculated as matching results based on weighting. In this case, the scores may be weighted and averaged using only the matching result having a high score equal to or higher than a predetermined score among the plurality of matching results.

  Next, operation | movement of the face authentication apparatus 200 concerning 2nd Embodiment is demonstrated using a flowchart. Note that description of operations similar to those of the face authentication apparatus 100 according to the first embodiment is omitted.

  As shown in the flowchart of FIG. 7, in the registration process by the registration unit 1 of FIG. 6, in step S <b> 701, the face direction weighting setting unit 12 performs weighting according to the registered image acquisition condition set by the acquisition condition setting unit 6. And the set weighting is stored in the registration database 2 in association with the registered image together with the registered image acquisition condition. In this case, the shooting direction weighting setting unit 12 sets the weighting according to the shooting direction of the registered image so that the weighting decreases as the relative angle of the shooting direction with respect to the front direction of the face increases. For example, as shown in FIG. 8, the registered image having the registration image acquisition condition of the area A1 corresponds to the face image taken from the front direction of the face, and thus the weight related to the registration image acquisition condition of the area A1 is the largest. W5. Further, since the registered image having the registration image acquisition condition of the area A2 corresponds to the face image taken from the right side of the face, the weight W4 related to the registration image acquisition condition of the area A2 is the registration image acquisition of the area A1. It is set smaller than the weighting W5 related to the conditions. In addition, since the relative angle in the shooting direction of the area A5 is the maximum, the weight for the registered image acquisition condition of the area A5 is the minimum W1.

  As described above, the shooting direction weighting setting unit 12 sets the weighting according to the shooting direction of the registered image so that the weighting decreases as the relative angle of the shooting direction with respect to the front direction of the face increases. In general, a face image taken from the front of the face includes many feature points such as the binocular, nose, and mouth, so that the matching process is performed using the face image of the front of the face. Since high collation accuracy can be obtained when performing, the shooting direction weighting setting unit 12 sets weighting so that the weight of the collation result increases as the shooting direction is closer to the face front direction.

  In step S102, the registration database 2 stores the weighting information corresponding to the photographing direction of the face portion of the registered image in association with the registered image together with the registered image acquisition condition.

  Next, collation processing by the collation unit 3 in FIG. 6 will be described using the flowchart in FIG. 9. If the tracking process ends in step S205 and there is no other image to be collated, the process proceeds to step S901.

  In step S <b> 901, the matching result adjustment unit 13 performs a weighted average of the plurality of matching results based on the weighting set by the shooting direction weighting setting unit 12. For example, when the collation process is performed based on the registered images having the registered image acquisition conditions of the areas A1, A2, and A5 shown in FIG. 8, the respective weights W5, W4, and W1 of the areas A1, A2, and A5 are included in the respective collation results. Is added to calculate a weighted average, whereby the matching processing unit 10 determines the final matching result.

  As described above, according to the present embodiment, processing based on the difference in collation accuracy depending on the photographing direction of the face is performed. Then, considering that the collation accuracy decreases as the shooting direction deviates from the front direction, weighting is performed according to the shooting direction of the face portion of the registered image, and the matching result is adjusted based on the weighting. Accuracy can be improved.

(Third embodiment)
As shown in FIG. 10, the face authentication device 300 according to the third exemplary embodiment of the present invention includes the first and second features in that the registration unit 1 includes a registered image composition unit 14 and a composite image weighting setting unit 15. Unlike the face authentication devices 100 and 200 according to the first embodiment, the face authentication device 100 according to the first embodiment is different from the face authentication device 100 according to the first embodiment in that the matching processing unit 10 of the matching unit 3 includes a matching result adjustment unit 13. In addition, about the structure similar to 1st and 2nd Embodiment, since the same code | symbol is attached | subjected and it bears the same function and operation | movement, description is abbreviate | omitted.

  When there is a missing registration image in the registration database 2 in the registration database 2, the registration image synthesis unit 14 combines the registration image in the missing shooting direction based on the registration image stored in the registration database 2. An image is generated, and the photographing direction of the face portion of the composite registration image is stored in the registration database 2 in association with the composite registration image as a registration image acquisition condition of the composite registration image.

  The composite image weight setting unit 15 sets weights according to the registered image and the composite registered image, and stores the set weight in the registration database 2 in association with the registered image together with the registered image acquisition condition.

  The collation result adjusting unit 13 adjusts the collation result based on the weighting set by the composite image weighting setting unit 15 when a plurality of collation results are obtained. For example, the matching result adjustment unit 13 performs weighted averaging of a plurality of scores calculated as matching results based on weighting. In this case, the scores may be weighted and averaged using only the matching result having a high score equal to or higher than a predetermined score among the plurality of matching results.

  Next, operation | movement of the face authentication apparatus 300 concerning 3rd Embodiment is demonstrated using a flowchart. In addition, description is abbreviate | omitted about the operation | movement similar to the face authentication apparatuses 100 and 200 concerning 1st and 2nd embodiment.

  As shown in the flowchart of FIG. 11, in the registration process by the registration unit 1 of FIG. 10, if there is no other image to be registered for the image from the image output unit 4 in step S103, the process proceeds to step S1101.

  In step S <b> 1101, the registered image composition unit 14 synthesizes the registered images in the missing shooting direction based on the registered images stored in the registration database 2 to generate a composite registered image. For example, as shown in FIG. 12, registered images in the shooting direction corresponding to the regions 1 and 3 are stored in the registration database 2, and a registered image in the shooting direction corresponding to the region 6 is missing from the registration database 2. If there is, the registered image composition unit 14 generates a composite registered image corresponding to the region 6 based on the registered images corresponding to the region 1 and the region 3. In this case, the registered image composition unit 14 generates a composite registered image by interpolation processing or the like. Specifically, the registered image combining unit 14 estimates a face image in the process of changing from the shooting direction of the region 1 to the shooting direction of the region 3, and generates a combined registered image. In addition, when a registration image (requested shooting direction) to be acquired is set in advance and the registration image in the requested shooting direction is missing from the registration database 2, the registered image composition unit 14 deletes the face image to be lost. The composite registration image may be generated.

  In step S106, the composite image weight setting unit 15 sets weights corresponding to the registered image and the composite registered image, and stores the set weight in the registration database 2 in association with the registered image together with the registered image acquisition condition. In this case, the composite image weight setting unit 15 sets weights according to the registered image and the composite registered image so that the weight of the composite registered image with respect to the registered image becomes small. For example, as shown in FIG. 13, since the face image corresponding to the area A1 and the area A3 is a registered image stored in the registration database 2, the weight for the registered image acquisition conditions of the area A1 and the area A3 is W7. Become. On the other hand, the face image corresponding to the area A6 is a composite registration image synthesized based on the face images corresponding to the area A1 and the area A3. Therefore, the weight related to the registration image acquisition condition of the area A6 is W6. The registered image weight W6 is set smaller than the registered image weight W7. Generally, a composite registration image is a face image estimated from a registration image, and an error is likely to occur in the process of synthesis. Therefore, when performing a collation process using a composite registration image, the collation accuracy is high. Therefore, the composite image weight setting unit 15 sets the weight of the composite registered image to be smaller than the weight of the registered image.

  Next, the collation process by the collation part 3 of FIG. 10 is demonstrated. The collation process of the face authentication apparatus 300 according to the present embodiment is the same as the collation process of the face authentication apparatus 200 according to the second embodiment shown in FIG.

  That is, in step S206, the matching result adjustment unit 13 performs weighted averaging of a plurality of matching results based on the weighting set by the composite image weighting setting unit 15. For example, when the collation process is performed based on the registered images having the registered image acquisition conditions of the areas A1, A3, and A6 shown in FIG. 13, the respective weights W7, W7, and W6 of the areas A1, A3, and A6 are included in the respective collation results. Is added to calculate a weighted average, whereby the matching processing unit 10 determines the final matching result.

  As described above, according to the present embodiment, even when there is a shooting direction in which the registered image is missing in the registration database, a composite registered image can be generated by combining the registered images, and more images can be generated. The collation accuracy can be improved by performing the collation process based on the above.

  Moreover, according to this Embodiment, the process based on the difference in the collation precision by the imaging | photography direction of a face part is performed. In consideration of the fact that the collation accuracy decreases as the shooting direction deviates from the front direction, weighting is performed according to the registered image and the composite registered image, and the collation result is adjusted based on the weighting. It can be improved.

  As a modification, the second embodiment and the third embodiment may be combined. In this case, two types of weighting processes may be performed. That is, even if the weighted average of the matching results is calculated using the weighting according to the shooting direction described in the second embodiment and the weighting of the registered image and the composite registered image described in the third embodiment. Good.

  As a further modification, one type of weighting may be set by combining the above two types of weighting. This combined weighting changes depending on the shooting direction, and changes depending on whether it is a registered image or a combined registered image. Then, the weighted average of the matching result may be calculated using the combined weighting. Such a configuration is also included in the scope of the present invention.

  The face authentication device according to the present invention has an effect that a high collation rate can be stably obtained even when a face image captured by a monitoring camera is used as a registered image for collation. This is useful as a face authentication device or the like that performs a matching process using a face image.

1 is a block diagram of a face authentication apparatus according to a first embodiment of the present invention. The flowchart of the registration process of the face authentication apparatus concerning the 1st Embodiment of this invention The figure for demonstrating the registration image acquisition process in a registration process, and a registration image acquisition condition setting process The figure for demonstrating the relative angle information of the horizontal angle of the imaging | photography direction with respect to the front direction of a face, and a vertical angle The flowchart of the collation process of the face authentication apparatus concerning the 1st Embodiment of this invention The block diagram of the face authentication apparatus concerning the 2nd Embodiment of this invention Flowchart of registration processing of face authentication apparatus according to second embodiment of present invention The figure for demonstrating the weighting set according to the imaging | photography direction of the registration image Flowchart of collation processing of face authentication device according to second embodiment of the present invention The block diagram of the face authentication apparatus concerning the 3rd Embodiment of this invention Flowchart of registration processing of face authentication apparatus according to third embodiment of the present invention The figure for demonstrating combining the registration image missing in a registration database based on the registration image stored in the registration database. The figure for demonstrating the weighting set according to the registration image and the composite registration image

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Registration part 2 Registration database 3 Collation part 4 Image output part 5 Registration image acquisition part 6 Acquisition condition setting part 7 Collation image acquisition part 8 Acquisition condition estimation part 9 Registered image selection part 10 Collation process part 11 Collation result storage part 12 Imaging | photography direction Weighting setting unit 13 Matching result adjusting unit 14 Registered image combining unit 15 Composite image weighting setting unit 30 Matching result determining unit 100, 200, 300 Face authentication device

Claims (4)

A registered image acquisition unit that detects a face from an image and acquires the image of the face as a registered image;
An acquisition condition setting unit that detects a shooting direction of the face portion of the registered image and sets the detected shooting direction as a registered image acquisition condition;
A registration database that stores a plurality of registered image acquisition conditions and the registered images in association with each other;
A collation image acquisition unit that detects a face portion of the person to be collated and obtains an image of the face portion as a collation image;
An acquisition condition estimation unit that estimates a shooting direction of the face portion of the collation image and sets the estimated shooting direction as a collation image acquisition condition;
A registered image selection unit that selects, from the registration database, the registered image associated with the registered image acquisition condition corresponding to the verification image acquisition condition;
A face authentication apparatus comprising: a matching processing unit that compares the registered image and the matching image by calculating a similarity between the selected registered image and the matching image. The registration database stores a weight according to the shooting direction of the registered image in association with the registered image together with the registered image acquisition condition,
When a plurality of collation results are obtained based on a plurality of collation images, the collation processing unit is based on the weights associated with the registered image acquisition conditions corresponding to the respective collation image acquisition conditions. , Adjust the plurality of matching results,
The face authentication apparatus according to claim 1, wherein the weighting is set to be smaller as the relative angle of the photographing direction with respect to the front direction of the face increases. When there is the missing shooting direction in the registration database, a registered image composition that generates a synthesized registration image by synthesizing the registration image of the missing shooting direction based on the registration image stored in the registration database Part
3. The face authentication apparatus according to claim 1, wherein the registration database stores an imaging direction of the face of the composite registration image in association with the composite registration image as the registration image acquisition condition. The registration database stores weights corresponding to the registered image and the composite registered image in association with the registered image together with the registered image acquisition condition,
When a plurality of collation results are obtained based on a plurality of collation images, the collation processing unit is based on the weights associated with the registered image acquisition conditions corresponding to the respective collation image acquisition conditions. , Adjust the plurality of matching results,
The face authentication apparatus according to claim 3, wherein the weight of the composite registered image is set smaller than the weight of the registered image.

Images