JP2008033810A - Face image collation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face image collation device capable of appropriately selecting a registration face image of an update object so as not to cause deterioration of collation accuracy when a plurality of registration face images corresponding to a photographic condition such as face direction in collation or the like is prepared. <P>SOLUTION: The face image collation device for performing individual collation by collating an input face image with a plurality of registration face images comprises a storage part storing the plurality of registration face images for the face of the same person taken in different photographic conditions and a candidate face image to be used for update for each of the plurality of registration face images; a candidate face image update part for calculating a similarity that is the degree of similarity of an input face image acquired by an input face image acquisition part to the plurality of registration face images, comparing the similarity of each of the plurality of registration face images to the input face image, selecting a registration face image having the largest similarity, and updating a candidate face image corresponding to the selected registration face image with the input face image; and a registration face image update part for updating the registration face image with the corresponding candidate face image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a face image collation apparatus that performs personal authentication using a face image, and more particularly to a face image collation apparatus that sequentially updates a registered face image of a person registered in advance.

  In recent years, so-called biometric authentication (biometrics) that verifies the identity of a person based on their physical characteristics when managing entry / exit of users or restricting access to confidential data in facilities that require a high level of security. (Metrics) method is used. Physical features such as fingerprints, eyeball irises, and faces are used as the physical features.

  In a face image collation device using a face image, the input face image is collated with a registered face image, which is a face image of the person registered in advance, with the input face image captured by a video camera or the like at the time of collation. Determine whether or not. Such a face image collation device is used for room entry / exit management, and if the collation result is determined to be the person, the electric door lock is unlocked, and if it is not the person, a warning or the like is issued. .

  By the way, it is common to prepare a face image taken facing the front as a registered face image. However, in a scene where collation is performed by the face image collation device, the image may be photographed in a direction other than the front due to various reasons such as photographing timing, user's habit, and installation status of the device. In some cases, a user who has registered with glasses wears off his / her glasses. Therefore, if only one registered face image facing the front is prepared, the user may not be correctly determined as the user.

  For this reason, conventionally, a face image collation apparatus prepares a plurality of registered face images having different shooting conditions. For example, in Patent Document 1, the face orientation is first detected, and a registered face image photographed under the same conditions as the detected face orientation is selected, and then matching processing is performed.

  In addition, since the human face fluctuates due to aging, exchange of glasses, makeup, and the like, if the registered face image is not updated at all, it may be determined that the person is not the person.

For this reason, conventionally, in the face image matching device, the registered face image is appropriately updated. For example, in Patent Document 2, a registered face image is presented to a user, a manual input is prompted, and a determination is made as to whether or not the update is allowed.
JP 2002-288670 A JP-A-2005-115481

However, in the conventional face image matching device, the user has to select a registered face image to be updated from a plurality of registered face images with different shooting conditions, which is inconvenient for the user.
The registered face image is basically a similar image because it is a face image of the same person. On the other hand, the degree of similarity between the input face image and the registered face image in the face image matching device, that is, the degree of similarity, does not necessarily match the degree of recognition that a person is visually similar.
For this reason, there is a possibility that the registered face image of the face image facing upward is updated with the input face image that is a face image facing downward. Therefore, when the registered face image is updated with an input face image under different shooting conditions, the update is performed even though a plurality of appropriate registered face images are prepared to cope with variations in shooting conditions in actual operation. May result in a face image with biased shooting conditions.

  Therefore, the present invention has been made in view of such a problem, and in the case where a plurality of registered face images are prepared so as to be able to cope with face photographing conditions such as face direction fluctuation at the time of collation and presence / absence of glasses. It is an object of the present invention to provide a face image collation apparatus that automatically selects and updates a registered face image corresponding to a face photographing condition and does not cause degradation of collation accuracy.

  In order to solve such a problem, the present invention is a face image collation device that collates a plurality of registered face images stored in advance with an input face image and performs person collation, and an input image obtained by photographing a face of a collator An input face image acquisition unit that acquires an input face image based on the image, a plurality of registered face images obtained by photographing the same person's face under different shooting conditions, and candidate face images used for updating for each of the plurality of registered face images Calculating the similarity that is the degree of similarity between the input face image acquired by the storage unit and the input face image acquisition unit and the plurality of registered face images, and the similarity exceeds a matching threshold value Compare the similarity to the input face image for each of a plurality of registered face images calculated by the collation unit, the collation unit that determines that the user is the person, the output unit that outputs the collation result of the collation unit to the outside, and the most similar Select a registered face image with a high degree and select the registered face image Provided is a face image matching device having a candidate face image update unit that updates a candidate face image corresponding to an image with the input face image, and a registered face image update unit that updates a registered face image corresponding to the candidate face image To do.

  With this configuration, the present invention selects a registered face image having the highest similarity to the input face image and updates the selected registered face image used for updating a plurality of registered face images with different shooting conditions. The candidate face image corresponding to the face image is updated with the input face image.

  In a preferred aspect of the present invention, three or more candidate face images are stored for one registered face image in the storage unit, and the candidate face image update unit indicates the degree of similarity between the candidate face images. The degree of similarity is calculated, the degree of similarity between the candidate face image having the highest degree of mutual similarity and the input face image is calculated, and the candidate face is used using the input face image only when the degree of similarity exceeds the update threshold. Update the image.

  According to this aspect, the present invention can prevent the candidate face image from being updated with the input face image under another person or other shooting conditions.

  In another aspect of the present invention, the storage unit stores three or more candidate face images for one registered face image, and the candidate face image update unit has a similarity between the candidate face image and the input face image. The candidate similarity indicating the degree of the image is calculated and stored in the storage unit using the input face image when an intermediate candidate similarity that is neither maximum nor minimum exceeds the update threshold. The candidate face image being updated is updated.

  According to this aspect, the present invention can prevent the candidate face image from being updated with the input face image under another person or other shooting conditions.

  According to another aspect of the present invention, the registered face image update unit determines a deterioration state of the registered face image, and determines that the registered face image is deteriorated, and determines the candidate face image corresponding to the registered face image. The registered face image is updated at. According to this aspect, the present invention can update the registered face image at an appropriate time.

  Furthermore, in another aspect of the present invention, the candidate face image update unit updates the candidate face image only when the collation unit determines that the person is the person himself / herself. According to this aspect, the present invention reduces the possibility that the candidate face image is updated with the face image of another person, and improves the reliability as the collation device.

  The present invention is capable of updating an appropriately selected registered face image with an input face image even if the face image matching apparatus has prepared a plurality of registered face images with different shooting conditions. Usability and reliability can be improved.

  Hereinafter, an embodiment in which a face image collation device according to the present invention is applied to an access control system for a room where an electric lock is installed at an entrance door will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an overall configuration of an access control system 10 including a face image matching device 100 according to the present invention. The access control system 10 includes a face image matching device 100, an image sensor that captures a user's face, a camera 150 including a lens, and a key switch that is operated by the user to input user information (ID code, etc.) And the electric lock 160 installed on the door that operates based on the output from the face image matching device 100.

  The face image matching device 100 includes an input unit 102 that is an interface for acquiring a face image from a connected camera 150, a selection input unit 103 that is an interface for acquiring user information from a connected numeric keypad 151, and verification. A storage unit 104 that stores various face images and programs used for the image, a feature extraction unit 105 that is a module that extracts features from the face image, a matching unit 106 that is a module that compares and determines the features of two face images, and storage An update unit 107 that is a module for updating the registered face image 104a and the candidate face image 104b stored in the unit 104, an output unit 108 that outputs a lock or unlock signal to the connected electric lock 160, a collation result, etc. Display unit 109, and a control unit 101 that controls the overall processing of the face image matching device 10.

Next, each part of the face image matching device 10 will be described in detail.
The selection input unit 103 acquires a number (ID number) that is input from the numeric keypad 151 by the user and identifies the user, and outputs the number to the control unit 101. For example, in an office building, an employee number is input. A touch panel or an IC card reader may be used instead of the numeric keypad.

  If the ID number input by the user operating the numeric keypad 151 is an ID number registered in advance, the input unit 102 acquires an input image including a face image of a person photographed by the camera 150, and controls the control unit 101. Output to. The input image acquisition timing may be triggered by installing an ultrasonic sensor or the like and detecting the approach of the user.

  The storage unit 104 is realized by a storage medium such as a flash memory or a hard disk, and includes a plurality of registered face images 104a, a plurality of candidate face images 104b, an ID number for each user associated with the registered face images 104a and the candidate face images 104b, And the input image acquired in the imaging part 102 is memorize | stored temporarily. In the present embodiment, the image itself is stored. However, the feature amount extracted by the feature extraction unit 105 may be stored in place of various face images. If the feature amount is stored, the required storage capacity can be reduced and the processing speed can be improved.

  In addition, although not shown, the storage unit 104 includes an extraction module that is a program executed by the feature extraction unit 105, a verification module that is a program executed by the verification unit 106, and an update unit 107. Various parameters such as an update module, a verification threshold value used in those processes, and a flag indicating the state of the process are stored.

Here, the registered face image 104a and the candidate face image 104b will be described.
The registered face image 104a is a face image that serves as a reference for personal verification, and is used to verify whether the input facial image is a permitted facial image. The registered face image 104a can be face-checked if at least one registered face image 104a is used for each user. However, in order to cope with daily fluctuations such as face orientation, presence / absence of glasses, and changes in facial expressions, the registered face image 104a may be changed in advance. It is effective to store an image including In the present embodiment, three registered face images 104a are prepared for each person in order to cope with a change in face orientation, and an upward face, a front face, and a downward face are stored. The number of registered face images may vary depending on the user. Each registered face image 104a is updated using a candidate face image 104b by a registered face image updating unit 1071 described later.

The candidate face image 104b is a face image that is used to update the registered face image 104a, and is an input face image that is determined to be the person by comparing with the registered face image 104a. In this embodiment, three candidate face images 104b are prepared for each registered face image 104a, but three or more face images may be used. The candidate face image 104b is updated by candidate face image update means 1072 described later.
A candidate face image 104b is prepared for each registered face image 104a. In the present embodiment, since three registered face images 104a are prepared per person, nine candidate face images 104b are prepared.

  The plurality of registered face images 104a, the plurality of candidate face images 104b associated with each registered face image, and the ID number are stored in the storage unit 104 in association with each other. A set for each ID number is stored corresponding to the user of the face image matching device 100.

  The feature extraction unit 105, the collation unit 106, and the update unit 107 schematically show modules that are various programs stored in the storage unit 104 (not shown) as functional blocks executed by the control unit 101. Yes.

  The feature extraction unit 105 is useful for collation from face image extraction means 1051 that generates an input face image obtained by cutting out a face area from the input image, and various face images of the input face image, the registered face image 104a, and the candidate face image 104b. A feature amount calculation unit 1052 that calculates a feature amount is included.

  In the present embodiment, the input face image acquisition unit includes a face image cutout unit in the input unit 102 and a feature extraction unit 105 described later.

  The face image cutout unit 1051 generates an input face image from the input image input from the input unit 102 using a neural network learned in advance from a large number of face images and a non-face image in which no face is captured. For example, it is only necessary to change the position and size of the rectangle to be cut out and extract the rectangle with the largest output as the face area. The image of the rectangular face area is cut out as an input face image and stored in the storage unit 104. A method other than the neural network may be used.

The feature amount calculation unit 1052 calculates a feature amount for the individual determination from the input face image, the registered face image 104a, and the candidate face image 104b stored in the storage unit 104.
Specifically, feature points exhibiting human features such as the corners of the eyes, the corners of the eyes, the edge of the mouth, the center of the mouth, and the like are extracted from the face image from which feature amounts are to be extracted by the template matching method. Since the template matching method is a general method in which a template image prepared in advance for each feature point such as the corner of the eye and the input image are shifted to perform matching processing, detailed description thereof is omitted. Other methods may be used.

  Next, the feature amount calculation unit 1052 calculates a feature amount such as direction, frequency, spread, and position in the vicinity of each feature point by applying a Gabor filter to each extracted feature point with the feature point as the center. To do. By using a Gabor filter, feature quantities can be calculated from local images around the feature points, so it is possible to extract feature quantities while avoiding the effects of changes in overall brightness due to changes in lighting, etc. It becomes. The feature amount calculating unit 1052 stores the calculated feature amount in the storage unit 104 together with the face image. In addition, you may weight according to the site | part of a feature point, and the kind of feature. Also, other parameters may be used as long as the parameters express biological characteristics.

  The collation unit 106 compares two of the input face image, the registered face image 104a, or the candidate face image 104b stored in the storage unit 104, and collates whether the face images of the same person. The collation unit 106 includes similarity calculation means 1061 and determination means 1062.

  The similarity calculation unit 1061 calculates the similarity between the feature amounts calculated by the feature extraction unit 105 for the two face images. For example, the feature amount of the input face image and the feature amount of the registered face image 104a are read from the storage unit 104, respectively. Next, each feature amount is regarded as a vector, and the reciprocal of the length of the difference vector obtained by calculating the difference between the vectors is the similarity. The degree of similarity is the degree to which two images are conceptually similar. For this reason, in the present embodiment, digitization is attempted using the feature vector, but the present invention is not limited to this, and other methods such as Euclidean distance may be used. The similarity calculated by the similarity calculation unit 1061 is sent to the determination unit 1062 or the update unit 107.

  The determining unit 1062 compares the similarity calculated by the similarity calculating unit 1061 with a collation threshold value stored in advance, so that the two face images input to the collation unit 106 belong to the same person. Determine whether. For example, it is determined whether or not the input face image is the face image of the registered face image 104a. When the determination result is determined to be the person, the electric lock 160 is output to the electric lock 160 via the output unit 108 and the electric lock 160 is unlocked. If you are not the person, do not output. The determination result is output to the display unit 109 as a determination result. If the user is the person himself, the verification OK is displayed, and if it is not the person himself, the verification NG is displayed to the user.

  The update unit 107 includes a registered face image update unit 1071 that updates the registered face image 104a, a candidate face image update unit 1072 that updates the candidate face image 104b, and a registered face image selection unit 1073 that selects the registered face image 104a. have.

The registered face image selection unit 1073 is a process of selecting the registered face image 104a to update the associated candidate face image 104b from the plurality of registered face images 104a stored in the storage unit 104.
Specifically, for each of the plurality of registered face images 104a corresponding to the ID number input from the selection input unit 103, the similarity calculation unit 1061 calculates the similarity (identity similarity) with the input face image. However, this is a process of selecting the registered face image 104a showing the maximum similarity.
In the present embodiment, the registered face image 104a stores an upward face, a front face, and a downward face. For example, when a front face image is acquired as the input face image, the most similar face image is obtained. The registered face image 104a facing front, which has a high degree, is selected.

  The registered face image updating unit 1071 identifies a face image that is unlikely to be another person's face image from the plurality of candidate face images 104b, and updates the registered face image 104a.

  The registered face image update unit 1071 starts processing when the determination unit 1062 determines that the person is the person, and selects and selects the candidate face image 104b for updating the registered face image 104a from the plurality of candidate face images 104b. This is a process of updating the registered face image 104a with the candidate face image 104b.

  Specifically, the registered face image update unit 1071 reads three candidate face images 104b corresponding to the registered face image 104a selected from the storage unit 104 by the registered face image selecting unit 1073. Next, the similarity calculation unit 1061 calculates the mutual similarity of the read candidate face images 104b, and acquires three similarities as shown in FIG. Here, the similarity between the candidate face image 1 and the candidate face image 2 is the mutual similarity A, the similarity between the candidate face image 2 and the candidate face image 3 is the similarity B, and the candidate face image 3 and the candidate face image 1 For the sake of convenience.

  Next, the mutual similarity is calculated so that the candidate face image is similar to the other candidate face images. Specifically, in the candidate face image 1, the average value of the similarity A and the similarity C is calculated as the mutual similarity 1. In the candidate face image 2, the average value of the similarity B and the similarity A is calculated as the mutual similarity 2. In the candidate face image 3, the average value of the similarity C and the similarity B is calculated as the mutual similarity 3. That is, “mutual similarity 1 = (similarity A + similarity C) / 2” “mutual similarity 2 = (similarity B + similarity A) / 2” “mutual similarity 3 = (similarity C + similarity B)” / 2 "calculates the mutual similarity of each candidate face image.

  Next, the mutual similarity of each candidate face image is compared, and the candidate face image 104b with the highest similarity is specified. For example, if the comparison result is a result of mutual similarity 1> mutual similarity 2> mutual similarity 3, the mutual similarity 1 has the highest similarity, and thus the candidate face image 1 is specified. Then, the registered face image 104a is updated with the identified candidate face image 1.

  In the operation of the access control system, compared to the frequency of fraudulent acts by others, acceptance of others, and the appearance of others' face images in the candidate face image 104b, the use of the person and acceptance of the person to the candidate face image 104b There are overwhelmingly more. Therefore, the other person's candidate face image 104b has a low similarity to the other candidate face images 104b among the plurality of candidate face images 104b, and conversely, the person's candidate face image 104b has a higher mutual similarity. In this embodiment, the candidate face image 104b having the highest mutual similarity is updated. However, if the number of candidate face images 104b is increased, the number of candidate face images 104b may be higher than the highest. The candidate face image 104b may be specified.

  Furthermore, even if the input face image under other shooting conditions is updated in the candidate face image 104b, the possibility that the mutual similarity of the candidate face image 104b under the other shooting conditions is the highest is low. Therefore, the possibility that the input face image under other shooting conditions is updated to the registered face image 104a is reduced.

  In the present embodiment, the mutual similarity is obtained only for the candidate face image 104b, but the mutual similarity may also be obtained for the registered face image 104a.

  Next, the candidate face image update unit 1072 will be described. The candidate face image update unit 1072 starts processing when the determination unit 1062 determines that the person is the person, and updates the candidate face image 104b by replacing the input face image with the one most likely to be another person. To do.

The processing of the candidate face image update unit 1072 will be described in detail with reference to FIG. FIG. 3 is a processing flow of the candidate face image update means. First, in S200, the highest candidate face image is specified from the mutual similarity of the candidate face images 104b obtained in S208, which will be described later, and is read from the storage unit 104. That is, the mutual similarity of each candidate face image is compared, and the candidate face image 104b having the highest similarity is specified. For example, if the comparison result is a result of mutual similarity 1> mutual similarity 2> mutual similarity 3, the candidate face image 1 having the highest similarity is selected.
Here, the reason why the candidate face image 1 having the highest mutual similarity is selected is that the candidate face image 1 is unlikely to be the face image of another person and is likely to be the face image of the person himself / herself. Then, the similarity calculation unit 1061 calculates the similarity between the candidate face image 1 and the input face image (candidate similarity 1).

  In S202, it is determined whether or not the candidate face image 104b is updated by comparing whether or not the candidate similarity 1 with the candidate face image 1 exceeds the update threshold stored in the storage unit 104. That is, when the candidate similarity 1 is equal to or lower than the update threshold, it is determined that the input face image is inappropriate for updating the candidate face image 104b, and this candidate face image update process is terminated without updating.

  In S202, when the candidate similarity 1 is larger than the update threshold, it is determined that the input face image is appropriate for updating the candidate face image 104b, and the process proceeds to S204. The update threshold depends on the similarity calculation method, collation accuracy, and the like. The candidate face image 104b may be updated to the registered face image 104a, and it is necessary to eliminate the input face image under the possibility of others and other shooting conditions as much as possible. Therefore, it is better to set the update threshold value higher. Good. On the other hand, if a too high update threshold is set, it becomes impossible to follow the face change of the person due to aging or the like. Therefore, the update threshold is determined empirically while taking these factors into consideration.

  In S204, similar to S200, the similarity calculation unit 1061 uses the similarity between the candidate face image 2 and the input face image (candidate similarity 2), and the similarity between the candidate face image 3 and the input face image (candidate similarity 3). Is calculated.

  In S206, the candidate face image 104b having the lowest similarity among the candidate similarity 1, candidate similarity 2, and candidate similarity 3 is selected, and the candidate face image 104b is replaced with the input face image.

  The candidate face image 104b having the minimum similarity to the input face image means that the candidate face image 104b is most similar to the input face image determined to be the person. Even if another person has been accepted and the candidate face image 104b includes a face image of another person or another shooting condition, the candidate face image of the other person or the candidate face image of another shooting condition is the same and the same. Since the degree of similarity with the input face image is lower than the candidate face image under the shooting conditions, it is possible to update the candidate face image or the like of another person included in the candidate face image 104b to the input face image by using the person himself / herself. . Therefore, even if a face image of another person is included in the candidate face image 104b, the candidate face image 104a can be quickly updated with the input face image of the person.

In S208, the mutual similarity of the updated candidate face image 104b is calculated.
The candidate face image updating unit 1072 calculates the similarity between the candidate face images 104b including the updated candidate face image 104b, and acquires three similarities. Similar to the case of the registered face image update unit 1071, the similarity between the candidate face image 1 and the candidate face image 2 is the mutual similarity A, the similarity between the candidate face image 2 and the candidate face image 3 is the similarity B, and the candidate The similarity between the face image 3 and the candidate face image 1 is referred to as similarity C for convenience.

  Next, the mutual similarity is calculated so that the candidate face image is similar to the other candidate face images. Specifically, in the candidate face image 1, the average value of the similarity A and the similarity C is calculated as the mutual similarity 1. In the candidate face image 2, the average value of the similarity B and the similarity A is calculated as the mutual similarity 2. In the candidate face image 3, the average value of the similarity C and the similarity B is calculated as the mutual similarity 3. That is, “mutual similarity 1 = (similarity A + similarity C) / 2” “mutual similarity 2 = (similarity B + similarity A) / 2” “mutual similarity 3 = (similarity C + similarity B)” / 2 ", the mutual similarity of each candidate face image is calculated, and the process ends.

  Here, since the candidate face image 104b having the highest mutual similarity has the highest degree of similarity to any of the plurality of candidate face images, the possibility of the person's face image being the highest among the plurality of candidate face images 104b. Is a candidate face image 104b having a high value.

  In the operation of the access control system, the candidate face image 104b by the correct person acceptance by the person becomes the candidate face image 104b overwhelmingly more frequently than the frequency in which the candidate face image 104b is misunderstood by the misrepresentation by others or acceptance by others. Therefore, the candidate face image 104b of the person has the highest similarity with the other candidate face images 104b among the plurality of candidate face images 104b.

  The output unit 108 outputs an unlock signal or a lock signal to the electric lock 160 that is an external device connected to the face image matching device 100 when the determination unit 1062 determines that the person is the person. It is.

  Although the display unit 109 is realized by an LED in this embodiment mode, the display unit 109 may be shared with the numeric keypad 151 by using a liquid crystal display device or a touch panel.

  The control unit 101 is realized by a CPU, a DSP, a memory, and the like, and controls each unit of the face image matching device 100 according to various programs stored in the storage unit 104 and the like.

  The collation operation and face image update operation of the access control system 10 configured as described above will be described with reference to the flowcharts shown in FIGS. In the following description, as an initial setting, an administrator who has special authority for the face image matching device 100 has photographed each user's face, and the registered face image 104a has been stored in the storage unit 104. It is assumed that the ID number has been set. Three registered face images 104a are prepared for each person. A slightly upward face image is registered as a registered face image 1, a front face image is registered as a registered face image 2, and a slightly downward face image is registered as a registered face. Assume that image 3 is stored. The candidate face image 104b is a copy of the registered face image 104a associated with it in the initial state and stored.

FIG. 2 is a flowchart showing the entire verification operation and update operation according to the present embodiment.
In S <b> 100, when the user operates the numeric keypad 151 to select his / her ID number, the ID number is input from the selection input unit 103. The control unit 101 determines whether or not the input ID number matches the ID number stored in the storage unit 104. If they match, the process proceeds to step S102. On the other hand, although not shown, if a matching ID number is not stored, the process ends without performing a verification operation.

  In step S <b> 102, the control unit 101 controls the input unit 102 to acquire an input image including the user's face image captured by the camera 150.

  In S <b> 104, the control unit 101 reads one image from the plurality of registered face images 104 a stored in the storage unit 104 and sets it as a target for collation processing in the next step.

In S <b> 106, the collation unit 106 performs collation processing to determine whether or not the user is the user.
First, the face image extraction unit 1051 of the feature extraction unit 105 extracts a face area from the input image, generates an input face image, and temporarily stores it in the storage unit 104.
Next, the feature amount calculation unit 1052 of the feature extraction unit 105 calculates the feature amounts of the registered face image 104a and the input face image read in S104 or S110.
In the present embodiment, the registered face image is stored in the storage unit 104 and the feature amount is calculated each time collation is performed. However, when the registered face image 104a stores the feature amount instead of the image itself, Only for input face images.
In addition, when the feature amount of the input face image has already been calculated through S108 and S110 described later, it is not necessary to repeat the same processing.
Next, the similarity calculation unit 1061 of the matching unit 106 calculates the similarity from the feature amounts of the input face image and the registered face image 104a. This similarity is the person similarity. The identity similarity is temporarily stored in the storage unit 104.

In S <b> 108, the control unit 101 corresponds to the ID number input in S <b> 100, and determines whether or not the identity similarity calculation processing has been completed for all the registered face images 104 a stored in the storage unit 104. Investigate. If there is a registered face image 104a that is not subject to the processing in S106, the registered face image is read in S110, and the processing in S106 is repeated.
If the identity similarity calculation processing has been completed for all the registered face images 104a, the process proceeds to S112.

In S112, the determination unit 1062 compares the identity determination threshold stored in the storage unit 104 with all the identity similarities, and if any identity similarity is greater than the match threshold, the verification succeeds (uses). The process proceeds to S116.
If any person similarity is equal to or lower than the collation threshold, collation has failed (the user cannot be confirmed as the authorized person), and the process proceeds to S114.

  Here, the collation threshold value depends on the definition method of the feature amount, and is determined empirically. For example, after conducting experiments using a large number of face images in advance and investigating the relationship between the threshold, the acceptance rate of others, and the rejection rate of the person, it is determined so as to ensure appropriate matching accuracy according to the use of the face matching device .

  In S <b> 114, the control unit 101 outputs a collation failure message to the display unit 109 and waits for the input of the ID number in S <b> 100. Of course, in such a case, a signal for locking or unlocking the electric lock 160 is not output.

  In S <b> 116, the control unit 101 outputs a locking or unlocking signal to the electric lock 160 connected from the output unit 108, and displays a successful verification on the display unit 109.

In S <b> 118, the registered face image selection unit 1073 selects the largest registered face image 104 a from the personal similarity stored temporarily in the storage unit 104. This is a registered face image M.
The registered face image M is a registered face image closest to the state of the input face image among a plurality of registered face images 104 a prepared for each user of the access control system 10.
For example, when the input face image is acquired from the front, the registered face image 2 is selected as the registered face image M.

  Here, the registered face image 104a having the maximum identity similarity is the one most similar to the input face image among the registered face images having different shooting conditions. Therefore, selecting the registered face image M means selecting the registered face image 104a that has the same shooting conditions as the input face image.

  In S120, the candidate face image update unit 1072 of the update unit 107 performs the candidate face image update process shown in FIG. 3 on the candidate face image associated with the registered face image M selected in S116. The reason why the registered face image M is not directly updated with the input face image is that the registered face image M may not necessarily show only the difference in the photographing conditions in the difference in the identity similarity. Therefore, it is determined whether the candidate face image 104b is suitable for updating using the candidate face image 104b that is expected to have the same shooting conditions as the registered face image M.

  In S122, a registered face image update process is performed. Such processing will be described with reference to FIG. FIG. 5 is a flowchart relating to the processing. Each of the registered deterioration counters has 0 as an initial value.

  In S300, the candidate similarity 1 calculated when updating the candidate face image in S200 is compared with the identity similarity corresponding to the registered face image M. The identity similarity is the similarity between the registered face image M and the input face image, and the candidate similarity 1 is the similarity between the plurality of candidate face images 104b having the highest mutual similarity and the input face image. For this reason, the state that the identity similarity is lower than the candidate similarity 1 indicates that the candidate face image 104b is more similar to the input face image than the registered face image M. Therefore, if such a state continues, it is better to replace the registered face image M with the candidate face image 104b.

  Therefore, when the identity similarity corresponding to the registered face image M is smaller than the candidate similarity 1, 1 is added to the value of the registration deterioration counter (S302), and the process proceeds to the next S304. On the other hand, if the identity similarity corresponding to the registered face image M is equal to or greater than the candidate similarity 1, the registration deterioration counter is reset to 0 (S308), and the registered face image update process (S122) is terminated.

  In S304, the control unit 101 determines whether or not the registration deterioration counter has counted a predetermined number. This predetermined number is determined empirically. If a large number is set, the response to deterioration of the registered face image 104a is delayed. If a small number is set, the response to the deterioration is accelerated, but the risk of updating the face image of another person due to frequent updates of the registered face image 104a increases. . The predetermined number is determined empirically considering these.

  If it is determined in S304 that the predetermined number has not been reached, any registered face image 104a is not updated, and the registered face image update process (S122) is terminated. On the other hand, if the predetermined number has been reached, the process proceeds to S306.

  In S306, the registered face image update unit 1071 performs a process of updating the registered face image M. That is, the registered face image updating unit 1071 described above performs processing. In S308, the registration deterioration counter is cleared to 0, and the registered face image update process (S122) ends. Returning to FIG. 2, the process returns to S100 and waits for the operation of the numeric keypad 151 again.

The method for updating the candidate face image using the similarity between the candidate face images 104b has been described above, but the present invention is not limited to this.
The magnitude relationship between candidate similarity 1, candidate similarity 2, and candidate similarity 3 is examined, and an intermediate candidate face image 104b that is neither the largest nor the smallest is selected. For example, if candidate similarity 1> candidate similarity 2> candidate similarity 3, candidate face image 2 is selected because candidate similarity 2 is intermediate. Here, the candidate face image having an intermediate candidate similarity is selected when only one candidate face image is mixed in the plurality of candidate face images 104b and the current input face image is the same other person. The similarity with the candidate face image of the other person is the highest. In addition, if the current input face image is the face image of the person himself, the degree of similarity is the smallest with the candidate face image of another person. Therefore, the possibility of another person's candidate face image is the lowest because it is a candidate face image exhibiting an intermediate candidate similarity.

  Next, it is determined whether the intermediate candidate similarity 2 is larger than the update threshold. When the candidate similarity 2 is larger than the update threshold, it is determined that the input face image is appropriate for updating the candidate face image 104b. When the candidate similarity is small, it is determined that the input face image is inappropriate for updating the candidate face image.

  The update threshold depends on the similarity calculation method, collation accuracy, and the like. Since the candidate face image 104b may be updated to the registered face image 104a and it is necessary to eliminate the possibility of others as much as possible, it is better to set the update threshold value higher. On the other hand, if a too high update threshold is set, it becomes impossible to follow the face change of the person due to aging or the like. Therefore, the update threshold is determined empirically while taking these factors into consideration.

  Finally, the candidate face image 104b that exhibits the smallest similarity among the candidate similarity 1, the candidate similarity 2, and the candidate similarity 3 is updated with the input face image. Here, the reason why the candidate face image 104b that exhibits the smallest similarity to the input face image is selected is that when there is an input face image of the user, the candidate face image of another person has the smallest similarity. .

  In the present embodiment, since there are three candidate face images 104b, the candidate face image 104b having the second highest similarity was adopted. However, for example, nine candidate face images 104b are used, and up to two of the face images of other people are used. If it is possible to use a face image, the similarity between any of the third to seventh positions may be adopted and compared with the update threshold.

  In this embodiment, the candidate face image 104b is determined to be updated every time collation is performed, and the processing target is the entire candidate face image 104b. Once the candidate face image 104b is updated, the update is performed for a certain period. Only the candidate face image 104b that has been updated may be allowed to be updated, and the other candidate face images 104b may not be updated. If the user performs the operation of the access control system 10 before the fixed period has passed, the candidate face image that is the face image of the other person even if the candidate face image 104b is updated with a malicious face image of the other person. Since 104b is updated, the risk that the registered face image 104a is erroneously updated with another person's face image can be reduced.

  Therefore, in a face authentication apparatus that has prepared a plurality of registered face images to cope with face variations, the registered face image having the variation closest to the input face image is automatically selected and updated. Since each registered face image can be kept up-to-date while keeping the fluctuation included in as much as possible, it is possible to cope with the face fluctuation due to aging and the like, and to prevent the collation success rate from being lowered.

The block diagram which shows the structure of the whole access control system provided with the face image collation apparatus concerning invention Flow chart showing overall verification operation and update operation according to the present embodiment The flowchart which shows the process of the candidate face image update means concerning this embodiment Conceptual diagram showing mutual similarity of candidate face images Flow chart showing registered face image update processing according to the present embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Access control system 150 ... Camera 151 ... Numeric keypad 100 ... Face image collation apparatus 101 ... Control part 102 ... Input part 103 ... Selection input part 104 ... Memory | storage part 105 ... Feature extraction unit 106 ... Verification unit 107 ... Update unit 108 ... Output unit 109 ... Display unit 160 ... Electric lock

Claims (5)

A face image collation device that collates a plurality of registered face images stored in advance and an input face image to perform identity verification,
An input face image acquisition unit that acquires an input face image based on an input image obtained by capturing the face of the collator;
A plurality of registered face images obtained by photographing the face of the same person under different shooting conditions; and a storage unit storing candidate face images used for updating for each of the plurality of registered face images;
A similarity that is the degree of similarity between the input face image acquired by the input face image acquisition unit and the plurality of registered face images is calculated, and it is determined that the person is the person when the similarity exceeds a matching threshold. A matching unit to
An output unit for outputting the collation result of the collation unit to the outside;
The similarity with respect to the input face image for each of the plurality of registered face images calculated by the matching unit is compared, the registered face image with the highest similarity is selected, and the candidate face image corresponding to the selected registered face image is selected. A candidate face image update unit that updates the input face image;
A registered face image update unit for updating a corresponding registered face image in the candidate face image;
A face image matching device characterized by comprising: The storage unit stores three or more candidate face images for one registered face image,
The candidate face image update unit calculates a mutual similarity indicating a degree of similarity between the candidate face images, and calculates a similarity between the candidate face image having the largest mutual similarity and the input face image The face image matching device according to claim 1, wherein the candidate face image is updated using the input face image only when the similarity exceeds the update threshold. The storage unit stores three or more candidate face images for one registered face image,
The candidate face image update unit calculates a candidate similarity indicating the degree of similarity between the candidate face image and the input face image, and an intermediate candidate similarity with which the candidate similarity is neither maximum nor minimum is an update threshold. The face image matching device according to claim 1, wherein the candidate face image stored in the storage unit is updated using the input face image when the input face image is exceeded.   The registered face image update unit determines a deterioration state of the registered face image, and updates the registered face image with the candidate face image corresponding to the registered face image when it is determined that the registered face image is deteriorated. The face image collation device according to claim 1. The face image collation apparatus according to claim 1, wherein the candidate face image update unit updates the candidate face image only when it is determined that the collation unit is the person himself / herself.

Images