JP5808974B2 - Face image authentication device - Google Patents

Description

The present invention relates to a face image authentication apparatus, and more particularly to a face image authentication apparatus that detects a user who should update or additionally register a face image registered in advance.

In a face image authentication device that authenticates a person by collating a face image, an input face image that is a portion of a monitoring image obtained by capturing a monitoring area with a camera or the like and that includes a face of a person to be collated is registered in advance. It is determined whether or not the input face image is that of the registered person by collating with the registered face image in which the face of the registered person is reflected. However, if the input face image differs greatly from the registered face image due to differences in shooting conditions, aging of the registered person, hairstyle or makeup, etc., and authentication fails even though the input face image belongs to the registered person In some cases, there are registered persons who fail with each authentication and are less likely to succeed. Such a registered person generally reports to the administrator that the authentication is likely to fail and is inconvenient, and the administrator who received the report inputs that person was photographed from the authentication history of the face image authentication device. A face image is found and the registered face image is replaced with the input face image. However, a state in which the authentication is likely to fail continues until a person who is likely to fail the authentication reports that fact. In addition, the task of searching for a person who is likely to fail authentication from the authentication history of the face image authentication apparatus is time-consuming for the administrator.

Therefore, Patent Document 1 proposes a face image matching device that updates a registered face image, which is a reference for person verification, with an input face image that is determined to be the person at the time of verification. This face image matching device stores a candidate face image that is an input face image determined to be the person at the time of matching, and the candidate face image is more likely to be the input face image than the registered face image in the subsequent matching process. If the state determined to be similar continues, the registered face image is updated with the candidate face image.

JP 2007-304763 A

The face image matching device described in Patent Document 1 is registered when the candidate face image determined to be the person at the time of matching continues to be determined to be similar to the input face image in the subsequent matching process. Since the face image is updated, it is possible to eliminate a state in which the verification is likely to fail while reducing the possibility that the registered face image is erroneously updated with the face image of a person other than the registrant.
However, in this face image collation device, in order to determine that the registered face image should be updated, it is necessary to collate the person multiple times, so that it is possible to detect a person who is difficult to authenticate earlier. There is a demand for a face image authentication apparatus capable of performing the above.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to detect a person who is difficult to be authenticated and frequently fails in authentication among pre-registered persons at an earlier stage and with higher accuracy, and is used for updating a registered face image suitable for authentication. It is an object of the present invention to provide a face image authentication apparatus that promptly selects an image.

The present invention for solving such a problem includes an imaging unit that sequentially acquires a monitoring image obtained by photographing a person in a monitoring region, a storage unit that previously stores a registered face image that is a registered person's face image, and a monitoring image. Using a collation unit that performs person authentication, and the collation unit collates the face area image with the registered face image, and a face detection unit that extracts an image of an area including a person's face from the monitoring image as a face area image; A face matching unit that authenticates whether or not the person in the face area image is a registered person, and a predetermined request that the face matching means authenticates that the person in the face area image is not a registered person and that the person is to be authenticated And determining means for determining the person as a person who is difficult to authenticate on the condition that an action has been taken.

In the face image authentication device according to the present invention, the collation unit further includes a stationary detection unit that detects that the acquired facial region image is substantially stationary, and the determination unit includes at least the facial region image being substantially stationary. It is preferable that the predetermined desired behavior is that the current time is equal to or longer than the predetermined time.

Further, in the face image authentication device according to the present invention, the collation unit further includes face tracking means for tracking the face area image of the same person in the sequentially acquired monitoring images, and the stillness detecting means is the tracking position of the face area image. It is preferable to detect that is substantially stationary.

In the face image authentication device according to the present invention, the collation unit further includes a face direction detection unit that detects the face direction of the person, and the determination unit includes at least a face reflected in the face area image with respect to the imaging unit. It is preferable that the predetermined desired behavior is that the number of face area images facing substantially in the front direction is a predetermined number or more.

Further, in the face image authentication apparatus according to the present invention, the collation unit further includes a history management unit that associates the face area image with the authentication result of the face collation unit and stores the collation history in the storage unit, and the determination unit includes: It is preferable to refer to the verification history and add it on condition that the authentication result that the same person is not a registered person is equal to or higher than a history threshold.

Further, in the face image authentication apparatus according to the present invention, the collation unit further includes a history management unit that associates the face area image with the authentication result of the face collation unit and stores the collation history in the storage unit, and the determination unit includes: It is preferable to refer to the verification history and add it on the condition that the authentication result that the same person is not a registered person included in the predetermined history reference period tends to increase.

The face image authentication apparatus according to the present invention further includes a display unit and a registration unit that re-registers the registered face image with the face area image by an external instruction input, the registration unit including the registered face image, the registration It is preferable to display face area images side by side on the display unit in descending order of the number of authentication results indicating that the person is not a person, and re-register the registered face image displayed when receiving an instruction input with the face area image.

The face image authentication apparatus according to the present invention detects a person who is difficult to be authenticated among registered persons in advance and often fails in authentication more quickly and accurately, and updates a registered face image suitable for authentication. There is an effect that the face image to be used can be selected quickly.
Hereinafter, in this specification, a person who often fails in authentication will be referred to as a “person with difficulty in authentication”.

It is a schematic block diagram of the face image authentication apparatus to which this invention is applied. It is a schematic diagram showing the installation example of an imaging part in case a face image authentication apparatus is installed in the entrance of an office building. It is a schematic diagram of a collation table. It is a schematic diagram of the log | history table which is a part of collation table. It is a schematic diagram of the update screen of a registered face image. It is a main flowchart which shows the operation | movement of a determination process of a difficult-to-authenticate person in the face image authentication apparatus to which this invention is applied. It is a subflowchart which similarly shows the operation | movement of a determination process of a person with difficult authentication.

Hereinafter, an embodiment of a face image authentication apparatus according to the present invention will be described with reference to the drawings.
A person who requests authentication from the face image authentication apparatus tends to take a characteristic action such as stopping or slowly walking while directly looking at the camera when the person is not authenticated. Therefore, the face image authentication apparatus to which the present invention is applied sequentially acquires the monitoring images obtained by capturing the monitoring area, and tracks the same person over the plurality of sequentially acquired monitoring images. Then, the face image authentication device determines whether the person being tracked is taking a characteristic action to be authenticated, such as stopping or walking slowly while looking directly at the camera, and then refers to the authentication history. Thus, a person who often fails in authentication is detected as a person who has difficulty in authentication.
As a result, the face image authentication device detects a person who has difficulty in authentication at an early stage and with high accuracy, and promptly selects a face image suitable for authentication and used for updating the registered face image.

FIG. 1 is a diagram showing a schematic configuration of a face image authentication apparatus 10 to which the present invention is applied. As illustrated in FIG. 1, the face image authentication device 10 includes an imaging unit 100, an output unit 200, a display unit 300, an input unit 400, and an image processing unit 500. Hereinafter, each part of the face image authentication device 10 will be described in detail.

The imaging unit 100 is a monitoring camera that captures a predetermined monitoring area, and is, for example, a photoelectric conversion element (for example, a CCD sensor, a C-MOS, or the like) that is arranged two-dimensionally and outputs an electrical signal corresponding to the amount of received light. And an imaging optical system for forming an image of the monitoring region on the photoelectric conversion element.
The imaging unit 100 is installed so that the face of a person who passes through the monitoring area (a person in the monitoring area to be collated by the face image authentication apparatus 10 is referred to as a target person) can be sequentially photographed. Then, the imaging unit 100 acquires a monitoring image obtained by capturing the monitoring area at predetermined time intervals (for example, 200 milliseconds). The imaging unit 100 is connected to the image processing unit 500 and passes the acquired monitoring image to the image processing unit 500.

FIG. 2 schematically shows an installation example of the imaging unit 100 when the face image authentication device 10 is installed at the entrance of an office building. As shown in FIG. 2, for example, when the face image authentication device 10 is installed at the entrance 253 of an office building, the imaging unit 100 includes a wall on which the entrance 253 is installed so as to include a passage leading to the entrance 253 in the monitoring area. It is attached to the top or ceiling of the camera with the shooting direction slightly downward and toward the passage.
As a result, the imaging unit 100 can capture the target person heading toward the entrance 253 (heading in the traveling direction 254) at a predetermined time interval. FIG. 2 shows a state in which the imaging unit 100 sequentially captures the same target person 250, 251, 252 heading toward the entrance 253 at times t, t + 1, t + 2.

The monitoring image can be a grayscale or color multi-tone image. In the present embodiment, the monitoring image is a color image having horizontal 340 pixels × vertical 240 pixels and an 8-bit luminance resolution for each of the RGB colors. However, a monitor image having a resolution and gradation other than this embodiment may be used.

The output unit 200 includes a communication interface connected to, for example, an electric lock or an external device that controls the electric lock, and a control circuit thereof. When the output unit 200 receives a signal indicating successful authentication of the target person from the image processing unit 500, the output unit 200 outputs a signal requesting, for example, unlocking of the electric lock to the connected device.

The display unit 300 is connected to the image processing unit 500 and includes a display device such as a CRT, a liquid crystal display, or an organic EL display. The display unit 300 displays a screen in response to a request from the image processing unit 500.
The face image authentication device 10 may further include a speaker or a vibration device (not shown) that sounds a sound or generates a vibration in response to a request from the image processing unit 500.

The input unit 400 is an input interface configured by an input device such as a keyboard and a mouse, receives a switching operation from an administrator, and outputs a signal corresponding to the operation to the registration unit 529 of the image processing unit 500. Further, the display unit 300 and the input unit 400 may be integrated by configuring the display unit 300 with a touch panel display.

The image processing unit 500 is configured by, for example, a so-called computer. The image processing unit 500 authenticates the target person based on the monitoring image received from the imaging unit 100, determines that the person is difficult to be authenticated, and detects a person who is difficult to authenticate by referring to the history.
For this purpose, the image processing unit 500 includes a storage unit 510 and a collation unit 520. Furthermore, the collation unit 520 includes a face detection unit 521, a face tracking unit 522, a face collation unit 523, a behavior detection unit 524, a history management unit 527, a determination unit 528, and a registration unit 529.

The storage unit 510 includes a semiconductor memory such as a ROM and a RAM, or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 510 stores in advance a computer program and various parameters for controlling the face image authentication apparatus 10. The storage unit 510 stores a registered face image used for verification in association with a registered person ID of a corresponding registered person (hereinafter referred to as a registered person). The storage unit 510 also stores a collation table and a history table for managing information related to the target person generated by the image processing. Details of the collation table and history table will be described later.

Each means of the collation unit 520 is a functional module implemented by a microprocessor, a memory, a peripheral circuit thereof, and software operating on the microprocessor. Alternatively, these means may be integrated by firmware. Further, some or all of these means may be constituted by independent electronic circuits, firmware, a microprocessor, and the like. Hereinafter, each means of the collation part 520 is demonstrated in detail.

Each time the face detection unit 521 receives a monitoring image from the imaging unit 100, the face detection unit 521 extracts a face area that is an area in which the face of the target person is captured from the received monitoring image, and extracts a face area image. In order to extract a face area, the face detection unit 521 uses, for example, an inter-frame difference process or a background difference process, and changes in luminance values with time in a plurality of monitoring images acquired by the imaging unit 100 are performed. Extract regions. Then, the face detection unit 521 extracts, from the extracted change areas, a change area that seems to be a person from a feature amount such as the size of the change area as a person area. Then, the face detection unit 521 performs edge pixel extraction on the extracted person region using a Sobel filter or the like so that the inclination direction of the luminance change can be known. Then, the face detection unit 521 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels, and uses the area surrounded by the edge distribution as a face area. Extract. In this case, the face detection unit 521 can detect an elliptical edge distribution using, for example, a generalized Hough transform.

Alternatively, the face detection unit 521 may detect a face area using an Adaboost classifier. This method is described in a paper by P. Viola and M. Jones “Rapid Object Detection Using a Boosted Cascade of Simple Features” (Proc. The IEEE International Conference on Computer Vision and
Pattern Recognition, vol.1, pp.511-518, 2001) can be referred to.
Then, the face detection unit 521 cuts out the extracted face area from the monitoring image to create a face area image, and outputs the face area image and the coordinate information of the face area in the monitoring image to the face tracking unit 522.

The face tracking unit 522 performs a tracking process using a known tracking technique on the face area extracted from the face detection unit 521 over a plurality of monitoring images continuously acquired at predetermined time intervals, and the same person The face area image is tracked by associating the face areas in which the face is reflected.
For example, the face tracking unit 522 includes the position of the center of gravity of the face area extracted from the latest monitoring image (hereinafter referred to as the face area of the current frame) and the face area extracted from the monitoring image one frame before (hereinafter referred to as the previous area). The distance of the center of gravity position (referred to as a face area of the frame) is obtained, and when the distance is equal to or less than a predetermined threshold, the face area is associated with the same person.

The face area in the monitoring image before and after the movement when the target person moves a certain distance when moving away from the imaging unit 100 and when the target person moves the same distance when moving closer to the imaging unit 100. The position difference is different. Therefore, for example, by setting a predetermined threshold as the size of the face area, it is appropriately determined whether the face area of the current frame and the face area of the previous frame are from the same person regardless of the position of the target person in the monitoring area. Can be evaluated. When a plurality of face areas are extracted, it is checked whether or not face areas with the closest distance between the centroid positions correspond to each other.
Alternatively, the face tracking unit 522 may perform face area tracking processing using a method such as an optical flow or a particle filter.

The face tracking unit 522 updates the collation table stored in the storage unit 510 when the face areas are associated. FIG. 3 shows an example of the collation table. As shown in FIG. 3, the collation table 310 manages collation data for each target person.
The collation data is a set of data represented in each row of the collation table 310, and the face tracking unit 522 includes a trial number 311, a target person ID 312, a start time 313, a tracking flag 314, face image data 315, and the like. The tracking position information 316 is updated.

In the verification table 310 shown in FIG. 3, the trial number 311 is an identification number for identifying the verification data of the target person being tracked from other verification data, and while the target person exists in the monitoring area. In other words, the same identification number continues to be assigned while being tracked by the face tracking means 522.
The target person ID 312 is an identification number for identifying the target person being tracked from other target persons, and corresponds to the trial number 311 on a one-to-one basis. The target person ID 312 is assigned so as to be uniquely determined. For example, a random number may be generated based on the time when the face tracking unit 522 starts tracking. Alternatively, the target person ID 312 and the trial number 311 have a one-to-one correspondence and may be omitted.
The start time 313 represents the time when the face detection unit 521 first extracts a face area image for the target person and the face tracking unit 522 starts the tracking process.

The tracking flag 314 is a flag indicating whether or not the tracking of the target person is continued. The tracking flag 314 is turned on when the face tracking unit 522 starts tracking the target person, and turned off when the tracking is finished. That is, when the tracking flag 314 is ON, the target person exists in the monitoring area, and when the tracking flag 314 is OFF, the target person has already been authenticated and has entered the room or left without being authenticated. It will be that.

The face image data 315 is data of all face area images created by the face detection unit 521 and associated as the target person in the tracking process by the face tracking unit 522.
The tracking position information 316 is time information indicating the coordinate information of the face area in the monitoring image from which each face image data stored as the face image data 315 is cut out and the time when the tracking process is performed.

When the face tracking unit 522 cannot associate the face area of the current frame of interest with the face area of the previous frame, the face person of the new frame is reflected in the face area of the current frame. Assuming that the target person is newly added to the verification table 310, initialization processing is performed. That is, the face tracking unit 522 assigns a new trial number 311 and a target person ID 312 to the verification data, records the current time as the start time 313, and sets the tracking flag 314 to ON. The face tracking unit 522 records a face area image created from the face area of the current frame as the face image data 315 and also records coordinate information and the current time of the face area as tracking position information 316.
Further, the face tracking means 522 sets the authentication flag 317 to OFF and sets the desired action counter 318 to 0. Then, the face tracking unit 522 outputs the newly added matching data to the face matching unit 523.

On the other hand, when the face tracking unit 522 can associate the face area of the current frame of interest with the face area of the previous frame, collation data for the target person has already been created in the collation table 310. Therefore, the face tracking unit 522 adds the face area image created from the face area of the current frame to the face image data 315 of the corresponding matching data in the matching table 310, and also adds the face area image of the face area to the tracking position information 316. An update process for adding coordinate information and the current time is performed. Then, the face tracking unit 522 outputs the matching data to the face matching unit 523.

In addition, when there is a face area of the previous frame that is not associated with all face areas of the current frame, the face tracking unit 522 sets the tracking flag 314 of the corresponding matching data in the matching table 310 to OFF. Then, the tracking process for the target person is terminated.

The face matching unit 523 uses the latest face area image of the matching data in which the authentication flag 317 of the matching table 310 illustrated in FIG. 3 is OFF among the matching data output from the face tracking unit 522 and the storage unit 510. Each registered face image that has been read is checked to determine whether it is from the same person. If the face collating means 523 determines that the latest face area image of the collation data and the registered face image are from the same person, the face collation means 523 sets the collation data authentication flag 317 to ON. That is, the authentication flag 317 is a flag indicating whether or not the authentication for the target person has succeeded.

The face matching unit 523 can use various known matching methods as the matching process.
For example, the face matching unit 523 performs pattern matching between the face area image and the registered face image. The face matching means 523 calculates the sum of squares of the difference in luminance value between each pixel included in the face area image and the corresponding pixel of the registered face image while shifting the position of the face area image and the registered face image, and the calculated square sum The reciprocal of the normalized value obtained by dividing the smallest one by the number of pixels included in the face area image is obtained as the similarity.
When the highest similarity among the similarities obtained for each registered face image exceeds a predetermined authentication threshold, the face matching unit 523 determines the target person shown in the face area image as the highest similarity value. It is determined that the person is a registered person registered based on the registered face image (successful authentication).
On the other hand, if any similarity does not exceed a predetermined reference value, the face matching unit 523 determines that the target person shown in the face area image is not a registered person (authentication failure). Note that this authentication threshold is appropriately determined according to the environment, purpose, and the like in which the face image authentication device 10 is installed.

When the face collating unit 523 determines that the target person is a registered person, the face collating unit 523 sets the authentication flag 317 to ON and outputs a signal indicating successful authentication to the output unit 200. Further, the face matching unit 523 outputs the matching data of the target person that has been subjected to the matching process to the behavior detecting unit 524.

The behavior detecting unit 524 calculates a behavior feature amount that represents a behavior (hereinafter, referred to as a requested behavior) that the behavior of the target person shown in the face area image requests authentication. Then, the behavior detecting unit 524 detects that the target person is in a state where it is difficult to authenticate when the behavior feature quantity satisfies a predetermined condition indicating that the target person is taking the desired behavior, and will be described later. Increment the requested action counter.

Therefore, the behavior detection unit 524 determines, as the behavior feature quantity, the degree that the target person faces the direction of the imaging unit 100, that is, the face reflected in the face area image of the current frame is the front direction with respect to the imaging unit 100. The degree of direct view indicating the degree of facing is calculated.
In addition, the behavior detection unit 524 associates the target person being tracked by the face tracking unit 522 as a behavior feature amount by the face tracking unit 522 in a monitoring area, that is, in a plurality of monitoring images. The degree of stillness indicating the degree of change in the position of each face area is calculated.

Then, the behavior detection unit 524 determines whether or not the target person shown in the face area image is taking the desired behavior based on the direct view degree and the static degree. For this purpose, the behavior detection unit 524 includes a face direction detection unit 525 and a stationary detection unit 526.

The face orientation detection unit 525 functions as a feature amount calculation unit that calculates a behavior feature amount, and calculates the direct visibility of the face reflected in the face area image of the current frame for the collation data output from the face collation unit 523.
An orthonormal coordinate system for defining the face orientation of the person is shown on the target person 252 at time t + 2 in FIG. In this orthonormal coordinate system (X, Y, Z), the origin O is set to the center of gravity of the face area image of the person whose face orientation is to be detected. The X axis passes through the origin O and is set as a vertical axis along the midline of the face. The Y-axis is set as a horizontal axis that passes through the origin O and crosses the face left and right. The Z axis is set as a horizontal axis in a direction crossing the face back and forth.
In this coordinate system, the face direction of the person is a state where the person looks directly at the imaging unit 100 (that is, the person faces the front of the face toward the imaging unit 100, and the Z axis is a straight line connecting the imaging unit 100 and the face. State) is expressed as a set (ψ, θ, φ) of yaw angle ψ, pitch angle θ, and roll angle φ when the face-to-face reference state is set as a reference. The yaw angle ψ represents a left-right rotation angle (that is, a rotation angle in the YZ plane with the X axis as the rotation center) with respect to the face direction of the person in the face-to-face state. The pitch angle θ represents a vertical rotation angle (that is, a rotation angle in the XZ plane with the Y axis as the rotation center) with respect to the face direction of the person in the face-to-face state. Further, the roll angle φ represents a clockwise rotation angle with the Z axis as the rotation center with respect to the face direction of the person in the face-to-face state. Hereinafter, the yaw angle ψ, the pitch angle θ, and the roll angle φ are expressed in radians, and the rightward direction, the downward direction, and the clockwise direction are positive.

The face orientation detection means first extracts face feature points that are characteristic parts of the face from the face area image. Then, the face orientation detection unit 525 calculates the type of the extracted face feature point and position information on the face area image (for example, a two-dimensional coordinate value with the upper left corner of the face area image as the origin). For example, the face orientation detection means 525 extracts facial feature points such as both eye corners, both eye region centers, nose tips, mouth points, and mouth corner points. The face orientation detection means 525 can use various known methods for extracting face feature points. For example, the face orientation detection unit 525 performs edge extraction processing on the face area image and extracts edge pixels having a large luminance difference from surrounding pixels. Then, the face direction detecting means 525 checks whether or not the feature amount obtained based on the position and pattern of the edge pixel satisfies a predetermined condition for a characteristic part of the face such as eyes, nose, and mouth. By specifying the position of each part, each facial feature point can be extracted.
In addition, instead of performing edge extraction processing to extract edge pixels by performing edge extraction processing, the face orientation detection means 525 performs Gabor conversion processing or wavelet conversion processing on the face area image, and has a large local change in a plurality of different spatial frequency bands. Pixels may be extracted. Further, the face orientation detection unit 525 may extract face feature points by performing template matching between a template corresponding to each part of the face and a face area image to identify the position of each part of the face.

The face orientation detection unit 525 detects the face orientation based on the positional relationship between the extracted face feature points and the corresponding face feature points in the three-dimensional model of the human face. In this case, a three-dimensional shape model (for example, represented by a wire frame model or a surface model) imitating the standard shape of a human head is prepared in advance and stored in the storage unit 510. In addition, the storage unit 510 also stores the positions of 3D face feature points such as both eye corners, nose tips, and mouth points of the three-dimensional shape model.
When the face orientation detection unit 525 extracts the facial feature points from the face area image, the face orientation detection unit 525 adjusts the face orientation of the three-dimensional shape model according to a predetermined rotation amount, translation amount, or enlargement / reduction ratio, and By virtually projecting onto a plane parallel to the image plane of the image optical system, the position of the 3D face feature point on that plane is obtained. Then, the face direction detection unit 525 obtains the sum of the positional deviation amounts between each face feature point extracted from the face area image and the corresponding feature point among the projected 3D face feature points. The face orientation detection unit 525 changes the rotation amount, the translation amount, or the enlargement / reduction ratio, repeats the above-described procedure, and obtains the face orientation of the three-dimensional shape model when the sum of the displacement amounts is minimized. Then, the face direction detecting means 525 can obtain the set (ψ, θ, φ) of the yaw angle ψ, pitch angle θ, and roll angle φ from the face direction of the three-dimensional shape model. Alternatively, the technique disclosed in Japanese Patent Application Laid-Open No. 2009-237993 by the applicant of the present application can also be adopted.

つまり直視度は、顔領域画像に写っている顔の向きが撮像部１００に対して正面方向に近いほど大きい値となる。例えば、対象人物が撮像部１００を直視している場合、ヨー角ψ及びピッチ角θはともに小さい値となるため、直視度は大きくなる。一方、撮像部１００に対して顔を背けている場合は、ヨー角ψ及びピッチ角θはともに小さい値となるため、直視度は大きくなる。
そして顔向き検出手段５２５は、直視度を算出すると、算出した直視度をその顔領域画像と対応付けて記憶部５１０に記憶しておく。 The direct visibility can be the reciprocal of the sum of the absolute values of the yaw angle ψ and the pitch angle θ, for example, as expressed by equation (1).

That is, the direct visibility becomes larger as the orientation of the face shown in the face area image is closer to the front direction with respect to the imaging unit 100. For example, when the target person is directly looking at the imaging unit 100, the yaw angle ψ and the pitch angle θ are both small values, and thus the degree of direct visibility increases. On the other hand, when the face is turned away from the imaging unit 100, the yaw angle ψ and the pitch angle θ are both small values, and the direct visibility increases.
Then, when the face orientation detection unit 525 calculates the direct visibility, it stores the calculated direct visibility in the storage unit 510 in association with the face area image.

つまり静止度は、同一人物の顔が写っている顔領域の監視画像内における位置の変化が小さいほど大きい値となる。例えば図２に示した例では、時刻tから時刻ｔ＋１の間に対象人物が移動した距離２５５よりも時刻ｔ＋１から時刻ｔ＋２の間に対象人物が移動した距離２５６の方が小さいため、時刻ｔ＋２における静止度は時刻ｔ＋１における静止度より大きくなる。 The stillness detection unit 526 also functions as a feature amount calculation unit that calculates a behavior feature amount, and uses the tracking position information 316 of the collation table 310 to calculate the degree of stillness for the face area associated with the face tracking unit 522. For example, a two-dimensional coordinate system is set in which the upper left end of the monitoring image is set as the origin, the x axis is set in the horizontal direction from left to right, and the y axis is set in the vertical direction from top to bottom. That is, in this coordinate system, when the imaging unit 100 is installed so as to capture the monitoring area from a slightly upward direction, the x axis is in the direction in which the target person moves from left to right as viewed from the imaging unit 100. The y axis is set in the direction in which the target person approaches along the shooting direction. In this coordinate system, the center of gravity of the face area at time t is defined as (x _t , y _t ). In that case, the stillness of the face area at time t is perpendicular to the horizontal difference between the centroid position of the face area of the current frame relative to the centroid position of the face area of the previous frame, for example, as expressed by equation (2). It can be the reciprocal of the sum of absolute values of the direction differences.

That is, the degree of staticity increases as the change in the position of the face area in which the face of the same person is captured in the monitoring image is smaller. For example, in the example illustrated in FIG. 2, the distance 256 that the target person moved from time t + 1 to time t + 2 is smaller than the distance 255 that the target person moved from time t to time t + 1. The stillness is greater than the staticity at time t + 1.

Then, the stillness detection means 527 calculates the stillness, and stores the calculated stillness in the storage unit 510 in association with the face area image.
Note that when the imaging conditions such as the height of the imaging unit 100 from the floor, the depression angle, and the focal length are already known, the actual distance between the imaging unit 100 and the target person can be roughly calculated, so the stationary detection unit 526 May calculate the reciprocal of the absolute value of the time change of the distance as the degree of stasis according to the equation (2).

The history management unit 527 records the similarity calculated by the face matching unit 523 and the authentication result determined by the same unit in the storage unit 510.
FIG. 4 shows an example of a table storing similarity. This table is a part of the collation table shown in FIG. 3, but will be referred to as a history table for convenience of explanation. Each row of the table shown in FIGS. 3 and 4 is a matching history recorded by one trial.
The history management means 527 manages history data obtained by collation for each collation data shown in FIG.
The history data is a set of data represented in each row of the history table 410. The target person ID 312, face image data 315, desired action counter 318, group number 321, authentication difficult person flag 322, authentication time 323, similarity 412 and An authentication result 416 is included. The history table 410 also manages a registered person ID 413 for identifying the registered person from other registered persons and a registered face image 414 representing the face image of the registered person, and the similarity 412 for each registered person ID 413. Managed.

Note that only one face image data 315 in FIG. 4 is shown for easy understanding.
The similarity 412 is a similarity calculated by the face matching unit 523 as a result of matching the face area image in which the target person is captured with each registered face image 414.
The registered person ID 413 is an ID for identifying the registered person from other registered persons. The registered person ID 413 may be a continuous positive integer, but may be anything as long as it can uniquely identify the registered person, such as an employee number. The registered face image 414 is a face image of a registered person that has been registered in advance or additionally registered. There may be one or more registered face images for each registered person. When a plurality of registered face images 414 exist for one registered person, it is preferable to assign a registered person ID 413 to each registered face image 414. In FIG. 4, an image representing a face is shown as a registered face image 414 for easy understanding. However, it is stored in a data format necessary for the face matching unit 523 to perform face image matching processing. May be. For example, when the face matching unit 523 performs the matching process only by comparing predetermined face feature points, the storage capacity can be saved by storing only the information of the face feature points.

When the face matching unit 523 determines that the authentication is successful, the history management unit 527 records the similarity in the frame that is determined to be successful for the registered face image 414 that is determined to be successful, and does not determine that the authentication is successful. For the registered face image 414, the similarity calculated for each registered face image 414 in the same frame is recorded. Further, the fact that the target person at that time was successfully authenticated and the registered person ID 413 determined to be successful are recorded in the authentication result 416.
On the other hand, the history management unit 527 records the highest similarity for the registered face image 414 for which the highest similarity was calculated for the target person that the face matching unit 523 did not finally determine as successful authentication. For the other registered face images 414, the similarity calculated for each registered face image 414 in the frame for which the highest similarity is calculated is recorded.
Alternatively, for the registered face image 414 that the face collating unit 523 did not determine that the authentication was successful, the history management unit 527 calculated all the similarities calculated for the face area image and the registered face image 414 representing the target person. Or an average value of a recent predetermined number of similarities (for example, the similarity calculated in the last 5 seconds) may be recorded as the similarity 412. As a result, the degree of similarity between the face area image showing the target person and the registered face image can be determined with higher accuracy.

Note that the similarity 412 is determined until the target person is authenticated as a registered person by the face matching unit 523, or until the target person leaves the monitoring area without being authenticated as a registered person, or the face image authentication device 10 is calculated until the authentication process for the target person is completed due to a timeout or the like.

Further, the history management unit 527 selects a face image most suitable for updating the registered face image based on the direct visibility calculated by the face direction detection unit 525 and the stillness calculated by the stillness detection unit 526, and The face image 319 is recorded in the collation table 310.

The action detection unit 524 of the present embodiment determines whether or not the target person in the monitoring area stops or walks slowly with the face facing the imaging unit 100 within the authentication time.
For this purpose, the behavior detection unit 524 compares the direct visibility calculated by the face direction detection unit 525 with the direct visibility threshold value, and compares the static degree calculated by the stillness detection unit 526 with the static degree threshold value. The direct visibility threshold is determined so that it can be determined that the direction of the face of the target person shown in the face area image is in an angle range where it can be considered that the imaging unit 100 is directly viewed, for example, the yaw angle ψ and the pitch When the angle θ is expressed in radians, 4 is preferable. The stillness threshold value is determined so that it is possible to determine that the target person shown in the face area image is substantially stopped. For example, when processing a monitoring image of 5 frames per second, the width of the face area image is determined. When W is W, 1 / (W × 0.5) is preferable.

The behavior detection unit 524 increments the desired behavior counter when the direct visibility is equal to or greater than the direct visibility threshold and the stillness is equal to or greater than the staticity threshold. This desired behavior counter is a counter that indicates the number of times that the target person is determined to be taking the desired behavior, and is recorded in the matching table 310 shown in FIG. 3 (that is, also in the history table 410 shown in FIG. 4). The
On the other hand, when the direct visibility is less than the direct visibility threshold, or when the stillness is less than the staticity threshold, the target person is considered to be facing away from the imaging unit 100 or moving quickly and authenticated. There is a high possibility that it is not a psychological state of wanting. Therefore, in that case, the behavior detection means 524 does not update the desired behavior counter.

The history management unit 527 is not authenticated until the face matching unit 523 determines that the matching is successful until the face tracking unit 522 starts tracking the face area image of the target person or until the target person is a registered person. The authentication time is calculated until the face image authentication device 10 is away from the monitoring region or until the authentication process for the target person is completed due to a timeout or the like.

The time when the face tracking unit 522 starts tracking the face area image of the target person is stored in the start time 313 of the corresponding matching data in the matching table 310. The time when the face matching unit 523 determines that the matching is successful is the time of the tracking position information 316 stored in the matching table 310 in association with the face area image used for the determination of the matching as the face image data 315. The latest one. Therefore, the history management unit 527 calculates the authentication time from the difference between the two.

Note that the authentication time to be timed out is the time when the authentication processing should be repeated as much as possible in order to obtain a result of successful authentication, considering the time generally considered to be required for the authentication processing as the operation of the face image authentication device. Suppose that it is determined based on. In the present embodiment, as shown in FIG. 2, after the face tracking unit 522 starts to track the face area image of the target person for the target person who is walking, the face matching unit 523 is timed out and the authentication failure result is displayed. Assuming that the time for taking out is determined as a reference, it can be set to 10 seconds, for example.
Alternatively, it may be increased or decreased depending on the angle of view, depression angle, and operation location of the imaging unit 100. Further, the predetermined threshold value for the desired action counter is an integer value of 1 or more. For example, when the target person is still on the spot while directly looking at the imaging unit 100 for 3 to 4 seconds, it can be determined that the registered face image for the target person should be updated. Therefore, for example, when processing a monitoring image of 5 frames per second, the predetermined threshold value can be set to 20. Since the presence / absence of increment of the requested action counter is determined for each monitoring image, when the processing interval of the monitoring image is taken into account, the target person is directly looking at the imaging unit 100 and is stationary. Represents the time.

The operation of the history management unit 527 will be described below using the history table 410 shown in FIG. In the example illustrated in FIG. 4, the similarity 412 with a frame 415 indicates the similarity when it is determined that the authentication is successful. In addition, the possible range of the similarity is 0-10, the authentication threshold is 5, and the threshold of the desired action counter is 5.
For example, in a target person whose target person ID 312 is “0511”, the desired action counter 318 is “0” and does not exceed the threshold value of the desired action counter, and the face image showing the target person and the registered person ID 413 are “2”. Is similar to the registered face image “9.87”, which exceeds the authentication threshold. In this case, the face image authentication apparatus 10 determines that the target person is a registered person and immediately determines that the authentication is successful, and the history management unit 527 indicates that the authentication is successful in the corresponding column of the authentication result 416. The authentication time from the start of tracking the face area by the face tracking unit 522 to the successful authentication by the face matching unit 523 was “0.6” seconds, indicating that the registered person ID 413 was a registered person of “2”. And record the results.

On the other hand, in the target person whose target person ID 413 is “0011”, the desired action counter 318 is “5” and exceeds the threshold of the desired action counter, and the face image in which the target person is reflected and the registered person ID 413 are “1”. The similarity with the registered face image is “5.52”, which exceeds the authentication threshold. In this case, the face image authenticating device 10 determines that the target person is successfully authenticated. However, the result that the authentication time from the start of tracking the face area by the face tracking unit 522 to the authentication success by the face collating unit 523 was “5.2” seconds is recorded. Compared to the target person whose target person ID 412 is “0511”, the authentication time is longer by the amount of time he took the desired action before being authenticated, and the target person is requested until it is determined that the authentication is successful. It is thought that the person who took action was difficult to be authenticated.

However, in the target person whose target person ID 312 is “0205”, the desired action counter 318 is “314” and far exceeds the threshold value 5 of the desired action counter. The maximum similarity with the registered face image is “0.95”, which does not exceed the authentication threshold. Furthermore, as the authentication time 323, “10” seconds for which the face image authentication device 10 times out are recorded.
In this case, although the target person repeatedly took the desired action, the face image authentication apparatus 10 did not determine that the target person finally succeeded in authentication, that is, the authentication failed. This target person is considered to be a person who has been registered but is very difficult to authenticate due to hairstyle or makeup. Or it may be an unregistered visitor, or in some cases a suspicious person who attempts to intrude.

In addition, the history management unit 527 refers to the face image data 315 for the latest history data for which the authentication processing has been completed by the face matching unit 523, and whether the same target person has attempted authentication in the past, that is, the same It is determined whether or not history data is included in the history table 410 for the target person. In the latest history data, the group number 321 is not given, and the difficult-to-authenticate flag 322 is initialized to “OFF”.
For this purpose, the history management unit 527 uses the face image data 315 to perform collation processing similar to the face collation unit 523 and determines whether or not the history data of the same target person is included in the history table 410.
The history management unit 527 selects one face area image from the face image data 315 of the latest history data, and performs a matching process with the face image 315 of the history data to which the group number 321 has already been assigned. The face area image used for the collation process may be arbitrary, but it is preferable to select the face area image having the maximum value with reference to the direct visibility.
The history management unit 527 determines that the face image data 315 of the latest history data is the same person as any one of the face image data 315 of the history data to which the group number 321 has already been assigned. In addition, the same number as the group number 321 is stored in the group number 321 of the latest history data, assuming that the subject person who has attempted the authentication tries to authenticate again.

On the other hand, when the face image data 315 of the latest history data cannot be determined to be the same person as any face image data 315 of the history data to which the group number 321 has already been assigned, the history management means 527 A new number is generated and stored in the group number 321 column of the latest history data, assuming that this is the first person who has attempted authentication. The new number may be a number obtained by adding 1 to the maximum number already stored.
As a result of these processes, the same group number 321 is assigned to the history data of the same target person, and the frequency and the number of attempts to authenticate in the past are approximated to the entrance 253 in the schematic diagram of FIG. You will be able to figure out what happened.

Next, the operation of the determination unit 528 will be described using the history table 410 shown in FIG.
The determination unit 528 determines whether or not the subject person is difficult to authenticate, and eventually falls into an authentication difficult person who often ends in authentication failure.
Therefore, the determination unit 528 refers to the contents of the history table 410 and checks whether or not the same target person has failed in the past.
In the history table shown in FIG. 4, target persons whose target person IDs 312 are “0205”, “0642”, “0721”, and “0879” are determined to be the same target person, and the group number 321 is the same “59”. Is recorded. Further, referring to the respective authentication results 416, an authentication failure is recorded, and the subject person fails in the authentication every time he tries.
The determination unit 528 compares the number of history data that has failed in authentication with the history threshold value. The history threshold value is a threshold value regarding the number of times it can be determined that the person is difficult to authenticate when the number of authentication failures exceeds that value, and can be determined by the installation location and operating conditions of the face image authentication device 10. In this embodiment, it is 3.
In the case shown in FIG. 4, the target person who is determined to be the same person and assigned “59” to the group number 321 has four history data that failed authentication. Therefore, the determination unit 528 determines that the target person is a person who has difficulty in authentication, and sets the person with difficulty in authentication flag 322 to “ON”.

Next, the operation in the management mode of the face image authentication apparatus 10 will be described.
The face image authentication device 10 has a normal mode for collating the face image of the target person and an administrator mode for setting and maintaining the device. The registration unit 529 registers, updates, or deletes the registered face image in the administrator mode. The registration unit 529 functions as a display control unit for instructing the administrator to register, update, or delete the registered face image, and causes the display unit 300 to display a necessary screen.
The registration unit 529 registers, updates, or deletes the registered face image stored in the storage unit 510 in accordance with a signal corresponding to the instruction from the administrator received from the input unit 400.

The administrator of the face image authentication device 10 receives, for example, a report from a registered person of the face image authentication device 10 that it is determined that the person who has not been authenticated is determined to be difficult to authenticate, or periodically. When performing maintenance on the apparatus, a specific password or the like is input to the input unit 400 to instruct the face image authentication apparatus 10 to shift from the normal mode to the administrator mode. Upon receiving an instruction to shift to the administrator mode via the input unit 400, the registration unit 529 shifts the face image authentication apparatus 10 to the administrator mode and causes the display unit 300 to display a registered face image update screen.

FIG. 5A shows an example of a registered face image update screen. As shown in FIG. 5, the registered face image update screen includes a list display area 530 for displaying a list of target persons whose registered face images are to be updated, an update button 531, a list display button 532, a delete button 533, and a cancel button. 534.
In the list display area 530, the number of authentication failures 535, the candidate face image 536, and the registered face image 537 are displayed for each target person determined to be a person who is difficult to authenticate by the determination unit 528.
As the number of authentication failures 535, the number of history data for which authentication failure has occurred for the target person to whom the same group number 321 is assigned is displayed.
A candidate face image 319 of the matching data 310 is displayed as the candidate face image 536.
As the registered face image 537, a registered face image selected with reference to the similarity 412 calculated in the target person authentication process is displayed.

Immediately after shifting to the administrator mode, the registration unit 529 first causes the display unit 300 to display an update screen in which the list display area 530 is blank. When the registration unit 529 receives a signal indicating that the administrator has pressed the list display button 532 from the input unit 400, the registration unit 529 displays a list of information on persons whose authentication difficult person flag 322 is “ON”. The updated screen is displayed on the display unit 300.
At this time, the registration unit 529 displays a list of information on each target person, arranged in descending order of the number of authentication failures. Thereby, since it displays in order from the target person who frequently fails in authentication, the administrator can easily select the registered face image to be updated. Alternatively, with reference to the start time 313 of the collation data, target persons having the same number of authentication failures may be displayed in a list in order from the oldest.
The difficult-to-authenticate flag 322 is “ON”, as shown in FIG. 4, is a difficult-to-authenticate person who fails in authentication at each trial like the target person whose target person ID 312 is “0250”.

In FIG. 5A, “4”, which is the number of authentication failures of the target person to whom “59” is assigned as the group number 321 in FIG. Further, a candidate face image 538 of the target person is displayed in the candidate face image 536 column. Furthermore, the registered face image 537 displays a registered face image 539 of a person with a high similarity 412 calculated for the person of the candidate face image displayed in the candidate face image 536 field, and a selection button 540. ing.

The administrator of the face image authentication apparatus 10 visually compares the target face candidate face image 538 and the registered face image 539 based on the display result of FIG. 5A, and these are images representing the same person. It is determined whether or not there is. If it is determined that the images represent the same person, the administrator places the cursor on the update button 531 and presses it. When the registration unit 529 receives a signal indicating that the update button 531 is pressed from the input unit 400 in this state, the registration unit 529 updates the registered face image 539 with the candidate face image 538.
In this case, the registration unit 525 desirably displays the updated registered face image and the candidate face image 538 side by side on the display unit 300. Thus, the administrator can easily confirm that the registered face image has been updated to the desired candidate face image.

The selection button 540 is a button for changing the displayed registered face image 539 to another registered face image. For example, when the selection button 540 is pressed once, the registration unit 529 displays information 541 related to the registered face image displayed in FIG. 5B, and when the selection button 540 is pressed again, the registration unit 529 displays FIG. When the selection button 540 is pressed again to the information 542 related to the registered face image displayed on the screen, the display is changed to the information 543 related to the registered face image displayed in FIG. The display order of the information related to the registered face image is the order in which the similarity calculated for the registered face image and the face image of the target person of the candidate face image 538 is large. Accordingly, since the registered face images that are likely to be similar to the face image of the target person are displayed in order, the administrator can efficiently update the registered face images.

In addition, when the administrator visually compares the candidate face image 538 and the registered face image 539 of the target person, if the person in the candidate face image 538 is an unfamiliar person, the visitor is a bona fide visitor. Alternatively, it is possible to determine that the person is a suspicious person who has attempted to invade or attempted to inspect a crime.
In that case, the administrator does not update the registered face image. If it can be determined that the person is a suspicious person, an appropriate response such as a report can be made.

Hereinafter, with reference to the flowcharts shown in FIG. 6 and FIG. 7, the operation of the selection process of the candidate face image used for the detection of the difficult person and the update of the registered face image by the face image authentication apparatus 10 to which the present invention is applied will be described. To do. The operation flow described below is controlled by a control unit (not shown) that operates on the microprocessor constituting the image processing unit 500 and controls the face image authentication apparatus 10 as a whole.
Note that the collation table 310 and the history table 410 are initialized to be blank before performing the processing described below. Also, among the operations described below, the processing for FIG. 6 is performed every time one monitoring image is acquired.

First, the face image authentication apparatus 10 acquires a monitoring image obtained by capturing a monitoring area by the imaging unit 100 and sends the monitoring image to the matching unit 520 of the image processing unit 500 (step S701). Next, the face detection unit 521 of the matching unit 520 extracts a face area in which the face of the target person is shown from the acquired monitoring image, and cuts out the face area from the monitoring image to create a face area image ( Step S702). Next, the face detection unit 521 determines whether or not one or more face regions have been extracted (step S703). If no face area is extracted, the face detection unit 521 moves to step S701 and repeats the processes of steps S701 to S703.

On the other hand, when one or more face areas are extracted and a face area image is created in step S703, the face tracking unit 522 sets the tracking flag 314 in the matching table 310 to “ON” for all the extracted face areas. Is associated with the target person ID 312 (step S704). Then, the face tracking unit 522 determines whether there is a target person ID 312 in which the tracking flag 314 in the verification table 310 is “ON” that is not associated with any of the face areas of the current frame ( Step S705). If there is a target person ID 312 in which the tracking flag 314 in the matching table 310 that is not associated with any of the face areas of the current frame is “ON”, the target person of the target person ID 312 is successfully authenticated and enters the room. It is considered that the user has left the monitoring area without being authenticated. Therefore, in that case, the face tracking unit 522 sets the tracking flag 314 of the target person ID 312 that is not associated with any of the face areas of the current frame in the matching table 310 to “OFF” so that the tracking process is not performed thereafter. To do. Further, the history management unit 527 calculates the authentication time from the difference from the current time with reference to the start time 313 of the collation table 310, and records it in the history table 410 together with the authentication failure (step S706).
When a person in the face area of the previous frame times out without being authenticated, a time (for example, 10 seconds) indicating a predetermined time out is recorded.
If the authentication flag 317 is “OFF”, the similarity calculated so far is stored in the corresponding history table 410.

Then, the face tracking unit 522 adds or updates the collation data in the collation table for each face area of the current frame (step S707). When the face area of the current frame cannot be associated with the target person ID 312 in which the tracking flag 314 in the verification table 310 is “ON”, the face tracking unit 522 stores the verification data for the target person in the verification table 310. Add a new. That is, the face tracking unit 522 assigns a new trial number 311 and a target person ID 312 to the collation data, records the current time as the start time 313, and sets the tracking flag 314 to ON. The face tracking unit 522 records a face area image created from the face area of the current frame as the face image data 315 and also records coordinate information and the current time of the face area as tracking position information 316. Further, the face tracking means 522 sets the authentication flag 317 to “OFF” and sets the desired action counter 318 to “0”.
On the other hand, when the face area of the current frame can be associated with the face area of the previous frame, the face tracking unit 522 creates the face image data 315 of the matching data in the matching table 310 from the face area of the current frame. The updated face area image is added, and update processing for adding the coordinate information and the current time of the face area to the tracking position information 316 is performed. Then, the face tracking unit 522 outputs the updated matching data to the face matching unit 523.

The following processing in steps S708 to S716 is performed for each verification data in which the tracking flag 314 is “ON” and the authentication flag 317 is OFF. The history data in which the authentication flag 317 is ON among the verification data in which the tracking flag 314 is “ON” is not processed because it has already been authenticated. Therefore, the face collating unit 523 collates the latest face image of the history data output from the face tracking unit 522 with each registered face image read from the storage unit 510 (step S708). Then, the face matching unit 523 determines whether or not the authentication is successful (step S709).

If the authentication fails in step S709, the face orientation detection unit 525 calculates the direct visibility for the face area image in which the face of the target person whose authentication has failed is performed, and associates the face area image with the face area image to store 510 (Step S710). Next, the stillness detecting means 526 calculates the degree of stillness for the face area, and stores it in the storage unit 510 in association with the face area image (step S711). Then, the behavior detection unit 524 determines whether or not the target person is stationary while looking directly at the camera from the degree of direct vision and the degree of stillness (step S712). If the action detecting unit 524 determines that the target person is stationary while looking directly at the camera, the action detecting unit 524 increments the desired action counter 318 (step S713).

On the other hand, when the authentication is successful in step S709, the face matching unit 523 outputs a signal indicating the authentication success to the history management unit 527 via the behavior detection unit 524, and the history management unit 527 indicates that fact in the history table. It is recorded in the authentication result 416 of 410 (step S714).
Next, the face matching unit 523 outputs a signal indicating successful authentication to the output unit 200, and unlocks the electric lock via the output unit 200 (step S715). As a result, the target person who has succeeded in authentication can enter the room. It is assumed that the electric lock is locked immediately after the entry of the target person can be confirmed or automatically after a predetermined time has elapsed.

Next, the face matching means 523 sets the authentication flag 317 to “ON”. Further, the history management unit 527 calculates an authentication time from the start time 313 of the verification table 310 and the latest tracking position information 316, and stores it in the history table 410 together with the similarity calculated by the face verification unit 523 (step S716).
When the action detecting means 524 increments the desired action counter 318 in step S713, or when the face matching means 523 sets the authentication flag 317 to “ON” and the history management means 527 calculates the authentication time and records the similarity, It is determined whether or not collation processing has been performed for all face area images.
If there is a face image that has not been collated yet, the control returns to step S708, and the processes of steps S708 to S716 are repeated.

Next, a process for determining whether or not the target person recorded in the history table 410 is a person who has difficulty in authentication will be described with reference to FIG. This process is executed by the history management unit 527 and the determination unit 528 every time the authentication process is completed for one target person. Or it may operate when the face image authentication device 10 shifts to the management mode.

First, the history management unit 527 reads the history table 410 stored in the storage unit 510 (step S801).
Next, the history management unit 527 checks whether there is history data to which the group number 321 is not assigned in the history data (step S802). If a group number is assigned to all history data, the process proceeds to step S807. If there is history data to which no group number is assigned, the processing of steps S803 to S806 is performed for all of the history data.

The history management unit 527 uses the face image data 315 for the history data to which no group number is assigned, and calculates the similarity with the face image 315 of other history data to which the group number is assigned (step S803). ). This is intended to check whether the target person of the history data to which the group number is not assigned has already attempted authentication before that, that is, whether the history data of the same target person exists.
If the similarity obtained in step S803 is greater than or equal to a certain level, history data for the same target person already exists, so the history management means 527 determines the group number of the already existing history data. Is recorded in the group number 321 of the history data to which no group number is assigned (step S806). Note that when there are a plurality of pieces of history data in which the degree of similarity obtained in step S803 exceeds a certain level, the history data having the maximum degree of similarity is adopted. This is because it seems to be the same person.

On the other hand, if the similarity obtained in step S803 does not exceed a certain level with any of the other history data to which the group number is assigned, the target person is considered to have attempted authentication for the first time, and therefore the group number 321 Is given a new number. The new number may be a value obtained by adding 1 to the maximum number already assigned (step S805).
By repeating steps S803 to S806, group numbers are assigned to all the history data recorded in the history table 410, and the same target person is grouped. For example, the number and timing of authentication failures can be grasped. Become.

Next, the determination unit 528 refers to the value of the desired action counter 318 and the number of history data included in the face image group for a set of history data (face image group) to which the same group number is assigned. Steps S807 to S809 are performed.
Here, as described above, the value of the requested action counter 318 is incremented when the degree of direct vision is high and the degree of stillness is high. Therefore, when the value of the requested action counter 318 is high, the target person captures the face. It seems that it is trying to be authenticated because it is facing the front and stops or moves much slower than the normal walking speed. On the other hand, even if the result of authentication failure is reached, it is inappropriate to determine a person who does not take a desired action such as stopping or turning his face to the camera as a person who has difficulty in authentication. Therefore, by referring to the desired action counter 318, it is possible to prevent a person having a low value from being erroneously determined as a person who has difficulty in authentication.
The reason why the number of history data included in one face image group is referred to is that it is considered that a certain number of authentication histories are necessary to determine that the person is difficult to authenticate. For example, even if a registrant tries to authenticate for the first time, if he / she does not know his / her identity and accidentally fails, it is still inappropriate to immediately determine that he / she is difficult to authenticate.

The determination unit 528 includes authentication data for a face image group that includes history data with a desired action counter 318 value of a threshold value (for example, 5) or more and the number of history data items included in the face image group is a predetermined history threshold value or more. The number of trials that ended is checked (step S807).
If it is less than the history threshold, the determination unit 528 does not determine that the target person of the face image group is a person who is difficult to authenticate.
If the number of trials that result in authentication failure exceeds the history threshold, the determination unit 528 determines that the target person of the face image group is a person who has difficulty in authentication, and sets the person having difficulty in authentication flag 322 to ON (step S808). ).
The history threshold value is determined by the installation location and operating conditions of the face image authentication device 10 and can be set to 3, for example.

Next, the history management unit 527 selects candidate face images. This candidate face image is for updating the registered face image 414 when the target person determined to be a difficult-to-authenticate is a registrant, and is recorded in the collation table 310 (step S809).
The history management unit 527 selects a face area image obtained by cutting out the face area when the degree of direct view is the highest among the face area images recorded for the target person as a candidate face image, and the matching table shown in FIG. The candidate face image 319 is recorded in 310. The high degree of direct vision indicates that the target person is about to be authenticated and the face is directed toward the imaging unit 100 in the front direction, so that the registered face image can be updated to a face image suitable for collation.
Alternatively, the history management unit 527 selects a face area image obtained by cutting out the face area when the degree of stillness is the highest among the face area images recorded for the target person, and records the selected face area image in the collation data. But you can. When the degree of stillness is high, the target person may stop trying to be authenticated, or may be moving much slower than the normal walking speed. In such a case, the target person captures a face to be authenticated. There is a high possibility of being directed to the part 100. Therefore, by selecting the face area image at that time as a candidate face image, there is a high possibility that the registered face image can be updated to a front-facing face image suitable for collation.
Alternatively, the history management unit 527 selects a face area image having the highest degree of similarity to any registered face image from among the face area images recorded for the target person, and records the selected face area image in the collation data. But you can. In this case, the updating process in the registration unit 529 can be efficiently performed to update the registered face image.

As described above, the face image authentication apparatus 10 to which the present invention is applied sequentially acquires monitoring images, and associates face areas in which the same person's face is reflected across the sequentially acquired monitoring images. Then, the face image authentication device 10 determines the orientation of the face shown in the face area image of the current frame, and the position of each face area where the face of the same person extracted across the plurality of monitoring images is shown in the monitoring image. The behavior feature amount is calculated from the change and recorded together with the authentication result. Then, the face image authentication device 10 detects the person as a person who has difficulty in authentication when the authentication failure frequently occurs despite the person taking the desired action from the action feature amount. As a result, the face image authentication device 10 adds a person who is difficult to authenticate to the action and authentication history of the person because the registered face image is different from the face area image acquired at the time of authentication. Based on this, it can be detected with high accuracy early. Then, the administrator of the face image authentication apparatus 10 can quickly recognize the necessity of updating the registered face image and select a face used for updating the registered face image suitable for authentication.

The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments.
For example, the determination process in the determination unit 528 that the target person is a person who is difficult to authenticate is not limited to the number of authentication failures for the target person to which the same group number 321 is assigned.
The determination unit 528 has failed in the authentication for the total number of history data to which the same group number 321 is assigned in the case where the history data to which the same group number 321 is assigned may be successful. It can also be determined that the person is difficult to authenticate using the ratio of the number of history data. In this case, the threshold for determining that the person is difficult to authenticate may be 80%, for example. As a result, the administrator of the face image authentication apparatus 10 needs to promptly update the registered face image for users who feel inconvenience because authentication failure occurs at a high rate although authentication is not possible at all. Can be recognized.

Alternatively, the determination unit 528 may determine that the individual is difficult to authenticate by referring to the date and time of the trial for each history data, limiting the number to the latest history data, and referring to the number of authentication failures. That is, with respect to history data to which the same group number 321 is assigned, referring to the start time 313 of the collation table 310, out of the most recent history data numbers (for example, 10), the history data that has failed in authentication is a predetermined number ( For example, in the case of 8) or more, it can be determined that the target person is a person who is difficult to authenticate. As a result, the administrator of the face image authenticating apparatus 10 places importance on the latest authentication result than the past and updates the registered face image for users whose authentication success tendency has changed due to changes in makeup preferences or hairstyles. The need can be recognized.

Alternatively, the determination unit 528 refers to the date and time of the trial for each history data, and refers to the fact that the time interval of the trial that failed in authentication is shorter than the past, and determines that the person is difficult to authenticate. Good. That is, the determination unit 528 refers to the start time 313 of the collation table 310 for the history data to which the same group number 321 is assigned, and includes the history included in the latest history reference period (for example, one week retroactive from the present time). Compare the data with the historical data included in the historical reference period of the past (for example, 3 months ago), and if the historical data that has failed in authentication is included in the historical data most recently than the past, the target person is authenticated It is determined that the person is difficult. Alternatively, by fixing the length of the history reference period and obtaining a moving average over time, it is detected that the history data that has failed in authentication tends to increase, and the target person is determined to be a person with difficulty in authentication. . Even in this method, the administrator of the face image authentication apparatus 10 places importance on the authentication result more recent than the past for the user whose tendency of success or failure of authentication has changed due to a change in makeup preference or hairstyle. Recognize the need for updates.

Alternatively, when the result of authentication failure is obtained by the face matching unit 523 without referring to the history data shown in FIG. 4, the determination unit 528 determines that the value of the desired action counter 318 exceeds the threshold value. The target person may be determined to be a person who is difficult to authenticate. For example, when the display unit 300 is a monitor screen installed in the management center of the office building in which the face image authentication apparatus 10 is installed, the determination result is displayed with a warning message on the screen, whereby the face image authentication is performed. The administrator of the apparatus 10 can recognize the necessity of updating the registered face image as soon as a result of authentication failure is obtained.

Furthermore, the behavior detection unit 524 does not determine that the target person who stops or walks slowly while directly looking at the camera is the person who is taking the desired action, but is simply the target person who is directly looking at the camera, or simply stops or slowly The person to walk may be determined as the person taking the desired action.
In that case, the behavior detection unit 524 calculates either or both of the direct visibility and the static degree, and the desired behavior when the direct visibility is equal to or greater than the direct visibility threshold or when the static degree is equal to or greater than the static degree threshold. Increment the counter. That is, the number of face area images that are determined that the target person has his / her face facing the imaging unit within the authentication time is greater than or equal to a predetermined number, or the target person is substantially stationary within the authentication time. When the time determined to be present is equal to or longer than the predetermined time, the behavior detection unit 524 determines that the target person is about to be authenticated but is in a state where it is difficult to authenticate, and increments the desired behavior counter.
The direct vision threshold or the stillness threshold at this time is the direct visibility threshold or the stillness threshold when the desired action counter is incremented when the directivity is equal to or greater than the direct visibility threshold and the stillness is equal to or greater than the stillness threshold. It may be the same. Alternatively, in order to suppress erroneous detection of a person who has not taken the desired action, the directivity threshold or the staticity threshold at this time is such that the directivity is equal to or greater than the directivity threshold and the staticity is equal to or greater than the staticity threshold. When the desired action counter is incremented in this case, the direct visibility threshold may be larger than the staticity threshold.
Also in this case, the face image authentication apparatus 10 can detect a person who frequently fails in authentication unlike the face area image acquired at the time of authentication, with high accuracy, and can detect the face image authentication. The administrator of the apparatus 10 can quickly recognize the necessity of updating the registered face image.

As described above, those skilled in the art can make various modifications in accordance with the embodiment to be implemented within the scope of the present invention.

DESCRIPTION OF SYMBOLS 100 ... Imaging part 200 ... Output part 300 ... Display part 400 ... Input part 510 ... Memory | storage part 520 ... Collation part 524 ... Action detection means 525 ... Face direction detection Means 526 ... Stationary detection means 527 ... History detection means 528 ... Determination means 529 ... Registration means

Claims (6)

An imaging unit that sequentially acquires a monitoring image obtained by photographing a person in the monitoring area;
A storage unit that stores in advance a registered face image that is a face image of a registered person;
A face image authentication device comprising a matching unit that performs person authentication using the monitoring image,
The collation unit
Face detection means for extracting an image of an area including a human face from the monitoring image as a face area image;
Face orientation detection means for detecting the face orientation of the person using the face area image;
A face collating means for collating the face area image with the registered face image and authenticating whether or not a person of the face area image is the registered person;
The face collating means authenticates that the person of the face area image is not the registered person, and at least a face shown in the face area image is that the person has taken a predetermined desired action to be authenticated. Determining means for determining that the person is a person who is difficult to authenticate on the condition that the number of the face area images facing substantially in front of the imaging unit is a predetermined number or more ;
A face image authentication apparatus characterized by comprising:
The collation unit further includes a stationary detection means for detecting that the acquired face area image is substantially stationary,
The face image authentication apparatus according to claim 1, wherein the determination unit determines that the predetermined desired action is that at least a time during which the face area image is substantially stationary is a predetermined time or more.
The collation unit further includes face tracking means for tracking the face area image of the same person in the monitoring images acquired sequentially,
The face image authentication apparatus according to claim 2, wherein the stillness detection unit detects that the tracking position of the face area image is substantially stationary.
The collation unit further includes a history management unit that associates the face area image with the authentication result of the face collation unit and stores it in the storage unit as a collation history,
The determination means refers to the verification history, any one of claims 1 to 3 was added to the condition that the same of the authentication result is not a the registered person relates person is above history threshold The face image authentication device described in 1.
The collation unit further includes a history management unit that associates the face area image with the authentication result of the face collation unit and stores it in the storage unit as a collation history,
The determination unit adds to the condition that, with reference to the verification history, an authentication result indicating that the same person is not the registered person included in a predetermined history reference period tends to increase. The face image authentication device according to any one of to 3 .
The face image authentication device includes:
A display unit; and a registration unit that re-registers the registered face image with the face area image by an external instruction input;
The registration means displays the registered face image and the face area image side by side on the display unit in descending order of the number of authentication results indicating that the person is not a registered person, and the registration displayed when receiving an instruction input. face image authentication device according to the face image to one of claims 1 to 5 which is characterized in that to re-register in the face region image.

Images