JP2006031387A - Image recognition device, image recognition method, image recognition program, and recording medium to which the image recognition program is recorded - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition device which can realize high-precision image recognition in a short period, and also provide an image recognition method, an image recognition program and a recording medium to which the program is recorded. <P>SOLUTION: Collation data on the face images of a plurality of persons are preliminarily stored in a collation data storage part 25. The collation data include individual collation data corresponding to the multiple sheets of face images and one representative collation data for each person. The representative collation data are the data representing a plurality of the individual collation data. A collation data selection part 42 first selects the representative collation data as an object to be collated. A similarity computation part 41 collates face image data to be collated with the representative collation data and computes the similarity. When the similarity is low, the collation data selection part 42 excludes a plurality of the individual collation data corresponding to the representative collation data from the object to be collated with the face image data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention processes image data and records an image recognition apparatus, an image recognition method, an image recognition program, and an image recognition program for recognizing a recognition object included in an image represented by the image data. Relates to the recording medium.

  When restricting access to specific areas, personal authentication is performed at the entrance. One example of an apparatus conventionally used for personal authentication is a card reader. In this case, an identification card such as a magnetic card is distributed in advance to those who are permitted to enter. A person who intends to enter the restricted access area causes the card reader arranged at the entrance to read the identification information recorded on the identification card. If the read identification information matches the identification information registered in advance, entry is permitted and the gate installed at the entrance is opened or the door lock is unlocked.

However, in such a system, if a person forgets to carry a card or loses a card, even a person who is permitted to enter cannot enter. Another problem is that security is easily broken due to theft of the card.
Another example of a conventional device for personal authentication is a password input device. In this case, a password is taught in advance to those who are permitted to enter. A person who wants to enter the restricted access area inputs a password from a password input device arranged at the entrance. If the input result matches a pre-registered password, entry is permitted and the gate installed at the entrance is opened or the door lock is unlocked.

With this system, there is no risk of forgetting or losing the card. However, if a person who is allowed access has forgotten his password, that person will not be able to enter. Also, if a password is leaked, security is broken.
Recently, personal authentication technology using human biometric information (biometric data) has been actively developed. If personal authentication using biometric information is used, all of the above-mentioned problems are solved.

  Personal authentication techniques using biometric information include those using fingerprints, those using irises, and those using facial images. However, when using a fingerprint, it is necessary to register the fingerprint in advance. Therefore, there are many users who have a sense of resistance. In addition, for personal authentication using an iris, the user must position his eyes at a position in close proximity to the camera at the time of authentication. Therefore, the user is strongly aware of the personal authentication procedure and feels annoyed. Personal authentication using face images does not have such disadvantages, and there is a possibility that a personal authentication system with less resistance can be constructed.

  For personal authentication using a face image, for example, an image recognition device configured by a computer system is used to recognize the face image (Patent Document 1 and Patent Document 2). This image recognition apparatus includes collation data storage means in which image data of a face image of a person permitted to enter a restricted access area or feature data of the image data is registered in advance as collation data. Further, the image recognition device includes an imaging device that captures a face image of a person to be recognized and converts the image into image data, a collation unit that collates image data output by the imaging device and registration data in the collation data storage unit, And determining means for outputting a recognition result based on the collation result by the collating means.

The collation means calculates the similarity between the face image captured by the imaging device and the face image corresponding to the collation data registered in the collation data storage means. The determination means compares the calculated similarity with a predetermined recognition threshold value, and if the similarity exceeds the recognition threshold value, the person to be recognized is one of the persons registered in the collation data storage means. Outputs the result of determination to match.
JP 2003-346149 A JP 2001-266151 A Yoav Freund et al., “A decision-theoretic generalization of on-line learning and an application to boosting”, Journal of Computer and System Sciences, 1997, p.119-139.

In the image recognition apparatus as described above, variation factors such as the illumination conditions at the time of imaging of the person to be recognized, the posture of the person to be recognized, and the facial expression thereof greatly affect the recognition accuracy. Therefore, collation data corresponding to a plurality of images (for example, 50 to 100 images per person) in consideration of variation factors is registered in advance in the collation data storage unit.
The recognition accuracy increases as the number of registered images for each person increases. However, since the collating unit collates all images registered in the recognition dictionary with the face image captured by the imaging device to obtain the similarity, the time required for the recognition processing increases as the number of registered images increases. become longer.

As a result, the time during which the user is restrained for the authentication procedure (waiting time at the entrance of the restricted access area) becomes long, so that a personal authentication system with little resistance cannot be constructed.
In other words, in the above-described prior art in which the face image of the person to be recognized is collated with all the face images registered in the collation data storage means, there is a conflict between the improvement in recognition accuracy and the reduction in the time for recognition processing. Therefore, it was not possible to achieve both of these.

When recognizing not only a face image but also an object having various appearances, there is a similar problem, and it has been a technical problem to realize accurate image recognition in a short time.
Accordingly, an object of the present invention is to provide an image recognition apparatus, an image recognition method, an image recognition program, and a recording medium on which the program is recorded, which can realize highly accurate image recognition in a short time.

  In order to achieve the above object, an invention according to claim 1 is an image recognition apparatus for processing image data and recognizing a recognition object in an image corresponding to the image data. Individual collation data storage means in which a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken are stored in advance, and representative collation data generation that generates representative collation data corresponding to a representative appearance representing the plurality of individual appearances And the image data to be processed and the representative collation data generated by the representative collation data generating means are collated, and the similarity between the image corresponding to the image data and the representative appearance corresponding to the representative collation data is calculated. Representative collation means, the image data to be processed and the individual collation data stored in the individual collation data storage means are collated, and the image corresponding to the image data Individual collation means for calculating the similarity with the individual appearance corresponding to the different collation data, and a plurality of individual collations stored in the individual collation data storage means based on the similarity calculated by the representative collation means An image recognition apparatus comprising: an individual collation control unit that determines whether data is to be collated by the individual collating unit or excluded from a collation target by the individual collating unit.

“Similarity” refers to the probability of being the same object (likeness of the same object) or the probability of being the same kind of object (likeness of the same object).
According to this configuration, the representative collation data corresponding to the representative appearance representing the plurality of appearances of the recognition object is generated, and the representative collation data and the image data to be processed are collated. Then, based on the similarity between the representative appearance and the processing target image, it is determined whether or not to perform the verification between the individual verification data representing the individual appearance and the processing target image data.

  As a result, a plurality of individual verification data with a low probability of obtaining a high degree of similarity can be collectively excluded from the verification target by a single verification process between the representative verification data and the image data to be processed. It can be significantly reduced. In particular, when the recognition object is an unregistered object (one in which individual collation data is not stored in the individual collation data storage means), there is a possibility that the recognition process can be completed only by the collation process on the representative collation data. There is.

  According to a second aspect of the present invention, the individual collation control means is configured to detect the image data to be processed and the individual collation data storage means when the similarity calculated by the representative collation means is greater than a predetermined threshold value. When the similarity calculated by the representative verification unit is less than or equal to the threshold value, each of the individual verification data stored in is stored in the verification data storage unit. 2. The image recognition apparatus according to claim 1, wherein the plurality of individual collation data are excluded from collation targets by the individual collation means.

  According to this configuration, if the degree of similarity between the image to be processed and the representative appearance exceeds a predetermined threshold value, individual verification data representing a plurality of individual appearances represented by the representative appearance is processed. Matched with image data. That is, the similarity between the image to be processed and a plurality of individual appearances is obtained. On the other hand, when the similarity between the representative appearance and the image to be processed is equal to or less than the predetermined threshold value, the plurality of individual verification data represented by the representative appearance is the target of the individual verification with the image data to be processed. Excluded.

  In this case, the functions of the representative collating means, the individual collating means, and the individual collating control means specifically correspond to the image corresponding to the image data to be processed and the representative appearance corresponding to the representative collating data or the individual collating data. Similarity calculation means for calculating the similarity to the individual appearance, and first, the representative collation data is selected as the target of similarity calculation by the similarity calculation means, and then the similarity calculation result for the representative collation data This can be realized by a configuration including collation candidate selection means for sequentially selecting a plurality of individual collation data stored in the individual collation data storage means on the condition that the value exceeds a predetermined threshold value.

  The “recognition object” may be an object whose appearance changes. Examples of changes in appearance include changes in the shape of the object itself (dynamic changes and secular changes), apparent changes depending on illuminance, and changes due to the presence or absence of accessories (such as glasses or makeup). Examples of such recognition objects include animal (especially human) faces and animal physiques. “Form” means a concept including shape, pattern or color, or any combination thereof.

In addition, the recognition target object may be the same kind of object having various appearances (an object having a different form or the like within a certain range). Examples of such recognition objects include fruits, empty bottles, automobiles, machine parts, and ground objects (such as buildings) viewed from an aircraft.
The image data to be processed may represent a still image or a moving image. It is preferable that the individual collation data to be collated with the still image data corresponds to the still image of the appearance of the recognition object, and the individual collation data to be collated with the video image data is the appearance of the recognition object. It is preferable that the video corresponds to the video. Of course, the image data of the still image may be image data of a specific scene extracted (clip) from the moving image data.

The image data to be processed may be input from an imaging device (camera) that captures the recognition object and outputs the image data, or image reproduction that reproduces image data recorded in a memory, an optical disk, or other recording medium. It may be input from a device or acquired by communication.
The individual collation data stored in the individual collation data storage means may be image data obtained by capturing an image of the recognition target in advance, or predetermined feature amount data extracted from such image data. May be. The individual collation data storage means preferably stores individual collation data corresponding to a plurality of images obtained by imaging various appearances of the recognition target object in advance.

  The representative collation data generated by the representative collation data generation means may be data obtained by performing a predetermined representative value calculation (for example, average value calculation) on a part or all of the corresponding individual verification data. . The representative collation data may be one representative individual collation data selected from a plurality of corresponding individual collation data. As in the case of the individual verification data, the representative verification data may be image data or feature amount data extracted from the image data.

Further, the representative collation data generation means does not have to generate one representative collation data for one recognition object (for example, the same person), and a plurality of representative collation data for one recognition object. (For example, two representative collation data corresponding to the presence or absence of an accessory such as glasses) may be generated.
The representative collation data generation means may be a representative collation data storage means for storing representative collation data calculated in advance, and the representative collation data generation means performs a computation on a plurality of individual collation data stored in the individual collation data storage means. It may be representative collation data calculation means for generating collation data.

  The invention according to claim 3 is an image recognition apparatus for processing image data and recognizing a recognition object in an image corresponding to the image data, and having a plurality of individual appearances that the recognition object can take. A plurality of corresponding individual collation data is classified and stored for each of a plurality of groups, and a plurality of individual collation data representing a plurality of groups stored in the individual collation data storage means. Representative collation data generating means for generating representative collation data, image data to be processed and a plurality of representative collation data generated by the representative collation data, respectively, and an image corresponding to the image data and the plurality of representative collations Representative collating means for calculating the degree of similarity with a plurality of representative appearances respectively corresponding to the data, an image corresponding to the image data to be processed, and the individual collation data storage Individual collating means for calculating the similarity with the individual appearance corresponding to the individual collation data stored in the stage, and based on the similarity calculated for each of the plurality of representative collation data by the representative collating means, A group of individual collation data to be collated with the image data to be processed by the individual collation means is selected from among a plurality of groups of individual collation data stored in the individual collation data storage means, and An image recognition apparatus comprising: individual collation control means for excluding individual collation data from a collation target by the individual collation means.

  According to this configuration, the individual collation data storage means stores a plurality of individual collation data classified for each group, and representative collation data representing the individual collation data of each group is represented by the representative collation data generation means. It is to be generated. Then, by collating the representative collation data with the image data to be processed, the individual collation data to be collated and the individual collation data to be excluded from the collation target are separated on a group basis.

  In this manner, a plurality of individual verification data with a low probability of obtaining a high degree of similarity can be collectively excluded from the verification target in units of groups by a single verification process between the representative verification data and the image data to be processed. . Therefore, the number of collation operations can be remarkably reduced, so that the number of collation operations can be significantly increased even when individual collation data corresponding to a large number of individual appearances are stored in the individual collation data storage means to improve recognition accuracy. There is nothing to do. Therefore, an accurate recognition result can be obtained by a short collation process.

The grouping of the plurality of individual verification data is preferably performed so that the individual verification data of the same object or the same type of objects belong to the same group. That is, for example, when a person's face image is a recognition target, individual collation data of the same person's face image belongs to the same group, and representative collation data representing each person's face image is generated. preferable.
Even in the case of individual verification data of the same object or the same kind of object, it may be preferable to perform grouping so as to belong to different groups. For example, when a face image of a person is a recognition target, individual verification data of the face image with an accessory such as glasses and an individual verification of the face image of the person with no accessory attached Data may be classified into different groups. It is also conceivable to classify the individual verification data into different groups for each type of facial expression even for the same person.

In addition, it is also conceivable to classify the individual verification data by using, as an index, characteristic items related to the appearance tendency such as gender, age range, and race.
Further, the classification (grouping) of the individual verification data may be performed hierarchically (multi-level classification). Specifically, when a person's face image is a recognition target, gender is used as a classification index of the first hierarchy (topmost hierarchy), an age range is set as a classification index of the second hierarchy, and an individual is classified as a third hierarchy. Classification may be performed using the presence / absence of an accessory, the facial expression tendency, or the illuminance tendency as the classification index of the fourth layer. In this case, representative collation data representing representative face images of men and women is generated by the representative collation data generation means corresponding to the classification of the first hierarchy. In addition, representative collation data representing representative face images of men and women for each age range is generated by the representative collation data generation means corresponding to the classification of the second hierarchy. Further, representative collation data representing a plurality of face images of each person is generated by the representative collation data generation means in correspondence with the classification of the third hierarchy. Then, corresponding to the classification of the fourth hierarchy, representative collation data for each face image of each person classified by the presence / absence of an accessory, a tendency of facial expression or a tendency of illuminance is generated by the representative collation data generation means. Become. Of course, the above four-layer classification is only an example, and it is possible to classify multiple levels of individual verification data using only two of these, and to use other feature items as classification indices. It may be introduced to classify the individual verification data into five or more layers.

  When the individual verification data is classified hierarchically, it is preferable that the verification of the representative verification data in the upper hierarchy and the image data to be processed are performed first. Then, based on the similarity obtained for the representative collation data of the higher hierarchy, if the individual collation data of the group corresponding to this representative collation data should be excluded from the collation target, the group belonging to the lower group The collation between the representative collation data and the image data to be recognized may be omitted. In this way, the number of collation operations can be further reduced.

  According to a fourth aspect of the present invention, the individual collation control means selects a group corresponding to representative collation data whose similarity calculated by the representative collation means is greater than a predetermined threshold value, and belongs to this group. 4. The image recognition apparatus according to claim 3, wherein the individual collation data is a collation target by the individual collation means, and the individual collation data of the remaining group is excluded from the collation targets by the individual collation means.

According to this configuration, the similarity between the image to be processed and the representative appearance is compared with a threshold value, and a matching operation (similarity calculation) is performed on the individual matching data of each group according to the comparison result. Whether or not
In this case, the functions of the representative collating means, the individual collating means, and the individual collating control means specifically correspond to the image corresponding to the image data to be processed and the representative appearance corresponding to the representative collating data or the individual collating data. First, the similarity calculation means for calculating the similarity to the individual appearance, and the similarity calculation by the similarity calculation means, first, representative matching data corresponding to the plurality of groups is sequentially selected, and then the similarity This can be realized by a configuration including collation candidate selection means for sequentially selecting a plurality of individual collation data belonging to a group corresponding to representative collation data whose calculation result exceeds a predetermined threshold value.

The threshold value to be compared with the similarity obtained for the representative collation data may be a fixed value, for example, a plurality of similarities obtained corresponding to a plurality of representative appearances by the representative collation means. It may be a value determined by performing a predetermined calculation on.
According to a fifth aspect of the present invention, the individual verification control unit selects a group corresponding to representative verification data in which the similarity calculated by the representative verification unit takes a value within the upper predetermined rank or the upper predetermined ratio. 5. The individual collation data belonging to the group is set as a collation target by the individual collation means, and the individual collation data of the remaining group is excluded from the collation targets by the individual collation means. This is an image recognition apparatus.

  That is, for example, the individual collation control means ranks (sorts) the similarity obtained for each of a plurality of representative collation data, and individually selects the group corresponding to the representative collation data in the upper predetermined number or predetermined range. The verification data is used as a verification target, and the individual verification data corresponding to the remaining representative verification data is excluded from the verification target. In this case, it can be understood that the threshold value is set so that the similarity within the upper predetermined rank or the upper predetermined ratio is extracted.

  The feature of claim 5 can be combined with the feature of claim 4. For example, the individual collation control means, even when the similarity obtained for the representative collation data takes a value within the predetermined rank or within a predetermined ratio, the similarity is equal to or less than a certain lower threshold value. In this case, the individual verification data of the group corresponding to the representative verification data may be excluded from the individual verification targets.

  According to a sixth aspect of the present invention, it is determined whether or not a maximum similarity that is a maximum value of a plurality of similarities calculated by the individual collating means for a plurality of individual collation data is greater than a predetermined recognition threshold value. And a recognition result output means for outputting a recognition result corresponding to the individual collation data of the highest similarity when the highest similarity is larger than the recognition threshold, and the highest similarity is recognized. When the threshold value is equal to or smaller than the threshold value, at least a part of the individual verification data excluded from the verification target by the individual verification control unit is added to the verification target by the individual verification unit, and the individual verification process by the individual verification unit is performed. 6. The image recognition apparatus according to claim 3, further comprising individual collation control means.

  According to this configuration, the similarity (maximum similarity) is compared with the predetermined recognition threshold for the individual collation data that has the highest similarity with the image data to be processed. When the highest similarity exceeds the recognition threshold value, a recognition result corresponding to the individual verification data is output. That is, it is determined that the recognition target image corresponding to the individual verification data is included in the recognition target image.

On the other hand, when the highest similarity is not more than the recognition threshold value, the individual verification data once excluded from the verification target by the individual verification unit by the individual verification control unit is restored as the verification target. As a result, it is possible to avoid a situation in which image recognition becomes impossible due to the exclusion of the individual collation data based on the collation result with the representative collation data.
The invention according to claim 7 further includes a representative appearance similarity storage unit that stores a plurality of similarities obtained by the representative verification unit, and the re-individual verification control unit is excluded from a verification target by the individual verification unit. Among the groups of individual collation data that have been processed, the individual collation data of the group corresponding to the representative collation data having the highest similarity stored in the representative appearance similarity storage unit is added to the collation target by the individual collation unit The image recognition device according to claim 6, wherein the image recognition device is a device.

  According to this configuration, a plurality of similarities obtained by collating representative collation data representing individual collation data of a plurality of groups with image data to be processed are stored in the representative appearance similarity storage unit. When the maximum similarity for the individual verification data does not exceed the recognition threshold, the individual verification data belonging to the group corresponding to the representative verification data are selected in order from the representative verification data having the highest similarity. It is revived as a verification target. Such processing is preferably executed repeatedly until individual verification data that reaches the recognition threshold is found. Of course, if the individual collation for all the individual collation data stored in the individual collation data storage means is completed, it is determined that the recognition target object is not included in the processing target image, and the image recognition processing is ended. become.

In this way, it is possible to efficiently find the correct individual collation data from the group excluded from the individual collation targets as a result of collation between the representative collation data and the processing target image data.
The individual collation data added to the individual recognition target by the re-individual collation control means only needs to include at least the individual data of the group having the highest similarity regarding the representative appearance among the groups of the individual collation data once excluded from the collation target. . That is, the individual collation data revived as an individual collation target by the re-individual collation control means may belong to a group of representative appearances having a similarity within the upper predetermined rank or the upper predetermined ratio, for example. .

  The invention according to claim 8 is an image recognition method for processing image data and recognizing a recognition object in an image corresponding to the image data, and having a plurality of individual appearances that the recognition object can take. A step of preparing an individual collation data storage means in which a plurality of corresponding individual collation data are stored in advance; a step of generating representative collation data corresponding to a representative appearance representing the plurality of individual appearances; and image data to be processed A representative collation step of collating the generated representative collation data with each other and calculating the similarity between the image corresponding to the image data and the representative appearance corresponding to the representative collation data, the image data to be processed, and the Similarity between the image corresponding to the image data and the individual appearance corresponding to the individual verification data by comparing with the individual verification data stored in the individual verification data storage means Whether to use a plurality of individual verification data stored in the individual verification data storage unit as a verification target in the individual verification step based on the individual verification step for calculating the similarity and the similarity calculated by the representative verification unit And an individual collation target selection step for determining whether to exclude from the collation target in the individual collation step.

By this method, whether or not the image data and the plurality of individual verification data need to be verified is determined based on the similarity between the processing target image data and the representative verification data. Data can be excluded from collation targets at once.
More specifically, as described in claim 9, the individual collation target selecting step includes the individual collation when the similarity calculated in the representative collation step is larger than a predetermined threshold value. A plurality of individual collation data stored in the data storage means are used as collation targets in the individual collation step, and when the similarity calculated in the representative collation step is equal to or less than the threshold value, it is stored in the collation data storage means. Preferably, the method further includes a step of excluding a plurality of individual verification data from the verification target in the individual verification step.

  The invention according to claim 10 is an image recognition method for processing image data and recognizing a recognition object in an image corresponding to the image data, and having a plurality of individual appearances that the recognition object can take. A step of preparing individual collation data storage means in which a plurality of corresponding individual collation data are classified and stored for each group, and individual collation data of a plurality of groups stored in the individual collation data storage means, respectively The step of generating a plurality of representative collation data, the image data to be processed and the plurality of generated representative collation data are collated, respectively, and an image corresponding to the image data and the plurality of representative collation data are respectively identified. A representative collation step for calculating a similarity with a plurality of corresponding representative appearances, an image corresponding to the image data to be processed, and the individual collation data storage means An individual verification step for calculating the similarity with the individual appearance corresponding to the stored individual verification data, and the individual verification data based on the similarity calculated for each of the plurality of representative verification data in the representative verification step A group of individual collation data to be collated with the image data to be processed in the individual collation step is selected from a plurality of groups of individual collation data stored in the storage means, and individual collation data of the group not selected Including an individual collation target selection step of excluding from a collation target in the individual collation step.

According to this method, the individual verification data that requires individual verification and the individual verification data that does not require individual verification can be separated in groups by comparing the image data to be processed with the representative verification data. As a result, even when there is a large amount of individual collation data, it is possible to suppress the individual collation processing calculation to a small amount, thereby enabling accurate image recognition processing in a short time.
More specifically, as described in claim 11, the individual collation target selection step corresponds to representative collation data in which the similarity calculated in the representative collation step is larger than a predetermined threshold value. It is preferable to include a step of selecting a group, setting individual collation data belonging to this group as a collation target in the individual collation step, and excluding individual collation data of the remaining group from the collation target in the individual collation step.

  In addition, as described in claim 12, the individual collation target selection step corresponds to representative collation data in which the similarity calculated in the representative collation step takes a value within the upper predetermined rank or the upper predetermined ratio. It is preferable to include a step of selecting a group to be selected, selecting individual collation data belonging to this group as a collation target in the individual collation step, and excluding individual collation data of the remaining group from a collation target in the individual collation step. .

  In a thirteenth aspect of the present invention, it is determined whether or not a maximum similarity that is a maximum value of the plurality of similarities calculated in the individual collating step for a plurality of individual collating data is greater than a predetermined recognition threshold value. A step of outputting a recognition result corresponding to the individual collation data of the highest similarity when the highest similarity is larger than the recognition threshold; and the highest similarity is equal to or less than the recognition threshold In this case, the method further includes an individual collation target reselecting step of adding at least a part of the individual collation data excluded from the collation target by the individual collation target selection step to the collation target in the individual collation step. An image recognition method according to any one of claims 10 to 12.

  By this method, when the maximum similarity regarding the individual collation data does not exceed the recognition threshold value, the individual collation data once excluded from the individual collation target is restored as the individual collation target on a group basis. Accordingly, it is possible to avoid a situation in which correct individual verification data is excluded from the individual verification targets by the verification processing related to the representative verification data, thereby realizing a reliable recognition process. “Correct individual collation data” is individual collation data corresponding to individual images whose similarity to the processing target image exceeds the recognition threshold value.

  In the invention according to claim 14, in the individual collation target reselecting step, the similarity calculated in the representative collation step is the highest among the groups of individual collation data excluded from the collation targets in the individual collation step. 14. The image recognition method according to claim 13, further comprising a step of adding individual collation data of a group corresponding to high representative collation data to a collation target in the individual collation step.

By this method, when the individual collation data once excluded from the individual collation targets is restored in units of groups, a group having a high degree of similarity obtained with respect to the representative collation data is preferentially selected. As a result, correct individual verification data can be quickly found out of the individual verification data once excluded.
The invention according to claim 15 is a computer program for operating a computer as an image recognition device for processing image data and recognizing a recognition object in an image corresponding to the image data. Comprises individual collation data storage means in which a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition object are stored in advance, and the computer program represents the plurality of individual appearances. Representative collation data generating means for generating representative collation data corresponding to the representative appearance, image data to be processed and representative collation data generated by the representative collation data generation means are collated, and the image corresponding to the image data and the representative Representative collation means for calculating the similarity with the representative appearance corresponding to the collation data, the image data to be processed Individual collation means for collating the individual collation data stored in the individual collation data storage means, and calculating the similarity between the image corresponding to the image data and the individual appearance corresponding to the individual collation data; and Based on the similarity calculated by the representative collating unit, a plurality of individual collation data stored in the individual collation data storage unit are to be collated by the individual collating unit or excluded from collation targets by the individual collating unit. A computer program that functions as an individual collation control means for determining whether or not to perform.

By executing this computer program on a computer, the image recognition apparatus of claim 1 can be configured.
According to a sixteenth aspect of the present invention, when the similarity calculated by the representative verification unit is greater than a predetermined threshold, the individual verification control unit is configured to store the image data to be processed and the individual verification data storage unit. When the similarity calculated by the representative verification unit is less than or equal to the threshold value, each of the individual verification data stored in is stored in the verification data storage unit. 16. The computer program according to claim 15, wherein the plurality of individual verification data are excluded from the verification targets by the individual verification means.

By executing this computer program on a computer, the image recognition apparatus of claim 2 can be configured.
The invention according to claim 17 is a computer program for operating a computer as an image recognition device for processing image data and recognizing a recognition object in an image corresponding to the image data, the computer Comprises individual collation data storage means for classifying and storing a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition target object for each of a plurality of groups, and the computer program Representative collation data generating means for generating a plurality of representative collation data representing the individual collation data of a plurality of groups stored in the collation data storage means, a plurality of representative collations generated by the image data to be processed and the representative collation data Data is collated, and the image corresponding to the image data and the plurality of representative collation data are Representative collating means for calculating the degree of similarity with a plurality of representative appearances respectively corresponding to the image, the similarity between the image corresponding to the image data to be processed and the individual appearance corresponding to the individual collation data stored in the individual collation data storage means Individual collating means for calculating the degree, and based on the similarity calculated for each of the plurality of representative collation data by the representative collating means, a plurality of groups of individual collation data stored in the individual collation data storage means Individual collation control for selecting a group of individual collation data to be collated with the image data to be processed by the individual collation means and excluding individual collation data of the unselected group from collation targets by the individual collation means A computer program characterized by functioning as a means.

By executing this computer program on a computer, the image recognition apparatus of claim 3 can be configured.
According to an eighteenth aspect of the present invention, the individual verification control unit selects a group corresponding to representative verification data whose similarity calculated by the representative verification unit is larger than a predetermined threshold value, and belongs to this group. 18. The computer program according to claim 17, wherein the individual collation data is a collation target by the individual collation means, and the individual collation data of the remaining group is excluded from the collation targets by the individual collation means.

By executing this computer program on a computer, the image recognition apparatus of claim 4 can be configured.
According to a nineteenth aspect of the present invention, the individual verification control unit selects a group corresponding to representative verification data in which the similarity calculated by the representative verification unit takes a value within the upper predetermined rank or the upper predetermined ratio. 19. The individual collation data belonging to this group is set as a collation target by the individual collation means, and the individual collation data of the remaining group is excluded from the collation targets by the individual collation means. It is a computer program.

By executing this computer program on a computer, the image recognition apparatus of claim 5 can be configured.
In the invention according to claim 20, the computer program further has a maximum similarity that is a maximum value of a plurality of similarities calculated by the individual verification unit for a plurality of individual verification data. Means for judging whether or not the recognition threshold value is larger than a recognition threshold value, and when the highest similarity is larger than the recognition threshold value, a recognition result output means for outputting a recognition result corresponding to the individual matching data of the highest similarity degree When the highest similarity is less than or equal to the recognition threshold, at least a part of the individual verification data excluded from the verification target by the individual verification control unit is added to the verification target by the individual verification unit, and this individual verification The function as a re-individual collation control means for performing the individual collation processing by the means, Is a computer program according to.

By executing this computer program on a computer, the image recognition apparatus of claim 6 can be configured.
The invention according to claim 21 is characterized in that the computer further includes representative appearance similarity storage means for storing a plurality of similarities obtained by the representative matching means, and the re-individual matching control means is based on the individual matching means. Among the individual collation data groups excluded from the collation target, individual collation data of the group corresponding to the representative collation data having the highest similarity stored in the representative appearance similarity storage unit is obtained by the individual collation unit. 21. The computer program according to claim 20, wherein the computer program is added to a verification target.

By executing this computer program on a computer, the image recognition apparatus of claim 7 can be configured.
A twenty-second aspect of the present invention is a computer-readable recording medium on which the computer program according to any of the fifteenth to twenty-first aspects is recorded.
By causing a computer to execute the computer program recorded on the recording medium, the computer can function as the image recognition apparatus as described above.

  The computer program may be recorded on the recording medium in any format, for example, may be recorded in a compressed format. As the recording medium, any recording format including a magnetic recording medium, an optical recording medium, and a magneto-optical recording medium can be applied.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram showing a configuration of a personal authentication system according to an embodiment of the present invention. This personal authentication system is used, for example, for entrance management to a specific area where entry is restricted. This personal authentication system is arranged at the entrance to the restricted access area, and has a camera 1 as an imaging device that captures a face image of a person appearing at the entrance, and a computer connected to the camera 1 via a cable 2. 3, a display 4 disposed at the entrance, and a gate switch 7 that opens and closes a gate installed at the entrance (or unlocks / locks the door lock).

  The computer 3 is connected to a display 5 as a man-machine interface and an input operation unit 6 (for example, a keyboard and a mouse). The display device 4 may be a two-dimensional display device that can display a message in response to a command signal from the computer 3, for example, and includes a plurality of color LED lamps controlled by the computer 3. It may be a lamp display device. The gate switch 7 opens and closes the gate (or unlocks / locks the door lock) in accordance with a control signal from the computer 3 to permit or prohibit entry of a person who appears at the entrance.

  The computer 3 includes a CPU 10, a ROM 11 and a RAM 12, and has a general configuration in which these are connected to a bus 13. The bus 13 further includes a hard disk drive (HDD) 14 as an external storage device, a CD-ROM drive 15 as a recording medium reader capable of reading data recorded on a CD-ROM 8 which is an example of a recording medium, A display controller 16 for controlling the display 5 and an input unit controller 17 (for example, a keyboard controller and a mouse controller) for controlling the input operation unit 6 are connected. The image input interface unit 18 for receiving image data corresponding to the captured image generated from the camera 1, the display control interface unit 19 for outputting a control signal to the display 4, and the control signal to the gate switch 7 Is connected to the bus 13.

  The face image of the person permitted to enter the restricted access area is captured in advance by the camera 1, for example, and the individual verification data corresponding to the face image is a predetermined storage area in the hard disk drive 14 in advance. Registered in the storage unit 25. As will be described later, representative verification data representing individual verification data of each person is also registered in the verification data storage unit 25.

  On the other hand, an image of a person who appears at the entrance is captured by the camera 1. Image data representing the captured image is transmitted to the computer 3, and the person's face image data is extracted from the image data. The extracted face image data is collated with the individual collation data registered in advance in the collation data storage unit 25 of the hard disk drive 14. As a result of this collation, if any face image registered in the form of the individual collation data matches the face image of the person appearing at the entrance, that is, the recognition threshold value determined by the similarity between both face images is set in advance. If it exceeds, the computer 3 opens the gate (or unlocks the door lock) by the gate switch 7 and permits the person to enter.

  A computer program for realizing such an operation by the computer 3 may be provided by being recorded on a CD-ROM 8 which is an example of a recording medium. In this case, by loading the CD-ROM 8 into the CD-ROM drive 15, performing a predetermined installation operation, and installing the computer program in the computer 3, the computer 3 can realize the functions as described above. Become.

  When the installed computer program is executed on the computer 3, for example, a screen as shown in FIG. The computer 3 executing the computer program operates as an image recognition device, extracts a face image from a registration mode for registering a face image of a person permitted to enter, and an image captured by the camera 1. It operates in one of the recognition modes for performing recognition processing.

  When registering individual verification data of a face image of a person permitted to enter the restricted access area in the verification data storage unit 25, the operator operates the input operation unit 6 to operate the computer 3 in the registration mode. Let In this state, a person who is permitted to enter is imaged by the camera 1. At this time, the image data generated from the camera 1 is transmitted to the computer 3 and stored in the image memory 12a constituted by a storage area in the RAM 12 (and a virtual memory area in the hard disk drive 14 as necessary). . Further, face image data is extracted from the image data. Individual collation data corresponding to the extracted face image data is registered in the collation data storage unit 25. The individual verification data may be face image data itself or feature amount data representing a predetermined feature amount extracted from the face image data.

  Human face images change in various ways depending on facial expressions, change greatly depending on the presence or absence of accessories such as glasses, and are also affected by imaging conditions (such as illuminance). Therefore, when registering the individual verification data of a person who is allowed to enter, various facial expressions can be taken for the person, and a face image can be taken, and images can be taken with and without accessories. Imaging is performed under imaging conditions. In this way, a plurality of (for example, 50 to 100) face images are captured for each person, and individual matching data corresponding to each face image is registered in the matching data storage unit 25.

Hereinafter, a person who has completed the registration of the individual verification data as described above is referred to as a “registrant”, and a person who has not registered the individual verification data is referred to as an “unregistered person”.
FIG. 3 is a block diagram for explaining a functional configuration of the computer 3 in the execution state of the computer program. The computer 3 substantially includes a plurality of function processing units realized by the CPU 10 cooperating with the ROM 11 and the RAM 12 by causing the CPU 10 to execute the computer program. The plurality of function processing units extract a face image area from the image data generated by the camera 1 to generate face image data, and the face image data extracted by the face image detection unit 30. A face recognition processing unit 40 for collating with individual collation data registered in advance is provided.

  The face image detection unit 30 cuts out a face image area from the input image data, and further outputs face image data normalized in size and inclination (left and right inclination) as face image data to be verified. The face image area to be cut out is an area that is not easily affected by the hairstyle and is not easily affected by the vertical movement of the jaw during speech. More specifically, for example, a rectangular region including the range from the eyebrows to the mouth is included as a face image region including the space between both eyes in the left-right direction and the range from the eyebrows to the mouth in the up-down direction. The normalized face image data corresponds to, for example, a square image of 32 pixels × 32 pixels.

  The face recognition processing unit 40, if a certain degree of agreement is found between the face image data to be collated and any individual collation data, the face of the registrant corresponding to the individual collation data corresponds to the face image data to be collated. Judged to be an image. In this case, it is determined that the person imaged by the camera 1 is a registered person, and the computer 3 opens the gate (or unlocks the door lock) by the gate switch 7 and permits the person to enter. On the other hand, the face recognition processing unit 40 determines that the face image to be checked is the face image of the registrant unless individual matching data that matches a certain degree of matching with the face image data to be checked is registered in the matching data storage unit 25. Judge that there is no. In this case, it is determined that the person imaged by the camera 1 is an unregistered person, and the computer 3 holds the gate in the closed state (or holds the door lock in the locked state) by the gate switch 7. Is prohibited from entering.

  The face recognition processing unit 40 calculates a similarity (similarity of person) between a face image to be verified and a face image corresponding to the verification data (individual verification data or representative verification data), and a verification data storage. A collation data selection unit 42 for selecting which collation data is to be collated among the collation data registered in the unit 25, and the similarity calculated by the similarity calculation unit 41 as a predetermined recognition threshold value. And a determination unit 43 that determines whether the face image to be compared is a face image of the registrant by comparing.

  If the highest value (highest similarity) among the similarities obtained for the plurality of individual verification data exceeds the recognition threshold, the determination unit 43 determines that the person of the face image to be verified is a registrant. It is judged that there is, and recognition result data indicating that is output. The recognition result data may include information (for example, name information and registration number information of the registrant) for specifying the registrant corresponding to the individual matching data with the highest similarity. Receiving the recognition result data, the display unit 4 displays the recognition result. This display may simply be a display indicating that the user is a registrant, or may be a display of specific information (for example, name) of the registrant.

On the other hand, if there is no individual verification data whose similarity exceeds the recognition threshold value, the determination unit 43 determines that the person of the face image to be verified is not a registrant, and outputs identification result data indicating that fact. The In this case, the display 4 displays that the person's registration cannot be confirmed.
The similarity calculating unit 41 is based on the collation data stored in the collation data storage unit 25 and the collation feature dictionary stored in the collation dictionary storage unit 26 configured by the storage area in the hard disk drive 14. The similarity between the face image to be collated and the registered face image (stored in the form of collation data) is calculated. The matching feature dictionary holds data (matching feature data) serving as an index when calculating the similarity between two images in a fixed data structure. The similarity calculation unit 41 extracts a feature amount from the face image data to be collated based on the collation feature data held in the collation feature dictionary, and collates it with the collation data.

  FIG. 4 is an illustrative view for explaining a registration state of collation data in the collation data storage unit 25. FIG. 4 shows an example in which there are m registrants and n individual verification data corresponding to n face images are registered for each registrant. More specifically, registration numbers 1, 2,..., M are assigned to individual registrants, and registrant names (name 1, name 2,. Name m), each representative collation data (representative collation data 1, representative collation data 2,..., Representative collation data m), and each n individual collation data (individual collation data 1-1, 1- 2, ..., 1-n; 2-1, 2-2, ..., 2-n; ...; m-1, m-2, ..., mn) are registered. Of course, the number of individual verification data may differ for each registrant.

The representative collation data of each registrant is collation data representing n pieces of individual collation data corresponding to the registrant. More specifically, the representative collation data of each registrant may be average value data of n pieces of individual collation data registered corresponding to the registrant.
Such representative collation data may be automatically generated by the operation of the CPU 10 or the like and written to the collation data storage unit 25 when the individual collation data is registered in the collation data storage unit 25. The representative collation data is created by an operator's manual operation using registered face image data or individual collation data, and is also written into the collation data storage unit 25 by the operator's manual operation. Also good.

  FIG. 5 is a diagram more intuitively showing the registration state of the collation data in the collation data storage unit 25. In this example, for each of nine registrants A to I, one representative face image data is registered as representative collation data, and each three face image data is registered as individual collation data. ing. As in this example, a plurality of face image data is registered as individual verification data for each registrant, and representative face image data obtained by calculating an average value of the plurality of face image data is registered as representative verification data. Also good. Also, instead of the image data itself, feature amounts used for similarity calculation by the similarity calculation unit 41 are extracted in advance from individual face images, and the feature amounts are registered in the verification data storage unit 25 as individual verification data. Alternatively, the average value of the individual verification data (feature value) for each registrant may be registered in the verification data storage unit 25 as representative verification data.

As described above, the collation data storage unit 25 functions as individual collation data storage means for storing individual collation data and representative collation data storage means for storing representative collation data (an example of representative collation data generation means). Yes.
FIG. 6 is a flowchart for explaining an example of processing by the face recognition processing unit 40. When the face image data to be collated is input to the face recognition processing unit 40, the collation data selection unit 42 refers to the collation data storage unit 25 and sequentially selects the representative collation data of the registrant (step S1). The similarity calculation unit 41 calculates the similarity between the representative face image corresponding to the representative collation data selected by the collation data selection unit 42 and the face image to be collated, and the representative similarity storage area 12b (see FIG. 1 and Fig. 3. Write to representative appearance similarity storage means (step S2). The functions executed in these steps S1 and S2 correspond to the function of the representative verification unit (representative verification step) of the present invention.

  Thus, when the similarity is obtained for the representative collation data of all registrants registered in the collation data storage unit 25, the collation data selection unit 42 extracts individual collation data to be collated with the face image data to be collated. Processing is executed (step S3). This process is simply a process for excluding the individual collation data of the registrant who is considered to be clearly not coincident with the face image data to be collated from the collation target. This corresponds to the operation of the selection step.

  One specific example of the processing content for extracting the individual collation data to be collated is shown in FIG. In this example, the collation data selection unit 42 compares the similarity obtained for the representative collation data with a predetermined rejection threshold (step S11). The matching data selection unit 42 excludes (rejects) the registrant corresponding to the representative matching data whose similarity does not exceed the rejection threshold (rejected) (step S12). That is, a plurality of individual verification data registered in the verification data storage unit 25 for the registrant are collectively rejected. As a result, only the registrant corresponding to the representative collation data whose similarity exceeds the rejection threshold value is subjected to collation (step S13). The collation data selection unit 42 performs the same process for all the representative collation data (step S14), and ends the extraction of the individual collation data to be collated.

  Another example of the process for extracting the individual collation data to be collated is shown in FIG. In this example, the collation data selection unit 42 ranks (sorts) the representative collation data of all registrants stored in the representative similarity storage area 12b (step S15), and registers a predetermined number or a predetermined ratio of the higher rank. For the person, the individual collation data is extracted as a collation target (step S16). The collation data selection unit 42 collectively excludes the plurality of individual collation data of the remaining registrants from the collation target (step S17). In this case as well, if there is a similarity within the upper predetermined number or within the upper predetermined ratio that does not exceed the predetermined rejection threshold, the individual verification data of the registrant corresponding to such similarity May be excluded from verification targets.

  The collation data selection unit 42 sequentially selects the individual collation data of a part of the registrants extracted in this way one by one as the target of the collation calculation by the similarity calculation unit 41 (step S4 in FIG. 6). . The similarity calculation unit 41 calculates the similarity between the face image corresponding to the selected individual verification data and the face image to be verified. Thus, the similarity is calculated for all the individual verification data extracted by the verification data selection unit 42 (step S5). The calculated similarity is written in the individual similarity storage area 12 c in the RAM 12. Thus, the function executed in steps S4 and S5 corresponds to the function of the individual verification means (individual verification step) of the present invention.

  Next, the determination unit 43 refers to the individual similarity storage area 12c and searches for the highest value (highest similarity) of the calculated similarity (step S6). Then, the determination unit 43 compares the retrieved maximum similarity with a predetermined recognition threshold value (step S7). As a result, if the highest similarity exceeds the recognition threshold value (YES in step S6), the determination unit 43 determines that the collated face image is a registrant's face image, and the highest similarity is determined. The registrant corresponding to the individual collation data is identified as the person of the face image to be collated (step S8). On the other hand, when the highest similarity is not more than the recognition threshold value, it is determined that the person of the face image to be verified is not registered (step S9). The function of step S8 by the determination unit 43 corresponds to the function of the recognition result output means of the present invention.

  In this way, the individual collation data to be collated can be narrowed down by collating the face image data to be collated with the representative collation data. Thereby, the collation calculation of the face image data to be collated and the individual collation data can be greatly reduced, and the time required for the recognition process can be greatly shortened. Also, when the number of individual verification data per registrant is increased to improve recognition accuracy, the time required for recognition does not increase significantly. Therefore, the face image can be accurately recognized by a short time process.

  FIG. 9 is a flowchart for explaining another example of processing by the face recognition processing unit 40. When the face image data to be collated is input to the face recognition processing unit 40, the collation data selection unit 42 refers to the collation data storage unit 25 and sequentially selects the representative collation data of the registrant (step S21). The similarity calculation unit 41 calculates the similarity between the representative image corresponding to the representative collation data selected by the collation data selection unit 42 and the face image to be collated, and writes it in the representative similarity storage area 12b (step S22). The functions executed in these steps S21 and S22 correspond to the function of the representative verification means (representative verification step) of the present invention.

Thus, when the similarity is obtained for the representative collation data of all registrants registered in the collation data storage unit 25, the collation data selection unit 42 executes a process for extracting individual collation data to be collated. This process is simply a process for excluding the individual collation data of the registrant who is considered not to agree with the face image data to be collated, from the collation target.
More specifically, the collation data selection unit 42 refers to the representative similarity storage area 12b and sorts the representative collation data of all registrants in descending order according to the similarity (step S23). Furthermore, the collation data selection part 42 divides a registrant into a some group according to the order of similarity (step S24). For example, if collation data as in the example of FIG. 5 is registered in the collation data storage unit 25, the three people with the highest similarity are classified into the first group, and the three people with the middle similarity are the first. You may classify | categorize into 2 groups and classify 3 people with low similarity into a 3rd group. Of course, the grouping of registrants based on similarity does not have to be equal among a plurality of groups.

  The collation data selection unit 42 further extracts the individual collation data of the registrant of the highest group (for example, the first group described above) as a collation target (step S25), and the other groups (second and second in the above example). Individual collation data of registrants of the third group is provisionally excluded from collation targets (step S26). The processing of these steps S25 and 26 corresponds to the function of the individual collation control means (individual collation target selection step) of the present invention.

  The collation data selection unit 42 sequentially selects the individual collation data of the registrants of the highest group extracted in this manner one by one as a collation target (step S27). The similarity calculation unit 41 calculates the similarity between the face image corresponding to the selected individual verification data and the face image to be verified. Thus, the similarity is calculated for all the individual collation data of the registrants in the highest group (step S28). The calculated similarity is written in the individual similarity storage area 12 c in the RAM 12. The processing of these steps S27 and 28 corresponds to the function of the individual verification means (individual verification step) of the present invention.

  Next, the determination unit 43 refers to the individual similarity storage area 12c and searches for the maximum value (highest similarity) of the calculated similarity (step S29). Then, the retrieved maximum similarity is compared with a predetermined recognition threshold (step S30). As a result, if the maximum similarity exceeds the recognition threshold value (YES in step S30), the determination unit 43 determines that the face image to be verified is a registrant's face image, and determines the highest similarity. The registrant corresponding to the individual verification data is specified as the person of the face image to be verified (step S31). The processing in step S31 corresponds to the function of the recognition result output means of the present invention.

  On the other hand, when the highest similarity is equal to or less than the recognition threshold value (NO in step S30), the collation data selection unit 42 collates the individual collation data for all registrants registered in the collation data storage unit 25. It is determined whether or not it has been completed (step S32). This determination is equivalent to the determination of whether or not the collation processing has been completed for all the groups in which the registrant is classified according to the similarity of the representative collation data. When the collation process for any registrant is not completed (NO in step S32), the collation data selection unit 42 is the highest group among the groups that have been temporarily excluded from the candidates (that is, The registrant belonging to the next group (for example, the second group in the above example) is set as a candidate for collation, and the processing from step S27 is repeated. That is, the individual verification data of the registrants of the next group are sequentially selected. The process of step S33 corresponds to the function of the re-individual collation control means (individual collation target reselection step) in the present invention.

Similar to the case described above, the similarity calculation unit 41 calculates the similarity between the face image corresponding to the selected individual verification data and the face image to be verified. Thus, the similarity is calculated for all the collation data of the registrants of the next group (step S28).
Next, the determination unit 43 searches for the highest calculated similarity (the highest similarity) (step S29). Then, the retrieved maximum similarity is compared with the recognition threshold (step S30). As a result, if the maximum similarity exceeds the recognition threshold value, the registrant corresponding to the individual matching data of the similarity is specified as the person of the face image to be compared (step S31).

  In this way, the same processing is performed until individual collation data having a similarity exceeding the recognition threshold is found. If the individual collation data whose similarity is higher than the recognition threshold value is not found even though the collation processing is performed on all the individual collation data stored in the collation data storage unit 25 ( The determination unit 43 determines that the person of the face image to be verified has not been registered (step S34).

In the process shown in FIG. 9, when an appropriate recognition result cannot be obtained, the individual verification data once rejected is restored as a verification target. As a result, it is possible to reliably avoid a situation in which the registrant is not recognized.
In the example of FIG. 9, when it is determined that the person of the face image to be verified is unregistered, the verification process for the individual verification data is performed for all of the registrants. However, when the similarity obtained as a result of matching between the face image data to be matched and the representative matching data is extremely low, the probability that a similarity higher than the recognition threshold value obtained by matching with the individual matching data is low. Therefore, for the group whose similarity calculated for the representative face image data does not reach the predetermined rejection threshold value, the individual collation data of the registrant may not be revived as a collation target.

  In this embodiment, an example is shown in which one representative collation data is registered for one registrant, but a plurality (two or more) of representative collation data are collated for one registrant. It may be registered in the data storage unit 25. For example, in the case of a registrant who uses spectacles, first representative collation data representing individual collation data in the state of wearing spectacles and second representative collation data representing individual collation data in a state of wearing no spectacles. And may be registered. Also in this case, it is only necessary to determine whether or not a corresponding group of individual collation data is to be collated for each representative collation data based on the similarity obtained for the representative collation data.

Next, calculation of the similarity of face images will be described.
As shown in FIG. 10, a case where the similarity between the first face image I ¹ and the second face image I ² whose size and inclination are normalized will be described. The first face image I ¹ is, for example, a registered face image corresponding to the individual collation data stored in the collation data storage unit 25, and the second face image I ² is a face from an image captured by the camera 1, for example. This corresponds to the face image data to be verified extracted by the image detection unit 30.

First, T feature regions A _j (j is a natural number of 1 ≦ j ≦ T) having different sizes, shapes, positions, and at least one of the types described later (T is a natural number of 2 or more), sequentially, It is set in common in the first face image I ¹ and the second face image I ² . The feature area A _j is an area that includes at least two pixels and has the same size as or smaller than the first and second face images I ¹ and I ² . As the shape of the feature region A _j , an arbitrary shape such as a rectangle (including a square), a circle, or a bowl shape can be selected, and even if the shape is different among the plurality of feature regions A _{1 to AT.} Good.

For example, assume that the shape of the feature region is a square, the first and second face images I ¹ and I ² are squares of 32 pixels × 32 pixels, and the feature region is divided into two parts for analysis (ie, Assuming a case where the number of divisions p _j described later is fixed to “2”), the minimum feature area is a square having a size of 2 pixels × 2 pixels, and the maximum feature area is a size of 32 pixels × 32 pixels. In addition, it is allowed only when the size is the even number of pixels (represented by the number of pixels along one side of the square area). However, since it has been empirically known that the feature area of 2 pixels × 2 pixels is too small to be meaningful, the size of the minimum feature area is actually 4 pixels × 4 pixels. The feature region of 4 pixels × 4 pixels can take 29 × 29 different positions within the square region of 32 pixels × 32 pixels, and the feature region of 6 pixels × 6 pixels of the next size is 32 pixels × 32 27 × 27 different positions can be taken within the square area of the pixel. Based on the same consideration, a total of 4495 feature areas can be set in the first and second face images I ¹ and I ^{2 of} 32 pixels × 32 pixels. Further, a separate feature region is distinguished between the case where the region having the same position and size is divided into two in the horizontal direction (horizontal direction) x and the case where the analysis is divided into two in the vertical direction (vertical direction) y. In this case, a total of 8990 feature areas can be set. Some or all of these feature regions are used as feature regions A _j (j = 1, 2,..., T) for calculating similarity.

When the feature region A _j is set, next, p _j (p _j is a natural number of 2 or more) partial regions a _k (k is 1 ≦ k ≦) in the feature region A _j in the first face image I ¹ . The region value x ¹ _k in the natural number of p _j and the region value x ² _k in the corresponding p _j partial regions a _k in the feature region A _j in the second image I ² are obtained. The number of divisions p _j of the feature area A _j may be different for each feature area, and the value p (= p ₁ = p ₂ = …… = p _T ) common to the feature areas A _{1 to} A _T. Also good.

The partial area a _k, for example, FIG. 11 (a), the region (Fig obtained by equally dividing the inside characteristic region A _j to p _j pieces vertically horizontal dividing line 55 along the horizontal direction x 11 (a) may be a rectangular area long in the x direction). Further, as shown in FIG. 11B, the partial area a _k is an area obtained by equally dividing p _j in the horizontal direction by a vertical dividing line 56 along the vertical direction y in the characteristic area A _j (FIG. 11). (b) may be a rectangular region that is long in the y direction. Further, the division direction of the regions may be different among the plurality of feature regions A ₁ to _AT . In the following, feature regions with different region division directions are treated as different feature regions even if they have the same shape, size, and position.

Domain values x ¹ _k in the partial area a _k, x ² _k, for example, an average value of data of a plurality of pixels included in the partial area a _k (average pixel value) of the partial region a _k It corresponds to the luminance value. However, when the partial area _ak includes only one pixel (for example, in the case of a feature area of 2 pixels × 1 pixel), the data of the pixel itself becomes the area value of the partial area _ak . Domain values x ¹ _k, x ² _k of the partial regions a _k of feature areas A _j is the feature amount of the feature region A _j first and second face images related to I ^1, I ^2.

When the above-described region values x ¹ _k and x ² _k are obtained for the p _j partial regions a _k , the correlation in the feature region A _{j of} the first and second face images I ¹ and I ² is then obtained. It is done. More specifically, the correlation value C _j is calculated by the following equation.

However, when the division numbers p _j in all the feature regions A _{1 to} A _T are all equal, the coefficient 1 / p _{j on the} right side may be omitted.
The operation for obtaining the correlation between the images may be performed using a partial region obtained by dividing the feature region in two directions of the x direction and the y direction (that is, dividing in a lattice shape). However, in this embodiment, an operation for obtaining the correlation is performed using the partial region a _k obtained by dividing the feature region A _j in either the x direction or the y direction. Hereinafter, the correlation obtained in this way is referred to as “unidirectional correlation”.

Such unidirectional correlate seek operation, the are performed on a plurality of characteristic regions A ₁ to A _T, thereby, the correlation values _{C 1, C 2, ......,} C T is determined.
The correlation value C _j of each feature area A _j thus obtained is applied to a weak classifier (Weak Classifier) h _j (I ¹ , I ² ) defined for each feature area A _j .

That is, the correlation value C _j for each feature area A _j is compared to a predetermined threshold value theta _j determined in advance for each feature region A _j. As a result, if the correlation value C _j exceeds the threshold value θ _j , the first identification value “+1” (1) indicating that the first and second face images I ¹ and I ² are face images of the same person. A weak hypothesis (Weak Hypothesis) is generated. On the other hand, if the correlation value C _j is equal to or _smaller than the threshold value θ _j , the second identification value “−1” (1) indicating that the first and second face images I ¹ and I ² are not the same person's face image. A weak hypothesis representing a negative conclusion) is generated.

When the output values h ₁ (I ¹ , I ² ) to h _T (I ¹ , I ² ) (weak hypothesis) of the weak classifiers for each of the _T feature regions A _{1 to} A _T are obtained, Next, these are multiplied by a weighting coefficient w _j determined for each feature region A _j and added (linear combination). Thereby, the similarity (likeness of the same person) between the first and second face images I ¹ and I ² is obtained. That is, the similarity is given by the following equation.

The selection of the feature regions A _{1 to} A _T applied to the calculation of the unidirectional correlation value C _j , the threshold value θ _j in the weak classifier, and the weighting coefficient w _j are determined by prior learning. For this learning, for example, a boosting learning algorithm (AdaBoost) described in Non-Patent Document 1 is applied.
Specifically, as shown in the flowchart of FIG. 12, a plurality of pairs of different face image data of the same person are prepared in advance as positive samples (samples in which the conclusion that they are the same person is correct), Prepare multiple pairs of face image data of other people as negative samples (samples where the conclusion that they are the same person is incorrect), and as an initialization process, weight these samples uniformly (equally). (Weight) is distributed (step S50).

Next, a feature region A (for example, a square feature region) is set in the normalized face image data, and the unidirectional correlation value C expressed by the above equation (1) is set to all samples (positive samples). And negative samples). This calculation is executed with respect to all feature areas A (for example, square areas in which at least one of position, size, and division direction of the areas is different) that can be set in the normalized face image (step). S51). Then, T feature regions A _{1 to} A _T are selected, and T times of learning are performed to determine _T weak classifiers h ₁ (I ¹ , I ² ) to h _T (I ¹ , I ² ). (For t = 1 to T loop) is started (step S52).

  Specifically, first, all positive samples in which weights (uniform weights when t = 1) are given to each unselected individual feature area A (all feature areas A when t = 1). With respect to all negative samples, a histogram for each unidirectional correlation value C (weighted histogram corresponding to a sample given a weight) is created (step S53). An example of this histogram is shown in FIG.

In general, the distribution of negative samples appears in a relatively small range of correlation values C, and the distribution of positive samples appears in a relatively large range of correlation values C. The threshold value θ is determined so that the misidentification rate (error rate) when the threshold value and the correlation value C are compared is minimized (step S54).
For example, when the threshold θ is set to a large value θ _L , misidentification (False Accept error, false positive error) that an image pair (negative sample) of different persons is an image pair of the same person is Although the number is reduced, misidentification (False Reject error, false negative error) that the image pair (positive sample) of the same person is an image pair of different persons increases. Conversely, if the threshold value θ is set to a small value θ _S , misidentification (an error for accepting another person, a false positive error) that images pairs of different persons (negative samples) are image pairs of the same person increases. In addition, misidentification (person rejection error, false negative error) that an image pair (positive sample) of the same person is an image pair of different persons is reduced.

The threshold value θ is set to a value that minimizes the erroneous identification as a whole. In general, the threshold θ is a value between the distribution range of the positive sample and the distribution range of the negative sample, or an overlap interval thereof. Along with this threshold value θ, an erroneous identification rate corresponding to the threshold value θ is obtained.
In this way, by determining the threshold value θ of all unselected feature regions A, the weak classifier h corresponding to all the feature regions A is created.

Among these weak classifiers h, the weak classifier having the lowest misclassification rate (error rate) is the t-th classifier h _t (when t = 1, the first classifier h ₁ (feature area A _1). )) (Step S55), and a weighting coefficient w _t (weighting coefficient w ₁ when t = 1) corresponding to the misidentification rate is determined (step S56).
Then, the samples correct by the t weak classifier h _t is obtained given a small weight, redistribution of the weights so as to provide a large portion in the sample to be incorrect by the t weak classifier h _t (re- weighting), and learning from step S52 is performed on this (step S57). That is, the entire feature area unselected (feature region A the non-characteristic regions A ₁ to A _t), weighted histogram is generated for the sample in which the weight is given (step S53). Of the weak classifiers corresponding to the feature region A, the weak classifier having the smallest misclassification rate is set as the (t + 1) th classifier h _{t + 1} (step S55). That is, for example, a weak classifier that gives a correct answer with high probability to a sample that is easily misidentified by the first classifier h ₁ is selected as the second classifier h ₂ . Then, the first (t + 1) weighting factor w _{t + 1} according to the misidentification rate of weak classifier h _{t + 1} of is determined (step S56).

The same operation is repeated until _T weak classifiers h _{1 to} h _T are obtained, and T feature regions A _{1 to} A _T suitable for calculating similarity are selected by performing T learning. Will be. Then, by multiplying the _T weak discriminators h _{1 to} h _T by the respective weighting coefficients w _{1 to} w _T and linearly combining them, the similarity calculation formula according to the above-mentioned formula (3) is obtained.
Then, the similarity obtained by the equation (3) is compared with a predetermined recognition threshold value _Rth . When the obtained similarity exceeds the recognition threshold value R _th , it is determined that the first and second face images I ¹ and I ² are face images of the same person, and the recognition threshold value R _th or less. The first and second face images I ¹ and I ² are determined to be face images of different persons.

  That is, the determination result is given by the following equation.

FIG. 14 is a diagram for explaining the contents held in the collation dictionary storage unit 26 (see FIGS. 1 and 3). The collation dictionary storage unit 26 stores information on the feature areas A _{1 to} A _T corresponding to the similarity calculation expression of the expression (3). That is, corresponding to the feature numbers _{1 to T} , it is applied to the feature region designation information for designating the feature regions A _{1 to AT and} the correlation values C ₁ to CT of the feature regions A ₁ to _AT . and threshold theta ₁ through? _T is, the weighting factor is given and w ₁ to w _T is stored for weak classifiers h ₁ to h _T of the feature regions a ₁ to a _T.

The feature area designation information is information for designating the position, shape, size, and type of the feature area. FIG. 14 shows an example in which the shape of the feature region is fixed to a fixed shape (for example, a square) and the number of divisions p _j of the feature region is fixed to a fixed value p. Therefore, the position, size, and type are registered as the feature area designation information. The position of the feature region represents the position in the two face images I ¹ and I ² to be collated. Specifically, for example, as shown in FIG. 11, when the feature area is rectangular, the lower left corner is regarded as the origin, and the position of the feature area is represented by the x and y coordinates of the origin. The size is represented by, for example, the length of one side (number of pixels) in the case of a square feature region. The type is information indicating whether the feature region is divided in the x direction (horizontal direction) (see FIG. 11B) or divided in the y direction (vertical direction) (see FIG. 11A).

FIG. 15 is a block diagram for explaining a configuration example of the similarity calculation unit 41 (see FIG. 3). The similarity calculation unit 41 of this configuration example includes a first face image I ^{1 to be} collated (for example, a representative face image corresponding to representative collation data or a registered face image corresponding to individual collation data) and a second face image I ^2. A feature region setting unit 45 that sets a feature region A _{j in} common (for example, a face image to be checked captured by the camera 1), and a first feature amount calculation that calculates a feature amount of the first face image I ¹ Unit 46 and a second feature amount calculation unit 47 that calculates the feature amount of the second face image I ² .

The feature region setting unit 45 reads the feature regions A _{1 to AT} from the collation dictionary storage unit 26 and sequentially sets them in the first and second face images I ¹ and I ² .
The first feature quantity calculation unit 46 refers to the collation dictionary storage unit 26, and sets the region according to the “type” information for the feature region A _j in the first face image I ¹ set by the feature region setting unit 45. dividing the obtained area value x ¹ _k of that broken portion each area a _k as a feature quantity. Similarly, the second feature amount calculation unit 47 refers to the collation dictionary storage unit 26 and sets the “type” of the feature region A _j in the second face image I ² set by the feature region setting unit 45. dividing the area according to the information, it obtains the domain values x ² _k of that broken portion each area a _k as a feature quantity.

The similarity calculation unit 41 further includes a correlation value calculation unit 48 that calculates a correlation value C _j based on the region values x ¹ _k and x ² _k obtained by the first and second feature amount calculation units 46 and 47. The comparison unit 49 that compares the correlation value C _j obtained by the correlation value calculation unit 48 with the threshold value θ _j corresponding to the feature region A _j , and the comparison result h _j (I ¹ , A weighting addition unit 50 that multiplies I ² ) by a weighting coefficient w _j and adds the multiplication result.

The comparison unit 49 reads the threshold value θ _j corresponding to the feature region A _j with reference to the matching dictionary storage unit 26. If the correlation value C _j is larger than the threshold value θ _j , the comparison unit 49 has the first identification value “+1” (a weak hypothesis indicating that the first and second face images are face images of the same person). Is output. On the other hand, if the correlation value C _j is less than or equal to the threshold value θ _j , the comparison unit 49 determines that the second identification value “−1” (a weak hypothesis indicating that the first and second face images are the face images of others). ) Is output.

The weighting addition unit 50 multiplies the output value of the comparison unit 49 by a weighting coefficient corresponding to the feature region A _j , and linearly combines the output values of the comparison unit 49 for the feature regions A _{1 to} A _T to obtain the (3 The similarity represented by the formula is obtained and output. That is, the weighting addition unit 50 multiplies the comparison result h _j (I ¹ , I ² ) for the feature regions A _{1 to AT} given from the comparison unit 49 by the weighting coefficient w _j and adds them.

FIG. 16 is a block diagram for explaining another configuration example of the similarity calculation unit 41. In FIG. 16, the same functional parts as those shown in FIG. 15 are given the same reference numerals as those in FIG. The configuration of FIG. 16 is suitable when face image collation data (representative collation data and individual collation data) are registered in the collation data storage unit 25 in the form of feature amounts. In this case, the collation data storage unit 25 stores the feature amounts related to the feature areas A _{1 to AT} for each registered face image as individual collation data. Similarly, as representative collation data, feature quantities related to the feature areas A _{1 to} A _T for the representative face image are stored. These feature quantities are stored in the matching data storage unit 25 for each of the matching areas A _{1 to AT} for each matching data. The stored feature amount is a region value x ¹ _k of the partial region a _k obtained by dividing each feature region A _j .

The feature region setting unit 45A refers to the collation dictionary storage unit 26 and sequentially sets the feature regions A _{1 to AT} in the second face image I ² . The feature quantity calculation unit 47 refers to the collation dictionary storage unit 26 and calculates the feature value x ² _k for each partial area a _k for the feature area A _j set in the second face image I ² . Calculate as
The calculated region value x ² _k is given to the correlation value calculation unit 48. The correlation value calculation unit 48 reads the region value x ¹ _k corresponding to the feature region A _j of the first face image I ¹ from the collation data storage unit 25. The correlation value calculation unit 48 calculates the correlation value C _j based on the region value x ¹ _k read from the collation data storage unit 25 and the region value x ² _k calculated by the feature amount calculation unit 47. To do.

The functions of the comparison unit 49 and the weighting addition unit 50 are the same as those in the configuration of FIG.
In the case of this configuration example, for the first face image I ¹ (registered face image or representative face image), the region value x ¹ _k is obtained in advance and registered as collation data in the collation data storage unit 25. . Therefore, the amount of calculation for calculating the similarity can be reduced. Accordingly, the calculation time can be shortened, so that the time required for recognition of the face image can be shortened.

Next, the process by the face image detection unit 30 (see FIG. 3) will be outlined.
FIG. 17 is a functional block diagram for explaining a functional configuration of the face image detection unit 30. The face image detection unit 30 extracts a face image region from an input image captured by the camera 1 and generates face image data (matched face image data) whose size and inclination are normalized. The face image detection unit 30 executes a rough detection process as a first stage process and a fine detection process as a second stage process. In the rough detection process, the entire area of the image captured by the camera 1 is roughly scanned, and a part of the area that is likely to contain a face image is extracted as a peripheral area of the face image, and the remaining area is rejected. This is a process of excluding from the precise detection process target. In other words, the precision detection process is a process of extracting and cutting out a face image area within the face image peripheral area extracted by the rough detection process.

  In order to execute the rough detection process, the face image detection unit 30 includes a rough detection window setting unit 31 that sets a partial region in the captured image as a processing target region (rough detection window), and the rough detection window setting unit 31. A peripheral area identification unit 32 for identifying whether the coarse detection window set by the step is a face image peripheral area including a face image, and a peripheral area feature dictionary storing information for determination by the peripheral area identification unit 32 33. The peripheral area feature dictionary 33 is stored in a storage area in the hard disk drive 14 as shown in FIG.

  In addition, in order to execute the precision detection process, the face image detection unit 30 further sets a partial region of the face image peripheral region extracted by the peripheral region identification unit 32 as a processing target region (precision detection window). The precise detection window setting unit 34 for performing the detection, the face image identifying unit 35 for identifying whether the precise detection window set by the precise detection window setting unit 34 corresponds to the face image region, and the determination by the face image identifying unit 35 And a face image feature dictionary 36 storing information for. The face image feature dictionary 36 is stored in a storage area in the hard disk drive 14 as shown in FIG.

  Further, the face image detection unit 30 includes a face image output processing unit 37 that normalizes and outputs an image in the precision detection window determined to be a face image region by the face image identification unit 35. The face image output processing unit 37 normalizes the image in the precise detection window to a certain size (for example, 32 pixels × 32 pixels), corrects the inclination of the face image in the precise window, and Collation face image data is generated.

  The face image extracted from the captured image is a rectangular area having a predetermined shape inside the outer corners of the human face, such as three images in the center row of FIG. 18 (three images arranged in the vertical direction). (A region having the same shape as the image to be collated by the face recognition processing unit 40. For example, a square region). That is, face images included in various sizes in the captured image are cut out by a rectangular region having a constant aspect ratio. The face image in the middle row in FIG. 18 shows a case where the line segment connecting both eye corners is substantially parallel to the horizontal side of the rectangular area (0 degree), and the face image in the upper row in the middle row in FIG. In contrast, the face image in the lower row of the center row in FIG. 18 is rotated right (+5 degrees) by 5 degrees with respect to the image in the middle row of the same row.

  In this case, the face image feature dictionary 36 holds information for identifying a rectangular image as shown in the center column of FIG. 18 as a face image. More specifically, information regarding the face image detection filter corresponding to the feature pattern of the face image is held in the face image feature dictionary 36. For example, as shown in FIG. 19, the face image detection filter has, for each of a pair of rectangular areas 71 and 72 set in the processing window 70, an average luminance value of each of the plurality of pixels included in the areas 71 and 72. And the difference between the average luminance values is compared with a predetermined threshold value.

  The positions of the rectangular regions 71 and 72 in the processing window 70, the shape (vertically and horizontally long), and the threshold value for the average luminance value are predetermined by learning. This learning is performed using a large number of image samples. More specifically, a large number of image samples having the same shape and size as the processing window 70 are collected and stored in the face image sample storage unit 61 (see FIG. 17) in the storage area of the hard disk drive 14 (see FIG. 1). Stored. The image samples in this case include a face image sample including a face image (true sample) and a non-face image sample not including a face image (fake sample).

  The face image feature calculation unit 63, which is a function processing unit in the computer 3, performs learning based on the image samples stored in the face image sample storage unit 61 and can best separate the true / false samples. Find the detection filter. That is, various combinations of the positions of the rectangular areas 71 and 72 in the processing window 70 and the vertical and horizontal directions are set, and the combination that can best separate the true / false samples is selected. In this case, “being able to best isolate a true / false sample” means that misidentification with a true sample as a false sample and misidentification with a false sample as a true sample are minimized. For the combination of the rectangular regions 71 and 72 selected in this way, a face image detection filter is obtained by setting a threshold value that minimizes the misidentification rate. Information regarding the face image detection filter is stored in the face image feature dictionary 36 in advance.

  On the other hand, the peripheral area feature dictionary 33 holds information for identifying each of the three face images in the left and right columns in FIG. 18 as the face image peripheral area in addition to the face image in the central line in FIG. Will be. In the three images in the left column of FIG. 18, the face image extraction frame is shifted by a predetermined number of pixels (one pixel in the example of FIG. 18; -1 pixel) to the left from the positions of the three images in the central column of FIG. It is an image obtained by making it. In the three images in the right column of FIG. 18, the face image extraction frame is set to a predetermined number of pixels in the right direction from the position of the three images in the center column of FIG. It is an image obtained by shifting.

  The peripheral area feature dictionary 33 holds information related to the peripheral image detection filter similar to the face image detection filter. This peripheral image detection filter is determined by learning similar to the case of the face image detection filter. However, in this learning, in addition to the face image as shown in the center row of FIG. 18, face images as shown in the left and right rows are used as the face image peripheral region sample (true sample). A large number of such true samples and false samples that do not include a face image are stored in advance in the peripheral area image sample storage unit 62 that is a storage area in the hard disk drive 14.

A peripheral region detection filter capable of best separating the genuine samples stored in the peripheral region image sample storage unit 62 is detected by the peripheral region feature calculation unit 64 which is a function processing unit in the computer 3. It is obtained by learning similar to the case of the filter. Information about the surrounding area detection filter is stored in advance in the surrounding area feature dictionary 33.
The face image sample storage unit 61, the peripheral region image sample storage unit 62, the face image feature calculation unit 63, and the peripheral region feature calculation unit 64 are configured for prior learning. That is, these function processing units together with the peripheral area feature dictionary 33 and the face image feature dictionary 36 form an offline execution system 51.

On the other hand, the process of extracting the face image from the captured image is executed by the rough detection window setting unit 31, the peripheral region identification unit 32, the fine detection window setting unit 34, the face image identification unit 35, and the face image output processing unit 37. That is, these function processing units together with the peripheral area feature dictionary 33 and the face image feature dictionary 36 form an online execution system 52.
The coarse detection window setting unit 31 sequentially sets the coarse detection windows in the captured image so as to scan the entire area of the captured image with various sizes of coarse detection windows similar in shape to the processing window 70. More specifically, as shown in FIG. 20, the coarse detection window setting unit 31 starts from a start position 81 at the lower left corner in the captured image 80 and reaches a stop position 82 at the upper right corner. The coarse detection window 77 is set so as to scan the entire range. That is, the rough detection window setting unit 31 sets the first rough detection window 77 at the start position 81, and thereafter sets the rough detection window 77 in order while moving the predetermined plurality of pixels (for example, three pixels) to the right. I will do it. When the coarse detection window 77 reaches the right end of the captured image 80, the coarse detection window setting unit 31 returns the set position of the coarse detection window to the left end, moves upward by one pixel, and opens a new coarse detection window 77. Set. Thereafter, the same operation is repeated until the rough detection window 77 reaches the stop position 82.

  The coarse detection window setting unit 31 performs the above-described operation using the window having the minimum size that can be processed by the peripheral region detection filter as the initial coarse detection window 77, and thereafter sequentially sets the size of the coarse detection window 77 to a predetermined size. The same operation is repeated while increasing the size step by step (for example, by one pixel). Finally, a coarse detection window 77 having the same size as the captured image 80 is set.

  With respect to the partial images in the coarse detection window 77 sequentially set in this way, the peripheral area identifying unit 32 performs processing by the above-described peripheral area detection filter, and whether each coarse detection window 77 corresponds to the peripheral area of the face image. Determine whether or not. When the surrounding area detection filter is applied to the image in the rough detection window 77, the image is enlarged or reduced so as to match the window size of the surrounding area detection filter. With respect to the determined coarse detection window 77 that is the peripheral area of the face image, position information and size information are generated and stored in the RAM 12 (see FIG. 1).

  As described above, the peripheral region detection filter is obtained by learning using not only the sample in which the face image is in the center of the window but also the face image sample shifted by one pixel to the left and right. For this reason, if the rough detection window 77 is shifted and sequentially set in the captured image 80 with a shift width of 3 pixels or less, the region including the face image can be accurately extracted. Of course, if the coarse detection window 77 is sequentially set by shifting by 3 pixels, which is the maximum shift width, the determination of the peripheral area of the face image can be completed in the shortest time.

In general, in the case where the peripheral area detection filter is obtained by learning using the face image in the center row of FIG. 18 and the face image shifted by u pixels (where u is a natural number) to the left and right, 2u + 1 The coarse detection window 77 can be shifted by a shift width equal to or smaller than the pixels.
The precision detection window setting unit 34 sets a precision detection window in the vicinity of the area determined as the face image peripheral area by the peripheral area identification unit 32. For example, the fine detection window setting unit 34 sets a fine detection window having the same size as the rough detection window 77 within a face image peripheral region and a range shifted by one pixel to the left and right. A process by the face image identification unit 35, that is, a process by a face detection filter is executed on the set precision detection window. When a face detection filter is applied to an image in the precision detection window, the image is enlarged or reduced to match the window size of the face image detection filter.

  For the precision detection window determined to be a face image area by the face identification processing unit 35, information representing the position and size is output by the face image identification unit 35. In response to this, the face image output processing unit normalizes the size and inclination of the image in the precision detection window, and generates the face image data to be verified. In this case, “inclination normalization” refers to, for example, rotating an image having an inclination as shown in the upper or lower part of FIG. 18 to an image having substantially no inclination as shown in the middle part of FIG.

  As mentioned above, although one Embodiment of this invention was described, this invention can also be implemented with another form. For example, in the above-described embodiment, the example in which the collation data of a plurality of face images is registered in the collation data storage unit 25 has been described. However, the collation data of only one face image is registered in the collation data storage unit 25. Also good. In this case, for example, the similarity between the image data to be processed and the representative collation data is compared with a predetermined threshold value, and if the similarity does not exceed the threshold value, collation with the individual collation data is performed. What is necessary is just to complete | finish recognition processing, without performing. As a result, when the face image to be collated is clearly different from the face image of the registered person, it is possible to omit the collation processing for a plurality of individual collation data, so that the person of the face image to be collated is an unregistered person. You can make a quick decision that there is.

  In the above-described embodiment, the example in which the present invention for restricting entrance to the restricted access area is applied has been described. For example, the present invention is applied to a robot capable of dealing with people such as a home robot and a customer service robot. It is also possible to incorporate and use the image recognition apparatus. As a result, the robot can accurately recognize the face of the registered person in a short time, so that it is possible to deal with the person quickly and accurately. For example, such a robot can execute an appropriate response according to the recognized person with excellent responsiveness.

The recognition of a face image can also be used for a process of extracting a face image of a specific person from a huge amount of image data (still image or moving image). Specifically, face image recognition processing can be applied to processing of extracting a suspect's face image from video captured by a video camera in a criminal investigation.
Furthermore, although the case where the recognition target object is a face image has been described in the above embodiment, as described above, the present invention can also be applied to recognition of a recognition target object other than a face image.

  In addition, various design changes can be made within the scope of the matters described in the claims.

1 is an illustrative block diagram showing a configuration of a personal authentication system according to an embodiment of the present invention. It is a figure which shows the example of a display of the display connected to the computer which comprises an image recognition apparatus. It is a block diagram for demonstrating the functional structure of the computer in the execution state of the said computer program. It is an illustration figure for demonstrating the registration state of the collation data in a collation data storage part. It is the figure which expressed the registration state of the collation data in a collation data storage part more intuitively. It is a flowchart for demonstrating an example of the process by a face recognition process part. It is a flowchart which shows a specific example of the processing content for extracting the individual collation data of collation object. It is a flowchart which shows the other specific example of the process for extracting the individual collation data of collation object. It is a flowchart for demonstrating the other example of the process by a face recognition process part. It is a figure for demonstrating the calculation which calculates | requires the similarity of a pair of face image. It is a figure for demonstrating the characteristic area used in order to obtain | require the similarity of a face image. It is a flowchart for demonstrating the process for defining a similarity calculating formula by learning. It is a figure which shows an example of the weighting histogram when a common feature area is set to a pair of face images, and the correlation value of a pair of face images in the feature area is obtained. It is a figure for demonstrating the content held by the dictionary storage part for collation. It is a block diagram for demonstrating the structural example of a similarity calculating part. It is a block diagram for demonstrating the other structural example of a similarity calculating part. It is a functional block diagram for demonstrating the functional structure of a face image detection part. It is a figure which shows the example of the face image extracted from a captured image, and the example of the image detected as a peripheral region of a face image. It is a figure which shows the example of the filter for a face image detection process. It is a figure for demonstrating the scanning of the captured image by a rough detection window.

Explanation of symbols

1 Camera 2 Cable 3 Computer 4 Display 5 Display 6 Input Operation Unit 7 Gate Switch 8 CD-ROM
10 CPU
11 ROM
12 RAM
12a Image memory 12b Representative similarity storage area 12c Individual similarity storage area 13 Bus 14 Hard disk drive 15 CD-ROM drive 16 Display controller 17 Input controller 18 Image interface unit 19 Display control interface unit 20 Gate open / close control interface unit 25 Reference data Storage unit 26 Collation dictionary storage unit 30 Face image detection unit 31 Coarse detection window setting unit 32 Peripheral region identification unit 33 Peripheral region feature dictionary 34 Precision detection window setting unit 35 Face image identification unit 36 Face image feature dictionary 37 Face image output processing Unit 40 face recognition processing unit 41 similarity calculation unit 42 collation data selection unit 43 determination unit 45 feature region setting unit 45A feature region setting unit 46 first feature amount calculation unit 47 second feature amount calculation unit 48 correlation value calculation unit 49 comparison Part DESCRIPTION OF SYMBOLS 50 Weighting addition part 51 Offline execution system 52 Online execution system 55 Horizontal parting line 56 Vertical parting line 61 Face image sample storage part 62 Peripheral area image sample storage part 63 Face image feature calculation part 64 Peripheral area feature calculation part 70 Processing window 71 Rectangular Area 72 rectangular area 77 coarse detection window 80 captured image A _j feature area a _k partial area x ¹ _k area value x ² _k area value C _j correlation value I ¹ face image I ² face image

Claims (22)

An image recognition apparatus for processing image data and recognizing a recognition object in an image corresponding to the image data,
Individual collation data storage means in which a plurality of individual collation data corresponding to a plurality of individual appearances that the recognition object can take are stored in advance;
Representative collation data generating means for generating representative collation data corresponding to a representative appearance representing the plurality of individual appearances;
Representative collation means for collating image data to be processed with representative collation data generated by the representative collation data generating means, and calculating a similarity between an image corresponding to the image data and a representative appearance corresponding to the representative collation data When,
The image data to be processed and the individual collation data stored in the individual collation data storage unit are collated, and the degree of similarity between the image corresponding to the image data and the individual appearance corresponding to the individual collation data is calculated. Individual verification means for
Based on the similarity calculated by the representative collating unit, a plurality of individual collation data stored in the individual collation data storage unit are to be collated by the individual collating unit or excluded from collation targets by the individual collating unit. And an individual verification control means for determining whether or not to perform the image recognition.   When the similarity calculated by the representative verification unit is greater than a predetermined threshold, the individual verification control unit is configured to store a plurality of individual images stored in the image data to be processed and the individual verification data storage unit. The collation data is collated by the individual collation means, and when the similarity calculated by the representative collation means is equal to or less than the threshold value, a plurality of individual collation data stored in the collation data storage means are 2. The image recognition apparatus according to claim 1, wherein the image recognition apparatus is excluded from a verification target by the individual verification means. An image recognition apparatus for processing image data and recognizing a recognition object in an image corresponding to the image data,
A plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition object, and individual collation data storage means that is classified and stored for each of a plurality of groups;
Representative collation data generating means for generating a plurality of representative collation data representing the individual collation data of a plurality of groups stored in the individual collation data storage means;
The image data to be processed is collated with a plurality of representative collation data generated by the representative collation data, and the similarity between the image corresponding to the image data and the plurality of representative appearances respectively corresponding to the plurality of representative collation data A representative verification means for calculating
Individual matching means for calculating the similarity between the image corresponding to the image data to be processed and the individual appearance corresponding to the individual matching data stored in the individual matching data storage means;
Based on the similarity calculated for each of the plurality of representative collation data by the representative collation means, the processing by the individual collation means from among a plurality of groups of individual collation data stored in the individual collation data storage means. And an individual collation control unit that selects a group of individual collation data to be collated with target image data and excludes individual collation data of the group that has not been selected from the collation target by the individual collation unit. Recognition device.   The individual collation control means selects a group corresponding to representative collation data whose similarity calculated by the representative collation means is larger than a predetermined threshold, and the individual collation data belonging to this group is selected as the individual collation means. 4. The image recognition apparatus according to claim 3, wherein the individual collation data of the remaining group is excluded from the collation targets by the individual collating means.   The individual collation control means selects a group corresponding to representative collation data in which the similarity calculated by the representative collation means takes a value within the upper predetermined rank or within the upper predetermined ratio, and the individual collation data belonging to this group 5. The image recognition apparatus according to claim 3, wherein the individual collation unit is a collation target and the individual collation data of the remaining group is excluded from the collation target by the individual collation unit. Means for determining whether or not a maximum similarity that is a maximum value of a plurality of similarities calculated by the individual verification means for a plurality of individual verification data is greater than a predetermined recognition threshold;
A recognition result output means for outputting a recognition result corresponding to the individual matching data of the highest similarity when the highest similarity is larger than the recognition threshold;
When the highest similarity is less than or equal to the recognition threshold, at least a part of the individual verification data excluded from the verification target by the individual verification control unit is added to the verification target by the individual verification unit, and this individual verification unit 6. The image recognition apparatus according to claim 3, further comprising a re-individual collation control means for performing an individual collation process according to. Further including representative appearance similarity storage means for storing a plurality of similarities obtained by the representative verification means;
The re-individual collation control means corresponds to representative collation data having the highest similarity stored in the representative appearance similarity storage means among the groups of individual collation data excluded from the collation target by the individual collation means. 7. The image recognition apparatus according to claim 6, wherein the individual collation data of the selected group is added to a collation target by the individual collation means. An image recognition method for processing image data and recognizing a recognition object in an image corresponding to the image data,
Preparing individual collation data storage means in which a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition object are stored in advance;
Generating representative collation data corresponding to a representative appearance representing the plurality of individual appearances;
A representative collation step of collating the image data to be processed and the generated representative collation data, and calculating the similarity between the image corresponding to the image data and the representative appearance corresponding to the representative collation data;
The image data to be processed and the individual collation data stored in the individual collation data storage unit are collated, and the degree of similarity between the image corresponding to the image data and the individual appearance corresponding to the individual collation data is calculated. An individual verification step for
Based on the similarity calculated by the representative collating unit, a plurality of individual collation data stored in the individual collation data storage unit are set as collation targets in the individual collation step or excluded from collation targets in the individual collation step. An image recognition method comprising: an individual verification target selection step for determining whether to perform the verification.   In the individual collation target selection step, when the similarity calculated in the representative collation step is larger than a predetermined threshold, a plurality of individual collation data stored in the individual collation data storage means are converted into the individual collation data. When the similarity calculated in the representative collation step is equal to or less than the threshold value, a plurality of individual collation data stored in the collation data storage means are excluded from the collation targets in the individual collation step. 9. The image recognition method according to claim 8, further comprising the step of: An image recognition method for processing image data and recognizing a recognition object in an image corresponding to the image data,
Preparing individual collation data storage means in which a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition object are classified and stored for each of a plurality of groups;
Generating a plurality of representative collation data representing the individual collation data of a plurality of groups stored in the individual collation data storage means;
The image data to be processed is collated with the generated plurality of representative collation data, and the similarity between the image corresponding to the image data and the plurality of representative appearances respectively corresponding to the plurality of representative collation data is calculated. A representative verification step;
An individual collation step for calculating the similarity between the image corresponding to the image data to be processed and the individual appearance corresponding to the individual collation data stored in the individual collation data storage means;
Based on the similarity calculated for each of the plurality of representative collation data in the representative collation step, from among a plurality of groups of individual collation data stored in the individual collation data storage means, the processing target in the individual collation step An individual collation target selecting step of selecting a group of individual collation data to be collated with the image data and excluding the individual collation data of the unselected group from the collation target in the individual collation step. Recognition method.   The individual collation target selection step selects a group corresponding to the representative collation data in which the similarity calculated in the representative collation step is larger than a predetermined threshold, and the individual collation data belonging to this group is selected as the individual collation. The image recognition method according to claim 10, further comprising a step of excluding the individual collation data of the remaining group from the collation target in the individual collation step as a collation target in the step.   The individual collation target selection step selects a group corresponding to the representative collation data in which the similarity calculated in the representative collation step takes a value within the upper predetermined rank or the upper predetermined ratio, and the individual collation belonging to this group 12. The image recognition method according to claim 10 or 11, further comprising a step of making the data a collation target in the individual collation step and excluding the individual collation data of the remaining group from the collation target in the individual collation step. Determining whether the highest similarity, which is the highest value of the plurality of similarities calculated in the individual matching step with respect to a plurality of individual matching data, is greater than a predetermined recognition threshold;
Outputting the recognition result corresponding to the individual matching data of the highest similarity when the highest similarity is larger than the recognition threshold;
Individual collation target for adding at least a part of the individual collation data excluded from the collation target by the individual collation target selection step to the collation target in the individual collation step when the highest similarity is not more than the recognition threshold The image recognition method according to claim 10, further comprising a reselection step.   The individual collation target reselection step is a group corresponding to representative collation data having the highest similarity calculated in the representative collation step among groups of individual collation data excluded from collation targets in the individual collation step. 14. The image recognition method according to claim 13, further comprising the step of adding the individual collation data to a collation target in the individual collation step. A computer program for operating a computer as an image recognition apparatus for processing image data and recognizing a recognition object in an image corresponding to the image data,
The computer includes individual collation data storage means in which a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition object are stored in advance.
The computer program stores the computer,
Representative collation data generating means for generating representative collation data corresponding to a representative appearance representing the plurality of individual appearances;
Representative collation means for collating image data to be processed with representative collation data generated by the representative collation data generating means, and calculating a similarity between an image corresponding to the image data and a representative appearance corresponding to the representative collation data ,
The image data to be processed and the individual collation data stored in the individual collation data storage unit are collated, and the degree of similarity between the image corresponding to the image data and the individual appearance corresponding to the individual collation data is calculated. And a plurality of individual collation data stored in the individual collation data storage unit based on the similarity calculated by the representative collation unit or the individual collation data by the individual collation unit. A computer program which functions as an individual collation control means for determining whether to exclude from a collation target by the collation means.   When the similarity calculated by the representative verification unit is greater than a predetermined threshold, the individual verification control unit is configured to store a plurality of individual images stored in the image data to be processed and the individual verification data storage unit. The collation data is collated by the individual collation means, and when the similarity calculated by the representative collation means is equal to or less than the threshold value, a plurality of individual collation data stored in the collation data storage means are 16. The computer program according to claim 15, wherein the computer program is excluded from objects to be verified by the individual verification means. A computer program for operating a computer as an image recognition apparatus for processing image data and recognizing a recognition object in an image corresponding to the image data,
The computer includes individual collation data storage means that classifies and stores a plurality of individual collation data corresponding to a plurality of individual appearances that can be taken by the recognition target, for each of a plurality of groups,
The computer program stores the computer,
Representative collation data generating means for generating a plurality of representative collation data representing the individual collation data of a plurality of groups stored in the individual collation data storage means,
The image data to be processed is collated with a plurality of representative collation data generated by the representative collation data, and the similarity between the image corresponding to the image data and the plurality of representative appearances respectively corresponding to the plurality of representative collation data Representative verification means for calculating
Individual collating means for calculating the similarity between the image corresponding to the image data to be processed and the individual appearance corresponding to the individual collating data stored in the individual collating data storage means;
Based on the similarity calculated for each of the plurality of representative collation data by the representative collation means, the processing by the individual collation means from among a plurality of groups of individual collation data stored in the individual collation data storage means. A group of individual collation data to be collated with target image data is selected, and the individual collation data of the unselected group is made to function as an individual collation control unit that excludes from the collation target by the individual collation unit. Computer program.   The individual collation control means selects a group corresponding to representative collation data whose similarity calculated by the representative collation means is larger than a predetermined threshold, and the individual collation data belonging to this group is selected as the individual collation means. 18. The computer program according to claim 17, wherein the computer is a collation target, and the individual collation data of the remaining group is excluded from the collation target by the individual collation means.   The individual collation control means selects a group corresponding to representative collation data in which the similarity calculated by the representative collation means takes a value within the upper predetermined rank or within the upper predetermined ratio, and the individual collation data belonging to this group 19. The computer program according to claim 17, wherein the computer is a collation target by the individual collation unit, and the individual collation data of the remaining group is excluded from the collation target by the individual collation unit. The computer program further includes the computer,
Means for determining whether or not a maximum similarity that is a maximum value of a plurality of similarities calculated by the individual verification means for a plurality of individual verification data is greater than a predetermined recognition threshold;
A recognition result output means for outputting a recognition result corresponding to the individual collation data of the highest similarity when the highest similarity is larger than the recognition threshold;
When the highest similarity is less than or equal to the recognition threshold value, at least a part of the individual verification data excluded from the verification target by the individual verification control unit is added to the verification target by the individual verification unit, and this individual verification unit The computer program according to any one of claims 17 to 19, wherein the computer program functions as a re-individual collation control means for performing the individual collation processing according to the above. The computer further includes representative appearance similarity storage means for storing a plurality of similarities obtained by the representative matching means,
The re-individual collation control means corresponds to representative collation data having the highest similarity stored in the representative appearance similarity storage means among the groups of individual collation data excluded from the collation target by the individual collation means. 21. The computer program according to claim 20, wherein the individual collation data of the selected group is added to a collation target by the individual collation means.   A computer-readable recording medium on which the computer program according to any one of claims 15 to 21 is recorded.