JP2008146136A - Image recognition device, image recognition system, image recognition method, and control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition device for recognizing an image at high speed and high accuracy by comparing a taken image with a prepared image, an image recognition system, an image recognition method, and a control program. <P>SOLUTION: The image recognition device 1, when an image taken by an imaging device 10 is input thereto, detects a face image from the image taken, and determines the order of collating a plurality of registrants based on information indicating the relationship between the probability with which each registrant will be imaged and imaging time, and on the imaging time of the image taken; in accordance with the order of collating, the device collates each registrant's image with the detected face image to specify the registrant who corresponds to the face image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to an image recognition device that recognizes an object in an image, an image recognition system including the image recognition device, an image recognition method, and a control program.

There is known an image recognition technique for recognizing a candidate in a photographed image by collating a photographed image with a large number of candidate images registered in advance and searching for a corresponding image from the candidate images. As one application example of such a technique, a security system that performs personal authentication and entrance / exit management using an image taken at a management point is known (see, for example, Patent Document 1).
JP 2004-127285 A

When image recognition is performed by comparing a captured image with a plurality of candidate images as described above, the processing speed decreases as the number of candidate images increases. In order to increase the speed of this process, a method for simplifying the algorithm while permitting a decrease in recognition accuracy has been proposed, but there is a concern that the practicality may be decreased due to a decrease in accuracy.
Therefore, an object of the present invention is to provide an image recognition apparatus, an image recognition system, an image recognition method, and a control program for recognizing images with high speed and high accuracy even when the number of candidate images increases.

In order to achieve the above object, according to the present invention, a detection unit that detects an object image from a captured image, and the object image detected by the detection unit are collated with images of a plurality of candidate objects prepared in advance. By this means, collation means for specifying which candidate object the object image is, and control means for controlling a collation order when collating the images of the plurality of candidate objects by the collation means. The control means determines the collation order of the plurality of candidate objects based on information indicating the relationship between the probability that each candidate object is shot and the shooting time and the shooting time information of the shot image. An image recognition apparatus characterized by the above is provided.
According to this configuration, the collation order when collating the object image in the photographed image with the candidate object image, the information indicating the relationship between the probability that each candidate object is photographed and the photographing time, and the photographing of the photographed image Because it is determined based on the time information, it is possible to preferentially collate candidate objects that are highly likely to appear in the captured image, and quickly identify object images even if there are many candidate object images. This makes it possible to increase the efficiency and speed of image recognition.

In the above-described configuration, there is provided history information storage means for storing time information when each of the candidate objects was photographed in the past, and the control means is configured based on the information stored in the history information storage means. Information indicating the relationship between the probability that the candidate object is shot and the shooting time may be generated, and the collation order of the candidate objects may be determined based on this information.
In this case, information indicating the time when the candidate object was photographed in the past is stored, and information indicating the relationship between the probability that the candidate object is photographed and the photographing time is generated based on this information. Can be determined. In addition, each time an object image in a captured image is identified as a candidate object image, the shooting time information of the newly corresponding candidate object is updated, so the relationship between the probability that each candidate object is shot and the shooting time Even when the change occurs, the collation order can be determined according to the change.

Further, in the above configuration, the collating means can execute a plurality of types of collating processes, information indicating a relationship between the probability that each of the candidate objects is photographed and a photographing time, and photographing time information of the photographed image Based on the above, the type of the collation process may be changed.
In this case, when the type of collation processing is changed to high-precision processing, high-precision collation is applied to candidate objects with a low probability of being photographed, so that more reliable image recognition can be performed. In addition, if the type of collation processing is changed to processing with high processing load and high accuracy, collation may take time, but the above changes can be made only in limited cases, so It is possible to minimize the influence on the time required.

In the above configuration, the plurality of candidate objects are classified into a plurality of groups,
The collating unit selects one group from the plurality of groups, sequentially selects the candidate objects belonging to the selected group, performs collation, and the control unit performs each of the candidate objects belonging to each group. The order in which the group is selected by the collating unit may be determined based on time information when the image was captured in the past and shooting time information of the captured image.
In this case, since the collation order for each group is determined and processing is performed for each group according to the collation order, a plurality of candidate objects having a common or similar relationship between the photographing time and the probability of photographing are processed in groups. By doing so, processing load can be reduced, and efficiency and speed can be increased.

In the above-described configuration, for the specific group, the control unit includes an instruction unit that instructs that a factor that changes the association between the probability that the candidate object belonging to the group is shot and the shooting time has occurred. According to an instruction by the instruction unit, the order of selecting the group by the collating unit may be determined based on information indicating the relationship after the change.
In this case, when the relationship between the probability that the candidate object is shot and the shooting time changes due to an external factor, the relationship is changed in units of groups corresponding to this change. An appropriate collating sequence can be determined. In addition, since a plurality of candidate objects having the same or similar relationship between the shooting time and the shooting probability are processed together in units of groups, the work load is lighter than when instructions are given for each candidate object. When there are a large number of candidate objects, significant labor saving can be achieved.

Further, in the above configuration, for the specific group, the control unit includes an estimation unit that estimates that a factor that changes an association between a probability that the candidate object belonging to the group is shot and a shooting time has occurred. The order in which the group is selected by the collating unit may be determined based on the information indicating the relationship after the change based on the estimation result by the estimating unit.
In this case, it is estimated that the relationship between the probability that the candidate object is shot and the shooting time changes due to some factor, and the relationship is changed in units of groups corresponding to this change. As a result, further labor saving is achieved by omitting the input related to the occurrence of the factors,
Speeding up can be achieved by determining the ranking of candidate objects in units of groups.

In the above configuration, the detection means detects a human face image as the object image,
The collation means collates the face image detected by the detection means with a face image of a person registered in advance, so that the face image is one of the registered face images. It is good also as what specifies whether there exists.
In this case, the process of specifying a person's face image in a captured image as one of pre-registered persons can be performed at high speed even if there are many registered persons.

The present invention includes a photographing device that outputs a photographed image and an image recognition device, and the image recognition device detects the object image from the photographed image output by the photographing device, and the detection means detects the object image. The object image is collated with images of a plurality of candidate objects prepared in advance, thereby identifying which candidate object image the object image is, and a plurality of collation means by the collation means. Control means for controlling a collation order when collating images of the candidate objects, and the control means includes information indicating the relationship between the probability that each candidate object is photographed and the photographing time, Determining a collation order of the plurality of candidate objects based on shooting time information of a shot image; To provide an image recognition system.
According to this configuration, the collation order when collating the object image in the image photographed by the photographing device with the candidate object image, information indicating the relationship between the probability that each candidate object is photographed and the photographing time, Because it is determined based on the shooting time information of the shot image, candidate objects that have a high possibility of appearing in the shot image are preferentially checked, so even if there are many candidate object images, the object image can be quickly identified. Thus, it is possible to improve the efficiency and speed of a series of processes from photographing to image recognition.

The present invention detects an object image from a photographed image, and for a plurality of candidate objects prepared in advance, information indicating the relationship between the probability that each candidate object is photographed and the photographing time and photographing of the photographed image The collation order is determined based on the time information, and according to the collation order, the candidate object image and the detected object image are collated to identify which candidate object image the object image is. An image recognition method is provided.
According to this method, the collation order when collating the object image in the photographed image with the candidate object image, the information indicating the relationship between the probability that each candidate object is photographed and the photographing time, and the photographing of the photographed image Since it is determined based on the time information, candidate objects that are highly likely to appear in the captured image are preferentially collated, so even if there are many candidate object images, the object images can be quickly identified, Image recognition can be performed efficiently.

According to the present invention, the computer detects the object image from the captured image, and compares the object image detected by the detection unit with images of a plurality of candidate objects prepared in advance. Based on collating means for identifying which candidate object the object image is, information indicating the relationship between the probability that each candidate object is shot and the shooting time, and the shooting time information of the shot image And
Provided is a control program for functioning as a control unit that controls a collation order when collating images of a plurality of candidate objects by the collation unit.
If this control program is executed by a computer, the collation order when collating the object image in the photographed image with the candidate object image, information indicating the relationship between the probability that each candidate object is photographed and the photographing time, It is possible to realize speeding up and efficiency of image recognition by determining based on the shooting time information of the shot image and preferentially collating candidate objects that are highly likely to appear in the shot image.

Embodiments to which the present invention is applied will be described below with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram showing a configuration of an image recognition system 100 according to the first embodiment.
The image recognition system 100 is configured by connecting a photographing apparatus 10 including a plurality of cameras 12 to the image recognition apparatus 1.
The imaging device 10 captures images at predetermined time intervals with the camera 12 and outputs captured image data to the image recognition device 1.
The image recognition apparatus 1 performs an image recognition process for specifying an object (a person and an object shown as a subject) in a captured image input from the imaging apparatus 10. The image recognition apparatus 1 can execute image recognition processing for both moving images and still images, and can recognize both people and objects as objects. In the first embodiment and other embodiments described later,
As an example, a case will be described in which image recognition processing is performed on a still image and a human face is recognized as an object.

First, the configuration of the image recognition device 1 will be described.
As shown in FIG. 1, an image recognition device 1 includes a control unit 2 that executes control of each unit of the image recognition device 1 and an image recognition process described later, and a ROM 3 that stores various programs executed by the control unit 2. And a RAM 4 for temporarily storing programs and data processed by the control unit 2. The ROM 3 is a recording medium using a semiconductor storage device or the like, and stores a control program 31 executed by the control unit 2 and data related to the control program 31 in a nonvolatile manner.
In addition, the image recognition device 1 includes a storage unit 5 that stores an image processed by the control unit 2, and the like.
An input unit 6 for inputting an instruction to the image recognition apparatus 1, an output unit 7 for outputting a result of processing executed by the control unit 2, and an external interface (I not shown) connected to an external device (not shown). / F) 8. These units are connected to each other by a bus 9.

The storage unit 5 stores various information in a nonvolatile manner using a magnetic recording medium, an optical recording medium, or a semiconductor storage device. The storage unit 5 has a plurality of storage areas, and in these storage areas,
Various data referred to and processed in the image recognition process are stored.
FIG. 2 is a diagram schematically illustrating the configuration of the storage unit 5. As shown in FIG. 2, the storage area of the storage unit 5 includes a recognition DB (database) 51 that stores information used for recognizing images, and a recognition history information storage unit 52 (history information storage means). ), And a time distribution information storage unit 53.
The recognition DB 51 is a database that stores information used for recognizing a face image in image recognition processing to be described later. Specifically, the recognition DB 51 stores information indicating the characteristics of an object image. In the first embodiment, since a human face is recognized as an object, the recognition D
In B51, information indicating the characteristics of the face image for a plurality of human faces is stored in association with information for identifying the faces (person names, etc.).
In the image recognition process, the control unit 2 detects a face image from the recognition target image, and collates the detected face image with information stored in the recognition DB 51, so that the detected face image is a person's face. Determine if it exists. Accordingly, a plurality of persons stored in the recognition DB 51 are treated as recognition candidates in the image recognition process. That is, since it can be regarded as registered in the image recognition apparatus 1 as a candidate for image recognition, in the following description, a plurality of persons stored in the recognition DB 51 are represented as “
Called "registrant".

The recognition history information storage unit 52 stores recognition history information 52 </ b> A indicating a history recognized in the past for each registrant whose information is stored in the recognition DB 51.
FIG. 3 is a diagram schematically illustrating a configuration example of the recognition history information 52A. The recognition history information 52A shown in FIG. 3 includes the registrants “A”, “B”, “C”, “D”,.
The recognition times for 5 days from November 1 to November 5 are stored.
Here, the recognition time is the photographing time of the photographed image photographed by the photographing device 10. For example, when the registrant “A” appears in the captured image taken at 8:27 am on November 1, the recognition time of the registrant “A” on November 1 is “8:27”. Become.
Each time the image recognition apparatus 1 performs image recognition processing and identifies a person (registrant) shown in the captured image, the history of the identified registrant is added to the recognition history information 52A.
The configuration shown in FIG. 3 is merely an example, and the number of registrants whose information is stored in the recognition history information 52A and the number of days in which the recognition time is stored are arbitrary.

The time distribution information storage unit 53 illustrated in FIG. 2 stores time distribution information indicating the distribution of recognition times of each registrant based on the recognition history information 52A stored in the recognition history information storage unit 52.
4A and 4B are diagrams for explaining the outline of the time distribution information. FIG. 4A shows the distribution of photographing times of each registrant, and FIG. 4B shows the probability that each photographer is photographed and the photographing time. The relationship with is shown.
According to the recognition history information 52A illustrated in FIG. 3, the recognition times of the registrants “A” to “D” for five days are distributed as shown in FIG. For example, the recognition time of the registrant “A” is 8:03 am
It is concentrated around 0 minutes, and the recognition time of the registrant “B” is divided into around 8:30 am and 9 am.

Based on the information of the recognition history information 52A, a distribution function having the probability that each registrant is photographed as a variable is obtained. When this distribution function is graphed, a relationship as shown in FIG. 4B appears. For example, for registrant “A”, a relationship that peaks at around 8:30 am appears, and for registrant “B”, a wide range of relationships spanning from about 7:50 am to about 9:30 am appears. . The time distribution information storage unit 53 stores time distribution information indicating the relationship between the probability that each registrant is photographed and the photographing time, as shown in FIG.
By using this time distribution information, it is possible to rank each registrant in descending order of the possibility of being photographed at a certain time. For example, referring to FIG. 4B, the probability of shooting at 8:30 am is the highest for the registrant “A”, and in the order of the registrants “C”, “B”, and “D”. Become. Also, if the registrants are ranked in descending order of the probability of shooting at 9 am, the registrants “D”, “B
”,“ A ”, and“ C ”.
In addition, the specific aspect of time distribution information is not specifically limited, Histogram distribution,
Any probability distribution such as a gamma distribution, a normal distribution, or a mixed distribution can be used.

The input unit 6 shown in FIG. 1 includes various key switches such as keys for inputting numbers, letters, symbols, etc., function keys for instructing specific operations, a rotary switch, a power ON / OFF button, and the like.
An operation signal corresponding to the operation of these switches or buttons is generated and output to the control unit 2.
Further, the output unit 7 displays a photographed image taken by the photographing device 10, a processing result by the control unit 2, and the like as an image on a display screen 71 provided with an LCD (liquid crystal display) or the like, or a printing device (not shown). To print.

The external interface 8 is an interface connected to a device external to the image recognition device 1. Here, examples of the external device include a storage device incorporating a hard disk drive and a semiconductor memory, a personal computer, a digital still camera, a digital video camera, and a mobile phone. The specific configuration of the external interface 8 is US
B (Universal Serial Bus), IEEE1394, Ethernet (registered trademark) and other wired connection interfaces that support communication systems, IEEE802.11, Bluetooth (registered trademark), UWB, ZigBee
And a wireless communication interface corresponding to a communication method such as (registered trademark).
The external interface 8 may include an interface circuit capable of executing protocols defined by the various standards, in addition to the connectors conforming to the various standards.

A photographing device 10 is connected to the image recognition device 1 via an external interface 8.
The photographing apparatus 10 includes, for example, a plurality of cameras 12 installed near the entrance of a building, and a photographing control unit 11 that performs photographing with the cameras 12 and acquires photographed image data.
The imaging control unit 11 causes the camera 12 to perform imaging every preset time, and outputs the acquired captured image data to the image recognition device 1. Here, the photographing control unit 11 outputs information indicating the photographing time to the image recognition apparatus 1 together with the photographed image data.
The camera 12 is a digital camera, performs imaging using visible light or infrared light according to the control of the imaging control unit 11, and outputs captured image data to the imaging control unit 11.

FIG. 5 is a flowchart showing image recognition processing executed by the image recognition apparatus 1.
In the image recognition process shown in FIG. 5, the control unit 2 functions as a detection unit, a collation unit, and a control unit.
This image recognition process is executed each time a photographed image is input from the photographing apparatus 10 to the image recognition apparatus 1.
The control unit 2 acquires the captured image input from the imaging device 10 and the capturing time of the captured image (step S11), and detects a face image from the acquired captured image (step S12). In step S12, the control unit 2 detects a region that appears to be an image of a human face in the acquired captured image, and performs a process of cutting out the detected region. The region cut out in this process is processed as a face image by the following process. Here, for the process of detecting the face image, for example, a face detection technique using an SVM (support vector machine) based on a luminance pattern can be used.

Further, the control unit 2 refers to the recognition history information 52A stored in the recognition history information storage unit 52 and calculates information included in the recognition history information 52A, thereby generating time distribution information of each registrant. Is stored in the time distribution information storage unit 53 (step S13). Based on the time distribution information generated here, the control unit 2 determines the collation order of each registrant in descending order of the probability of being photographed at the photographing time acquired in step S11 (step S14).

Thereafter, the control unit 2 selects one registrant according to the determined collation order (step S
15). In this selection, registrants are selected in order from the top of the collation order.
When performing the process 5, the highest registrant is selected.
The control unit 2 reads information on the feature of the selected registrant's face image from the recognition DB 51, and collates the read information with the face image detected in step S12 (step S16).
Then, it is determined whether or not the features of the face image match (step S17).
Here, when the features of the face image do not match (step S17; No), the control unit 2
Returning to step S15, the next registrant is selected according to the collation order, and collation is performed again.

On the other hand, when the features of the face image match (step S17; Yes), the control unit 2 specifies that the person shown in the captured image is the selected registrant, and specifies the name of the specified registrant. Is output by the output unit 7 (step S18), and the recognition history information 52A corresponding to this registrant.
Is updated (step S19), and the image recognition process is terminated.

As described above, in the image recognition system 100 according to the first embodiment, the imaging device 1
The image recognition device 1 executes an image recognition process on the photographed image photographed by 0. In this image recognition process, the image recognition apparatus 1 specifies who the face shown in the photographed image is. In this image recognition processing, the image recognition device 1 uses the time distribution information stored in the time distribution information storage unit 53 in order of increasing probability that each registrant is photographed at the same time as the photographing time. The collation order is determined, and the face image detected in the captured image is collated with the face images of a plurality of registrants while selecting the registrant according to the collation order. For this reason, since a registrant who has a high probability of being photographed is collated first, there is a high possibility that the face of the registrant to be collated first matches the face image in the photographed image. As a result, the time from the start of the image recognition process to the matching of the face images can be significantly shortened, and the processing speed can be increased.
In particular, when there are many registrants, if collation is performed without considering the possibility of matching with the face image, it is necessary to perform enormous processing until the face images match, which is necessary for the processing. It takes a long time and occupies a huge amount of hardware resources. In the image recognition system 100 according to the first embodiment, face images can be matched in a short time even if the number of registrants is large by preferentially collating registrants who are highly likely to match face images. Who can be identified. Therefore, the image recognition process can be executed very efficiently and at high speed.

In the image recognition process, when the collation order is determined in step S14, the control unit 2
With reference to the recognition history information 52A, a registrant who has already been identified at a time before the same day may be placed in a lower order in the collation order regardless of the time distribution information. For example, when the image recognition system 100 is applied to an entrance / exit management system, most of the registrants are photographed once by the camera 12 installed near the entrance during the entrance time zone. Since it is very unlikely that a registrant who has already been identified in the entrance time zone will be photographed by the camera 12 again, the collation order will be lowered regardless of the time distribution information of the registrant, thereby further improving efficiency. The speed can be increased.
Further, when starting the image recognition processing, the image recognition apparatus 1 transmits control information for instructing the photographing apparatus 10 to output a photographed image, and the photographing control unit 11 captures the photographed image according to the control information. You may output to the recognition apparatus 1.
Moreover, the control part 2 produces | generates time distribution information, whenever it performs an image recognition process (step S13).
The time distribution information may be generated and stored in the time distribution information storage unit 53 before performing the image recognition process. In this case, it is possible to further speed up the image recognition process.

Note that the image recognition apparatus 1 may add the name of the registrant specified in the image recognition process as metadata to the captured image. Since the metadata generated here is information expressed by characters (so-called text data), it can be searched at a higher speed than still image and moving image data. For this reason, when metadata is added to an image, there is an advantage that the search and extraction of the image becomes much easier and the burden of image management is reduced.

[Second Embodiment]
Hereinafter, the image recognition apparatus 1 according to the second embodiment will be described. Since the configuration of the image recognition apparatus 1 according to the second embodiment is the same as that described in the first embodiment, the description is omitted by assigning the same reference numerals to the common components.
In the image recognition processing, the image recognition apparatus 1 according to the second embodiment changes the recognition method to a more accurate and reliable method when a lower-order registrant is selected in the image recognition process.
The operation for changing the recognition method includes, for example, an operation for changing an algorithm for comparing features of a face image detected from a photographed image and a registrant's face image, and an operation for changing a parameter used for the comparison. . In addition, image recognition processing is usually performed on a single photographed image taken by one camera 12, whereas face images are detected from a plurality of images taken by a plurality of cameras 12, respectively. The face image may be changed so as to collate with the selected registrant.

When the image recognition system 100 is used for security-related applications, the higher the recognition accuracy, the better. However, a method with high recognition accuracy takes a long time because the processing load is high. Therefore, the image recognition system 100 according to the second embodiment changes the recognition method only when a registrant with a lower collation order is selected. It can be said that a registrant whose collation order is low has a low probability of being photographed at the photographing time. In other words, it can be said that a registrant with a high collation order is photographed at the photographing time, and a registrant with a lower collation order is unnatural at the photographing time. . Therefore, if the recognition method is changed to a high-accuracy method only when a registrant having a lower collation order is selected, reliability can be improved while avoiding a long image recognition process.

FIG. 6 is a flowchart showing an operation of the image recognition system 100 according to the second embodiment, and specifically shows an image recognition process executed by the image recognition apparatus 1.
In the image recognition process shown in FIG. 6, the control unit 2 acquires a captured image input from the imaging device 10 and a capturing time of the captured image (step S <b> 21), and detects a face image from the acquired captured image. (Step S22). The process of step S22 is the same as step S22 described above.
12 (FIG. 5).
Subsequently, the control unit 2 calculates the information included in the recognition history information 52A, thereby generating time distribution information for each registrant and storing it in the time distribution information storage unit 53 (step S23).
. Based on the time distribution information generated here, the control unit 2 determines the collation order of each registrant in descending order of the probability of being photographed at the photographing time acquired in step S21 (step S24).

Thereafter, the control unit 2 selects one registrant according to the determined collation order (step S2
5) Information related to the feature of the face image of the selected registrant is read from the recognition DB 51, and the read information and the face image detected in step S22 are collated (step S26). It is determined whether or not (step S27).
Here, when the features of the face image do not match (step S27; No), the control unit 2
It is determined whether or not collation has been completed up to the nth registered person in the collation order (n is a preset natural number) (step S28). If the collation up to the nth registrant has not been completed, the control unit 2 returns to step S25, selects the next registrant in accordance with the collation order, and collates again.
On the other hand, when the collation of the top n persons in the collation order has been completed (step S28; No), the control unit 2 changes the recognition method (step S29). This change in the recognition method is a change to a highly accurate recognition method as described above. Then, the control part 2 returns to step SS25, selects the next registrant according to collation order, and performs collation.

If the features of the face image match (step S27; Yes), the control unit 2 specifies that the person shown in the captured image is the selected registrant, and specifies the name of the specified registrant. Is output by the output unit 7 (step S30), and the recognition history information 52A corresponding to this registrant.
Is updated (step S31), and the image recognition process is terminated.

As described above, in the image recognition system 100 according to the second embodiment, the imaging device 1
The image recognition apparatus 1 executes an image recognition process on the photographed image photographed by 0, and specifies a person shown in the photographed image. In this image recognition process, the image recognition device 1 uses the time distribution information stored in the time distribution information storage unit 53 to determine the registrants in descending order of probability that each registrant will be photographed at the same time as the photographing time. A collation order is determined, and a registrant is selected according to the collation order, and the face image detected in the captured image is collated with the face images of a plurality of registrants.
Furthermore, the image recognition apparatus 1 changes the recognition method and executes highly accurate recognition when the person of the face image is not specified even after the collation of the top n persons in the collation order is completed.
Thereby, by increasing the recognition accuracy in the case of collating registrants with a low probability of being photographed at the photographing time, more reliable image recognition can be performed and high reliability can be ensured. Further, by limiting the registrants who apply the recognition method with high recognition accuracy to the lower rank of the collation order, it is possible to improve the reliability while avoiding a long image recognition process.
It should be noted that the same value is not always used for n, but may be changed according to the situation. For a candidate whose shooting probability at that time is less than a fixed value without specifying n,
The verification method may be changed. For example, the collation order at a certain time is registrant A, B,
In the order of C and D, the probability calculated from the distribution of the past shooting times is 35 for the registrant A.
%, The registrant B is 30%, the registrant C is 25%, and the registrant D is 10%. .
On the other hand, the collation order at a certain time is the order of registrants A, B, C, and D, and the probability calculated from the distribution of the past shooting times is 90% for registrant A and 7 for registrant B. %, Registrant C is 2
% And the registrant D is 1%, the collation ranking second to fourth is regarded as almost the same probability, and the collation method is changed from the registrant B second in the collation ranking.

[Third Embodiment]
Next, an image recognition apparatus 1 according to the third embodiment will be described. Since the configuration of the image recognition system 100 according to the third embodiment is substantially the same as that described in the first embodiment, description thereof will be omitted by attaching the same reference numerals to the common components.
In the image recognition processing, the image recognition device 1 according to the third embodiment obtains time distribution information for each group indicating the relationship between the shooting time and the probability of shooting in units of groups to which a plurality of registrants belong. The collation order for each group is determined based on the time distribution information for each group. When collating, a group is selected according to the collation order for each group, and registrants belonging to the group are sequentially selected and collated.

FIG. 7 is a diagram illustrating a configuration example of the storage unit 5 according to the third embodiment.
As shown in FIG. 7, the storage unit 5 in the third embodiment includes a registrant group information storage unit 54, a group for each group, in addition to the recognition DB 51, the recognition history information storage unit 52, and the time distribution information storage unit 53. A time distribution information storage unit 55 is included.
The registrant group information storage unit 54 stores information for classifying registrants whose information is stored in the recognition DB 51 into groups. The group to which each registrant belongs is determined by the information stored in the registrant group information storage unit 54. In the third embodiment, a case where all registrants belong to one of the groups will be described as an example. However, for example, some registrants may not belong to the group. In this case, a registrant who does not belong to the group may belong to one group “not affiliated” for convenience, or may be processed separately from the processing for each group.

FIG. 8 is a diagram illustrating an overview of the group time distribution information stored in the group time distribution information storage unit 55.
The image recognition apparatus 1 detects all registrants belonging to a certain group based on information stored in the registrant group information storage unit 54, and recognizes recognition history information corresponding to all detected registrants as recognition history information. Read from the storage unit 52 to generate time distribution information for each group. The generated time distribution information for each group is represented, for example, as a probability distribution shown in FIG.
FIG. 8A shows group time distribution information for two groups, group P and group Q. The probability that a registrant belonging to the group P is photographed is distributed from around 7:30 am to past 8:30 am. Further, the probability that a registrant belonging to the group Q is photographed is distributed from about 8 am to after 9 am. For this reason, for example, when the shooting time is 8:00 am, the group P is ranked higher in the collation order, and when the shooting time is 9:00 am, the group Q is ranked higher.

By the way, in the image recognition system 100, some registrants may be photographed at a time significantly deviated from the past recognition time due to an external factor.
For example, when the image recognition system 100 is used as an entrance / exit management system and a photographed image taken by a camera 12 provided at the entrance of a building or room is recognized to manage entry / exit of registrants, many registrants are registered. If the transportation system to be used is greatly delayed due to an accident or the like, the registrants who use this transportation system will enter the time later than usual. Therefore, if registrants who pass through the same transportation system are classified into the same group, the admission times of all registrants belonging to one group will be delayed. In addition, it is conceivable that the entry times of registrants belonging to other groups are not affected and become the same time as usual.

Therefore, the image recognition apparatus 1 instructs the input unit 6 when the time when a registrant belonging to a specific group is photographed is significantly different from the past recognition time due to an external factor. In response to the input, the collation order of the registrants belonging to the corresponding group is changed.
For example, if it is expected that the photographing time of the registrant belonging to the group P out of the groups P and Q shown in FIG.
The fact that the time is delayed is input. In response to this instruction input, the image recognition apparatus 1 receives the group P.
The time distribution information is shifted to a later time as shown in FIG.
In the state shown in FIG. 8B, for example, the collation order when the shooting time is 8:00 am is higher in the group Q, and the collation order when the shooting time is 9:00 am is also higher in the group Q.

Furthermore, when the delay of the time when the registrant belonging to the group P is photographed is smaller than the instruction first input by the input unit 6 due to a change in the external factor thereafter, the registrant belonging to the group P is photographed. The time is earlier than the state of FIG. In this case, if an instruction is input again by the input unit 6, the time distribution information of the group P changes as shown in FIG.
In the state shown in FIG. 8C, for example, in the collation order when the shooting time is 8:00 am, the group Q is higher, and in the collation order when the shooting time is 9:00 am, the group P is higher.
As described above, the image recognition apparatus 1 shifts the time distribution information for each group on the time axis in response to an instruction input from the input unit 6. As a specific process, when a group to which the time distribution information for each group is shifted and a time width to be shifted are instructed by the input unit 6,
The control unit 2 updates the group time distribution information stored in the group time distribution information storage unit 55 corresponding to the instructed group. When recognizing a photographed image input from the photographing apparatus 10 after this update is performed, the collation order is determined based on the updated time distribution information for each group.
Thereby, the image recognition apparatus 1 can determine an appropriate collation order when the relationship between the shooting time and the probability of shooting changes based on an external factor. Also, if the process of changing the collation order for each registrant is performed, the processing load may become excessive when the number of registrants is large, but the processing load is suppressed by performing the process for each group. As a result, significant efficiency can be achieved.
In the third embodiment, the collation order of the registrants belonging to the corresponding group is changed in response to an instruction input from the input unit 6, but the image recognition apparatus 1 automatically performs external factors. May be estimated, and the collation order may be shifted based on the estimation result. In this case, the control unit 2 of the image recognition apparatus 1 functions as an estimation unit. In this case, information regarding changes in external factors may be input to the image recognition apparatus 10 from an external device (not shown) via the external I / F 8. Specifically, a configuration in which information is periodically acquired from an external server device via a communication line connected to the external I / F 8, and changes in external factors are estimated based on the acquired information. do it.

FIG. 9 is a flowchart showing an operation of the image recognition system 100 according to the third embodiment, and specifically shows an image recognition process executed by the image recognition apparatus 1. When the image recognition process shown in FIG. 9 is executed, the input unit 6 functions as an instruction unit.
The control unit 2 acquires the captured image input from the imaging device 10 and the capturing time of the captured image (step S41), and detects a face image from the acquired captured image (step S42). The process in step S42 is the same as that in step S12 (FIG. 5) described above.
Subsequently, the control unit 2 refers to the registrant group information storage unit 54, acquires registrants belonging to each group, and calculates information included in the recognition history information 52A for the registrants belonging to each group, Time distribution information for each group is generated and stored in the time distribution information storage unit 55 for each group (step S43). Based on the time distribution information for each group generated here, the control unit 2 determines the collation order of the groups in descending order of the probability of being photographed at the photographing time acquired in step S41 (step S44).

Thereafter, the control unit 2 selects one group according to the determined collation order (step S
45) One registrant is selected from the registrants belonging to the selected group (step S46).
). The control unit 2 reads out information on the characteristics of the selected registrant's face image from the recognition DB 51, and collates the read information with the face image detected in step S42 (step S47).
), It is determined whether or not the features of the face image match (step S48).
Here, when the features of the face image do not match (step S48; No), the control unit 2
It is determined whether or not collation has been completed for all registrants belonging to the currently selected group (step S49). When there is a registrant who is not collated among the registrants belonging to the selected group (step S49; No), the control unit 2 returns to step S46 and selects another registrant belonging to the selected group. Select and continue verification. On the other hand, if all the registrants belonging to the currently selected group have been verified (step S49; Yes),
The control unit 2 returns to step S45 and selects the next group according to the collation order.

If the features of the face image match (step S48; Yes), the control unit 2 specifies that the person shown in the captured image is the selected registrant, and specifies the name of the specified registrant. Is output by the output unit 7 (step S50), and the recognition history information 52 corresponding to the registrant is output.
A is updated (step S51), and the image recognition process is terminated.

As described above, in the image recognition system 100 according to the third embodiment, the imaging device 1
The image recognition apparatus 1 executes an image recognition process on the photographed image photographed by 0, and specifies a person shown in the photographed image. In this image recognition processing, the image recognition apparatus 1 groups a plurality of registrants, generates time distribution information for each group based on time distribution information of registrants belonging to each group, and this time distribution for each group. The collation order for each group is determined according to the information. Then, a group is selected according to the collation order for each group, and the face image detected from the photographed image and the face images of a plurality of registrants are collated while sequentially selecting registrants belonging to the selected group. Thus, by processing together registrants whose relationship between the shooting time and the probability of shooting is common or approximate, the processing load can be reduced, and efficiency and speed can be increased.

Further, when it is assumed that the time when a registrant belonging to a specific group is photographed is significantly different from the past recognition time due to external factors, the image recognition device 1 inputs an instruction from the input unit 6. The collation order of registrants belonging to the corresponding group is changed accordingly. When the relationship between the shooting time and the probability of shooting changes based on external factors, an appropriate collation order can be determined. Thereby, the image recognition apparatus 1 can determine an appropriate collation order when the relationship between the shooting time and the probability of shooting changes based on an external factor. Also, if the process of changing the collation order for each registrant is performed, the processing load may become excessive, especially when the number of registrants is large, but significant labor savings can be achieved by performing the process for each group. Can be planned.

In the first to third embodiments, when there is no registrant that matches the face image detected from the photographed image, the display screen 71 or the like is used to notify that the registrant is not applicable. Also good. In the first to third embodiments, the photographing apparatus 10 includes two cameras 1.
However, the number of cameras 12 is arbitrary. These multiple cameras 1
The place where 2 is specifically installed is arbitrary, and may be installed so as to photograph one place at a different angle, or may be installed so as to photograph a different place. Furthermore, in this case, the shooting control unit 11 may sequentially output the shot images shot by the plurality of cameras 12 to the image recognition apparatus 1, or out of the shot images shot by the plurality of cameras 12. One captured image may be selected and output to the image recognition apparatus 1.

Furthermore, the image recognition system 100 according to the first to third embodiments can be applied to an electronic device connected to a digital camera or incorporating a digital camera. in this case,
By mounting the function of the image recognition apparatus 1 on the digital camera body, the image recognition system 100 can be configured with a single digital camera. It is also possible to connect an image input device other than a camera to the image recognition device 1, and in this case, input to the image recognition device 1 via the external interface 8 or the input unit 6. What is necessary is just to input the imaging time of the image to be performed.
In addition, the specific detailed configuration of each part constituting the image recognition apparatus 1 can be arbitrarily changed without departing from the gist of the present invention.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment. It is a figure which shows the structure of a memory | storage part typically. It is a figure which shows the structural example of recognition history information typically. It is a figure explaining the outline | summary of time distribution information. It is a flowchart which shows an image recognition process. It is a flowchart which shows the image recognition process in 2nd Embodiment. It is a figure which shows the structural example of the memory | storage part which concerns on 3rd Embodiment. It is a figure explaining the outline | summary of time distribution information for every group. It is a flowchart which shows the image recognition process in 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image recognition apparatus, 2 ... Control part (Detection means, collation means, control means, estimation means), 3 ... R
OM, 4 ... RAM, 5, 5A ... storage unit, 6 ... input unit (instruction means), 7 ... output unit, 8 ... external interface, 10 ... imaging device, 11 ... imaging control unit, 12 ... camera, 31 ... control Program, 51 ... DB for recognition, 52 ... Recognition history information storage unit (history information storage means), 52A ... Recognition history information, 53 ... Time distribution information storage unit, 54 ... Registrant group information storage unit, 55 ... Time per group Distribution information storage unit, 71 ... display screen, 100 ... image recognition system.

Claims (10)

Detection means for detecting an object image from the captured image;
Collating means for identifying which candidate object image the object image is by comparing the object image detected by the detecting means with images of a plurality of candidate objects prepared in advance;
Control means for controlling the collation order when collating the images of the plurality of candidate objects by the collating means,
The control means determines a collation order of the plurality of candidate objects based on information indicating a relationship between a probability that each candidate object is shot and a shooting time, and shooting time information of the shot image. ,
An image recognition apparatus. History information storage means for storing time information when each of the candidate objects was photographed in the past,
The control means generates information indicating the relationship between the probability that each of the candidate objects is shot and the shooting time based on the information stored in the history information storage means, and based on the information, Determining the collation order of candidate objects,
The image recognition apparatus according to claim 1. The collation means can execute a plurality of types of collation processing,
Information indicating the relationship between the probability that each of the candidate objects is shot and the shooting time;
Changing the type of the matching process based on the shooting time information of the shot image;
The image recognition apparatus according to claim 1 or 2. The plurality of candidate objects are classified into a plurality of groups,
The matching unit selects one group from the plurality of groups, sequentially selects the candidate objects belonging to the selected group, and performs matching.
The control means determines the order in which the group is selected by the collating means based on time information when each of the candidate objects belonging to each group was photographed in the past and photographing time information of the photographed image. ,
The image recognition apparatus according to claim 1, wherein: Instructing means for instructing that a factor that changes the relationship between the probability that the candidate object belonging to the group is photographed and the photographing time has occurred for the specific group,
The control means determines an order in which the group is selected by the collating means based on information indicating the relationship after the change in accordance with an instruction by the instruction means;
The image recognition apparatus according to claim 4. With respect to the specific group, comprising: an estimation means for estimating that a factor that changes the association between the probability that the candidate object belonging to the group is shot and the shooting time has occurred,
The control means determines the order in which the group is selected by the matching means based on the information indicating the relationship after the change based on the estimation result by the estimation means;
The image recognition apparatus according to claim 4. The detection means detects a human face image as the object image,
The collation means collates the face image detected by the detection means with a face image of a person registered in advance, so that the face image is one of the registered face images. Identifying whether there is,
The image recognition apparatus according to claim 1, wherein: A photographing device for outputting a photographed image and an image recognition device;
The image recognition device includes:
Detecting means for detecting an object image from a photographed image output by the photographing device;
Collating means for identifying which candidate object image the object image is by comparing the object image detected by the detecting means with images of a plurality of candidate objects prepared in advance;
Control means for controlling the collation order when collating the images of the plurality of candidate objects by the collating means,
The control means determines a collation order of the plurality of candidate objects based on information indicating a relationship between a probability that each candidate object is shot and a shooting time, and shooting time information of the shot image. ,
An image recognition system characterized by this. While detecting the object image from the shot image,
For a plurality of candidate objects prepared in advance, the collation order is determined based on information indicating the relationship between the probability that each candidate object is shot and the shooting time, and the shooting time information of the shot image,
According to this collation order, by collating the image of the candidate object and the detected object image, identifying which candidate object the object image is,
An image recognition method characterized by the above. Computer
Detection means for detecting an object image from the captured image;
Collating means for identifying which candidate object image the object image is by comparing the object image detected by the detecting means with images of a plurality of candidate objects prepared in advance;
Collation when collating images of a plurality of candidate objects by the collating unit based on information indicating the relationship between the probability that each candidate object is shot and the shooting time and shooting time information of the shot image Control means for controlling the order;
Control program to function.

Images