JP6855175B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The present invention relates to a technique for matching the colors of a plurality of images captured by a plurality of imaging devices.

Conventionally, there is known a system for searching an image feature amount of an object (for example, a person) reflected in an captured image acquired from a plurality of cameras (imaging devices) (for example, Patent Document 1). When searching for the image features of an object from the images captured by multiple imaging devices, the colors may not match even though they are the same object due to ambient light, lighting, etc. The search accuracy of the object was reduced.

In Patent Document 1, a corrected image with white balance correction is generated by using images of white road markings taken by a plurality of image pickup devices attached to an automobile. Alternatively, a corrected image matching the colors is generated using an image of a road element having a color component such as a curb laid on the road surface.

Japanese Unexamined Patent Publication No. 2012-105090 JP-A-2009-49759 Patent No. 4715527

In the method of Patent Document 1, images of white road markings, images of curbs, and the like are extracted from all the images captured by a plurality of imaging devices. However, it is difficult to correctly extract an image suitable for generating a color-matched corrected image from these captured images captured in various environments, and there is a problem that correct color matching cannot be performed.

Therefore, the present invention has been made in view of such problems, and is an image processing device that correctly identifies an image of the same object and suppresses deterioration of the color matching accuracy of the images of the objects captured by a plurality of imaging devices. The purpose is to provide. It also aims to provide the method and program.

The image processing apparatus for solving the above-mentioned problems has the following configurations. That is, it is determined in advance from the detection means for detecting at least one object included in a plurality of captured images captured by a plurality of imaging devices and having a similarity equal to or higher than a threshold value, and the detected objects. selection means for selecting an object occurrence frequency than the frequency has a lower attribute that is, the so color corresponding to the object of the plurality of captured images including the selected objects match, the plurality of captured images It is characterized by having a determination means for determining color correction information used for color matching, and a correction means for correcting the colors of the plurality of captured images by using the color correction information.

By correctly identifying the image of the same object using the image of the same object specified from the objects having attributes of a predetermined frequency or less, the deterioration of the color matching accuracy of the image of the object captured by a plurality of imaging devices is suppressed. be able to.

It is a block diagram which shows the hardware configuration example of the image processing apparatus of this embodiment. It is a block diagram which shows the functional structure example of the image processing apparatus of this embodiment. It is a figure which shows the example of the image feature amount used in the image processing apparatus of this embodiment. It is a flowchart explaining the flow of the color matching process in this embodiment. It is a flowchart explaining the flow of the query list creation process. It is a figure which shows an example of the RGB color space used in the query list creation process. (A) This is an example of a table that stores information about an imaging device that satisfies a predetermined condition. (B) This is another example of a table that stores information about an imaging device that satisfies a predetermined condition.

<First Embodiment>
The configuration of the computer device constituting the image processing device of the present embodiment will be described with reference to the block diagram of FIG. The image processing device may be realized by a single computer device, or each function may be distributed and realized by a plurality of computer devices as required. When it is composed of a plurality of computer devices, they are connected by a Local Area Network (LAN) or the like so that they can communicate with each other. The computer device can be realized by an information processing device such as a personal computer (PC) or a workstation (WS).

In FIG. 1, the CPU 101 is a Central Processing Unit that controls the entire computer device 100. The ROM 102 is a Read Only Memory that stores programs and parameters that do not require changes. The RAM 103 is a Random Access Memory that temporarily stores programs and data supplied from an external device or the like. The external storage device 104 is a storage device such as a hard disk or a memory card fixedly installed in the computer device 100. The external storage device 104 may include an optical disk such as a flexible disk (FD) or a compact disk (CD) that can be attached to and detached from the computer device 100, a magnetic or optical card, an IC card, a memory card, and the like. The input device interface 105 is an interface with an input device 109 such as a pointing device or a keyboard that receives a user's operation and inputs data. The output device interface 106 is an interface with the monitor 110 for displaying the data held by the computer device 100 and the supplied data. The communication interface 107 is a communication interface for connecting to a network line 111 such as the Internet and inputting / outputting data to / from the outside. The network camera 112 is an imaging device such as a plurality of surveillance cameras, and is connected to the computer device via the network line 111. The system bus 108 is a transmission line that connects each unit in a communicable manner.

Each operation described later is executed by the CPU 101 executing a program stored in a computer-readable storage medium such as ROM 102.

[Image processing device]
The image processing device of the present embodiment is a device that cuts out an image area of an object from image data acquired from image pickup devices such as a plurality of cameras, accumulates it as an image of the object, and processes the accumulated image of the object. .. When collating images using the image processing device of the present embodiment, a corrected image is created by matching the color of the image captured by each imaging device with the color of the image captured by the reference imaging device (reference camera) as a reference. Match the images after generating.

Specifically, an image of an object (target object) suitable for color matching is specified from images of objects captured by a plurality of imaging devices. Next, using the image of the target object, a color correction parameter is estimated to match the color of the image captured by each imaging device with the color of the image captured by the reference imaging device (also referred to as a reference camera). At the time of collation, the image captured by each imaging device and the color correction parameter are used to generate a corrected image, and then the collation is performed.

When specifying an object (target object) suitable for color matching, the conditions such as the similarity of the images of the objects being equal to or higher than a predetermined condition are satisfied. Therefore, even if the same object is actually projected between a plurality of imaging devices, it may not be adopted as an object used for color matching unless a predetermined condition is satisfied. By selecting the target object and using it for color matching in this way, the probability of incorrect color matching can be reduced.

In the present embodiment, a person is assumed as an object, but it is not necessary to limit the object to a person, and any object suitable for color matching may be used.

Hereinafter, the functional configuration of the image processing apparatus of the present embodiment will be described with reference to FIG.

The input unit 201 accepts the input of the image of the object. Specifically, an image of an object cut out from the captured image acquired from the network camera 112 is input. Alternatively, the frame number in which the object appears in the video acquired from the network camera 112 and the coordinate information of the image area of the object are input.

Further, the input unit 201 may accept the input of the captured image and cut out the object from the captured image by the cutting unit (not shown) of the image processing device. Alternatively, the image of the object may be cut out in advance in the network camera 112, and only the image of the object may be transmitted to the image processing device.

The storage unit 202 stores an image of an object input from the input unit 201. Specifically, the image data of the object is stored in the external storage device 104. In addition, as the metadata of the object, the ID of the object, the imaging time, and the imaging camera are associated and stored. Instead of the image data of the object, the video data obtained from the network camera 112 may be stored, and the frame number in which the corresponding object appears and the coordinate information of the image area of the object may be stored.

The collation condition input unit 203 specifies an image of an object to be used for collation. Specifically, the image of the object stored in the storage unit 202 is displayed on the monitor 110, and the image of the object to be collated via the input device 109 is specified. Alternatively, a screen for specifying the imaging time, the attribute value of the image feature, and the like is displayed on the monitor 110, and the image of the object stored in the storage unit 202 is narrowed down and displayed on the monitor 110 under the conditions specified by the input device 109. You may try to do it. Alternatively, the image data stored in the external storage device 104 may be designated as the comparison source, and the image data of the object stored in the storage unit 202 may be used as the comparison destination. The method of specifying the object image used for the collation of the present embodiment is not limited to these.

When specifying the image of the object to be collated, the image of the object to be collated may be two images, one for the comparison source and one for the comparison destination. Alternatively, a plurality of images to be compared may be selected for one image to be compared. Alternatively, only one image to be compared may be selected, and all the images of the objects stored in the storage unit 202 as the images to be compared may be automatically selected. The method of designating the image of the object used for collation in the present embodiment is not limited to these.

The collation result display unit 204 displays the result of collating the images of the objects specified by the collation condition input unit 203. The details of the collation process will be described later, but the collation process determines the degree of similarity indicating the degree of similarity between the images of the objects of the comparison source and the comparison destination. The collation result is displayed using this similarity. For example, if there is one image of each object of the comparison source and one image of the comparison destination, the similarity between the images of the objects is displayed on the monitor 110. Alternatively, if there is one comparison source object image and there are a plurality of comparison destination object images, the images of the comparison destination objects are displayed on the monitor 110 in order of similarity. Alternatively, the object images of the comparison destination having a degree of similarity equal to or higher than a predetermined value may be classified for each camera and displayed in order of the imaging time. The method of displaying the collation result of the present embodiment is not limited to these.

The collation unit 205 performs collation processing between the object image of the comparison source and the object image of the comparison destination. Specifically, the collation unit 205 collates by comparing the image features of the image of the object to be compared with the image of the object to be compared. In the present embodiment, a person is assumed as an object, and an image feature amount of the person as shown in FIG. 3 is used. The image feature amounts shown in FIG. 3 are examples, and the present embodiment is not limited to these image feature amounts.

The calculation method of these image feature amounts is as follows. SIFT (Scale Invariant Feature Transfer) is used as an image feature amount that does not change so much even if the color changes significantly, and as an image feature amount that is less dependent on color. Since SIFT is obtained from a luminance image, it is less dependent on color. The SIFT feature amount of the facial organ points in FIG. 3 is obtained by detecting the face area of a person, obtaining the organ points such as eyes and mouth in the face area, and obtaining the SIFT feature amount corresponding to each organ point. .. In the matching process using this feature amount, the matching unit 205 obtains the sum of the similarity of the SIFT feature amounts corresponding to each organ point in the face region in the images of the respective persons of the comparison source and the comparison destination. Alternatively, the collating unit 205 may determine the similarity using the number of organ points having a similarity equal to or higher than a predetermined value.

In the collating unit 205, the skin color or the hair color is obtained by predetermining the corresponding partial region from the face region obtained by the face detection and obtaining the color histogram of the partial region or the like.

Further, in the collation unit 205, the color of the clothes is determined in advance from the face region obtained by face detection to be the torso region corresponding to the torso, and the color histogram of the torso region is obtained. Desired.

Further, in the collation unit 205, a classifier is created so that various attributes such as age and gender, which are various attributes of a person, can be estimated when an image of a face area is detected by using a technique such as machine learning. Then, use a classifier to obtain various attributes. In addition, when the likelihood of an attribute is obtained, an attribute vector in which the likelihoods are arranged for each attribute value is created, and the similarity of the attribute is calculated by the similarity between the attribute vectors.

Since the similarity of SIFT features, color histogram, attribute vector, etc. can be obtained individually, the similarity between the image of the object to be compared and the image of the object to be compared can be obtained by appropriately weighting and summing. Ask.

When the collating unit 205 searches for and specifies an image of a target object used for color matching of an image captured by an imaging device, it is desirable to use an image feature amount having a low degree of dependence on color. Therefore, in the present embodiment, the image feature amount and the attribute of the person extracted from the luminance image shown in FIG. 3 are used. As a result, even when the colors of the images of the same object captured by the plurality of imaging devices are different, the probability that the same object captured by the plurality of imaging devices can be determined to be the same increases.

On the other hand, after color matching is performed, it is desirable to perform collation using all the image feature amounts. In this embodiment, all the features shown in FIG. 3 are used. As a result, by using the image feature amount that depends on the color after the color matching, it is possible to prevent the deterioration of the matching accuracy of the objects as compared with the case where the color matching is not performed.

The image feature amount used by the collation unit 205 may be calculated in advance for each object and stored in the storage unit 202. For example, the image feature amount before color matching is calculated and stored when the image of the object is input to the input unit 201. On the other hand, the image feature amount after color matching is stored in the storage unit 202 when the image feature amount after color matching is calculated.

The query list creation unit 206 creates a query list for specifying an image of the target object used for color matching. Specifically, images of some objects are selected as queries from the images of objects stored in the storage unit 202, prioritized, and a query list that is a candidate for the query is created. Since the query is also described as the image of the comparison source object, the query list is also described as the image candidate list of the comparison source object.

Next, an outline of the processing of the query list creation unit 206 will be described.

In order to correctly detect the same object that appears across multiple imaging devices, it is important to select the image of the object that is the comparison source. There are two conditions for the query list creation unit 206 to select the image of the object to be compared.

The first condition is that the image of the object to be compared is unique and is not easily mistaken for the image of another object. When attributes are extracted in advance by an extraction unit (not shown), statistics on the frequency of attributes that are less dependent on color (gender, age, presence / absence of glasses, etc.) are saved, and the query list creation unit 206 uses attributes with a predetermined frequency or less. Priority is given to the object with. As a result, the image of the object to be compared can be easily distinguished from the image of another object, so that the probability of making a mistake in the search can be reduced.

The second condition is that the color of the image of the comparison source object is suitable for calculating the correction parameter. If the color of the image of the comparison source object is monochromatic or achromatic, the color correction parameter based on the color of the image of the target object cannot be calculated. Therefore, it is necessary for the query list creation unit 206 to preferentially specify an object whose clothes color can acquire color information other than achromatic colors such as black and gray so that the color correction parameter can be easily calculated.

In consideration of the above two conditions, the query list creation unit 206 creates an image candidate list of the comparison source object with the priority (priority).

The target object identification unit 207 specifies an image of the target object used for color matching of the image pickup apparatus. Specifically, using the image of the object selected from the image candidate list of the comparison source object created by the query list creation unit 206 as a query, the object displayed on each imaging device that can be determined to be the same as the query is displayed. It is specified from the storage unit 202. Then, one image of an object (target object) used for color matching is specified for each imaging device. For example, when the similarity with the query is equal to or higher than a predetermined value, it is determined that the image of the target object is used for color matching, and the image is selected as a candidate for the image of the target object.

Further, the case where the number of image pickup devices that image the same target object is the largest is selected, and the final image of the target object is specified.

When the images obtained by all the imaging devices capturing the same target object are obtained, it is possible to calculate the color correction parameters of the captured images of all the imaging devices.

Even if all the image pickup devices cannot obtain an image of the same target object, color matching can be performed by specifying the image of the same object when the number of image pickup devices that capture the same object is the largest. It is possible to increase the number of image pickup devices to the maximum. Alternatively, if the number of image pickup devices that have imaged the same target object is equal to or greater than a predetermined value in the total number of image pickup devices, the image of the same target object is specified.

Also, if multiple imaging devices that cannot image the same object at the same time or time zone identify candidates for the target object to be displayed on multiple imaging devices at the same time or time zone, one of them is incorrect. It may be. In this case, the image of the specified target object can not be used for color matching.

The color correction parameter calculation unit 208 calculates a color correction parameter which is color correction information used for color matching. Specifically, using the images of the two objects, the color correction parameters are obtained so that the colors of the images of the two objects match. When obtaining the color correction parameter, for example, in the case of a person, the skin area of the face area is used, or the clothing area is used. The area of the target object used for calculating the color correction parameter is not limited to these.

The corrected image generation unit 209 generates a corrected image in which the color is corrected by using the color correction parameter.

There are the following methods for calculating the color correction parameters and generating a corrected image using the parameters. For example, as shown in Patent Document 2, a plurality of corresponding points in an object may be obtained, and a look-up table may be configured as a color correction parameter based on the difference in color between the corresponding points. When generating a corrected image, the corrected image is generated by referring to the look-up table for each pixel of the input image data and replacing the corrected image with a different color.

Alternatively, as shown in Patent Document 3, a color correction matrix may be obtained and used as a color correction parameter. When generating a corrected image, it is replaced with a different color by applying a color correction matrix for each pixel of the input image data. In the present embodiment, the color correction parameter calculation unit 208 may refer to the obtained distribution of the color correction matrix group and obtain an average matrix excluding data that deviates from a predetermined value or more, or may deviate from a predetermined value or more. You may choose the data that is not available and find the average matrix.

The method of calculating the color correction parameter in the present embodiment is not limited to these. Further, when calculating the color correction parameter, there may be a difference in the color system or the like.

[Flow of color matching process]
Next, the flow of the color matching process in the present embodiment will be described with reference to the flowchart of FIG. Hereinafter, it is assumed that the flowchart is realized by the CPU executing the control program. In this process, first, an image of a target object that is commonly displayed on a plurality of imaging devices is specified. Then, the color correction parameters of the image of the object captured by each imaging device are calculated. Then, a corrected image of the image of the object captured by each imaging device is generated, and the image feature amount of the corrected image of the object is calculated.

When the color matching process is executed, it is assumed that the image feature amount before the color matching is calculated in advance from the image of the object stored in the storage unit 202. In addition, this process is given a list of cameras subject to color matching processing.

Hereinafter, specific processing contents will be described with reference to the flowchart of FIG.

In step S401, a query list (candidate list of comparison source objects) is created. Specifically, the query list creation unit 206 selects images of some objects from the images of the objects stored in the storage unit 202 to form a query list.

The details of the processing of the query list creation unit 206 will be described with reference to FIG. As mentioned earlier, there are two conditions for an image of an object to be compared properly.

The first condition is that the image of the object to be compared is unique and is not easily mistaken for the image of another object. When attributes are extracted in advance, statistics on the frequency of attributes that are less dependent on color (gender, age, presence of glasses, etc.) are saved, and the query list creation unit 206 gives priority to objects with attributes that are less than or equal to the predetermined frequency. To identify. As a result, the image data of the comparison source object can be easily distinguished from the image data of other objects, so that the probability of making a mistake in the search can be reduced.

The second condition is that the color of the image of the comparison source object is suitable for calculating the correction parameter. If the color of the comparison source object is monochromatic or achromatic, the color correction parameter based on the color of the image of the target object cannot be calculated. Therefore, it is necessary for the query list creation unit 206 to preferentially specify an object whose clothes color can acquire color information other than achromatic colors such as black and gray so that the color correction parameter can be easily calculated.

The processes of step S501, step S502, and step S503 are processes for satisfying the second condition. The processes of steps S504 and S505 are processes for satisfying the first condition.

First, in step S501, the query list creation unit 206 extracts a pixel-by-pixel color histogram of the image of the object stored in the storage unit 202.

FIG. 6 shows an example in which the RGB color space is divided into a plurality of bins. In this embodiment, an example of simply dividing into 64 bins of 4X4X4 will be described. The diagonal line (dotted line) that passes through the origin of the color space represents the grayscale color, and is expressed as the achromatic axis. When there are many pixels belonging to a bin (also referred to as a grayscale color bin) including this achromatic axis, it is difficult to obtain a color correction parameter. In step S502, the stored image data of the object is narrowed down to the image data of the object in which the number of pixels belonging to the grayscale color bin is less than a predetermined ratio with respect to the total number of pixels of the image data of the object.

Further, the RGB value of the pixel of the image of the object is converted into a color space represented by a luminance signal such as YUV and a color difference signal, and the cumulative value of the luminance component Y for the pixel and the cumulative value of the absolute value of the luminance component UV for the pixel are obtained. .. The absolute value of the color difference component UV is set because it can take a negative value. Then, the cumulative value of the color difference component may be narrowed down to objects having a predetermined ratio or more with respect to the cumulative value of the luminance component.

Further, even if the pixels have a tint (the absolute value of the color difference component UV is large), it is still difficult to obtain the color correction parameter if only the pixels having RGB values and G and B are zero. It is necessary that each of the R, G, and B components has image data of an object to some extent.

Therefore, in step S503, first, the RGB values of the pixels of the image of the object are converted into the YUV space, and the pixel average U_mean of the absolute value of the color difference signal U and the pixel average V_mean of the absolute value of the color difference signal V are calculated. Then, based on the fact that these are images of objects having R, G, and B components that are both equal to or higher than a predetermined threshold value to some extent, those with a good balance of R, G, and B components are preferentially specified. It is possible.

Further, when the bin of the RGB color space of FIG. 6 is used, the histogram of the pixels of the image of the stored object is obtained. Then, by confirming that the color is not a neutral color that is a mixture of the three colors of R, G, and B, and that all the pixels of R, G, and B are included, whether or not to query the image data of the object. It is possible to judge whether or not.

Of course, the process is not limited to the above-mentioned method as long as it is a process of estimating an object that each of the components of R, G, and B has to some extent.

Further, when the attributes having low dependence on color shown in FIG. 3 are extracted in advance by the extraction unit (not shown), the frequency of the attributes having low dependence on each color extracted from the entire stored object is determined. Ask in advance. For example, with respect to the attribute of the presence / absence of glasses extracted from the image of each object, the frequency of with / without glasses is obtained from the entire stored object.

In step S504, the frequency of attribute combinations is estimated using the frequency distribution. If you do it simply, you can calculate the product of the frequencies of each attribute. For example, since there is a strong correlation between gender and mustache or chin, if it is determined to be female, the product of age and frequency with glasses may be used.

In step S505, the objects narrowed down from the infrequent ones are sorted, and the creation of the query list is completed.

As a result, it is possible to obtain a query list consisting of images of objects that are easy to acquire color correction parameters and are unlikely to cause erroneous search.

Step S402 is a loop for sequentially processing the queries in the query list consisting of image candidates of N objects obtained in step S401, and it is assumed that the queries are numbered in order from 1. First, i is initialized to 1 in order to refer to the list using the variable i. Further, in step S407, when i is equal to or less than the predetermined number of queries N, the process proceeds to step S408, and when this is not satisfied, the loop is exited and the process proceeds to step S409.

In step S403, the image of the object similar to the i-th query from the storage unit 202 is searched by using the image feature amount having a low dependence on color. Specifically, in the collation unit 205, the image of the object to be compared is used as the i-th query, and the image of the object to be compared is used as the image of the object stored in the storage unit 202. Then, collation is performed using an image feature amount having a low degree of dependence on color. For example, an image feature amount extracted from the luminance image shown in FIG. 3 is used for collation. Finally, the image of the object to be compared is associated with the similarity to obtain the search result.

In step S404, an image pickup device such as a camera in which each object is imaged is specified by paying attention only to the image of the object associated with the similarity of a predetermined degree or more. If the identified imaging device includes all the imaging devices (YES), the process proceeds to step S405. If not (NO), the process proceeds to step S406.

In step S405, information such as a query as an image of the target object and the number of imaging devices satisfying a predetermined condition is stored. The stored information is stored in the table shown in FIG. 7 (a). The table shown in FIG. 7A is an example.

In the table as shown in FIG. 7A, "in which imaging device an object similar to the query was found" is indicated by a circle for each query. For example, for the first query Q1, it is shown that objects similar to the six image pickup devices A, B, C, D, F, and G have been found. The second query, Q2, shows that similar objects were found in all eight imaging devices. Since the third query Q3 exits the processing loop in step S404, the search is not performed, and N / A indicates that there is no similar object.

On the other hand, when similar objects are not found in all the imaging devices in step S404, for example, it is the case where the search is performed by the first query Q1 in step S406. In this case, the image pickup device that satisfies the query Q1 as an image of the target object and the condition is stored.

Then, in step S407, it is determined by the expression i ≦ N whether or not there is the next query, and if there is, the process proceeds to step S408. In step S408, i is updated by the expression i = i + 1, updated to the next query candidate, and the process returns to step S403 again.

The process proceeds to step S409 when a similar object is found in all eight imaging devices in step S405.

FIG. 7A shows a case where images of similar objects are found in all eight image pickup devices in the second query Q2 of the query list.

FIG. 7B shows a case where an image of a similar object was not found in all the imaging devices from the first query Q1 to the last third query Q3 in the query list, and step S409 was reached. is there.

In step S409, it is determined that the image of the target object when the number of image pickup devices satisfying the condition is maximized is used for estimating the color correction parameter. In the case of FIG. 7A, an image of an object similar to query Q2 is used to estimate the color correction parameters of the eight image pickup devices. In the case of FIG. 7B, an image of an object similar to query Q1 is used to calculate the color correction parameters of the seven image pickup devices. At this time, it may be determined whether or not the number of imaging devices satisfying the conditions is equal to or greater than a predetermined number, and if the number is less than the predetermined number, the following processing may not be performed.

If a plurality of imaging devices that cannot image the same object at the same time or at the same time zone search for images of similar objects from the images of objects captured at the same time or at the same time zone, the search is performed. Most likely wrong. The process of excluding the image of the searched object from the estimation of the color correction parameter may be performed in step S404.

In step S410, an image that serves as a reference for calculating the color correction parameter is selected.

Regarding the selection of the image to be the calculation standard, the image of the object hit by the search always contains the same image as the query, and this can be used as the calculation standard of the color correction parameter.

Further, the image ID of the object hit in the search may be selected as the reference with the smallest ID.

In step S411, the color correction parameter for the image captured by each imaging device is calculated using the image of the object. The color correction parameter of the image captured by another imaging device is obtained from the imaging device that has captured the reference image determined in step S410.

Although not shown in FIG. 7, the image ID and the like of each object are also stored, and the image data and the like of the object can be referred from the storage unit 202 by this image ID. Therefore, the stored image can be used. it can.

In step S412, the color correction parameter is reflected. Specifically, the correction image generation unit 209 generates a correction image by matching the color of the image of the object captured by each imaging device with the color of the reference image by using the color correction parameter.

The following methods can be used for calculating the color correction parameter in step S411 and generating the corrected image in step S412. For example, as shown in Patent Document 2, a plurality of corresponding points in an object may be obtained, and a look-up table may be configured as a color correction parameter based on the difference in color between the corresponding points. When generating a corrected image, the corrected image is generated by referring to the look-up table for each pixel of the input image data and replacing the corrected image with a different color.

Alternatively, as shown in Patent Document 3, a color correction matrix may be obtained and used as a color correction parameter. When generating a corrected image, it is replaced with a different color by applying a color correction matrix for each pixel of the input image data.

The method of calculating the color correction parameter and the method of generating the corrected image in the present embodiment are not limited to these.

The generated corrected image is stored in the storage unit 202 so that the collation unit 205 can use it when collating the objects. Alternatively, the attribute information used by the collating unit 205 is calculated in advance. For example, the color-dependent image feature amount extracted from the color image shown in FIG. 3 is recalculated. Then, it may be stored in the storage unit 202.

In the example of FIG. 7A, it was possible to generate a corrected image of the image of the object captured by all the imaging devices, but in the example of FIG. 7B, the image of the object captured by the imaging device A was generated. Only will remain as an uncorrected image.

As described above, deterioration of the matching accuracy of the image of the object due to the difference in the color of the image of the image of the different imaging apparatus that imaged the same object is suppressed. For example, since the installation environment of a surveillance camera or the like is different, the colors of images captured by different surveillance cameras may differ even for the same object due to the influence of shadows, sunlight, different light sources, and the like. Deterioration of collation accuracy due to such an influence is suppressed.

Further, the corrected image generated by using this color correction parameter may be used not only for collation but also for display. For example, when displaying the result on the collation result display unit 204, the generated corrected image may be displayed. By doing so, the color difference of the display image of the same object is eliminated, and the viewer can comfortably view the image.

In the above, the case where one object is detected by two or more imaging devices has been described, but when a plurality of objects are detected, the same processing as described above is performed for each of the detected objects. Just do it.

<Other Embodiments>
In each of the above embodiments, an example in which a person is used as an object has been described. However, the object is not limited to a person. As an object, a car, a poster, a signboard, or the like may be used. Alternatively, a cart such as a supermarket may be used. In addition, a mascot character such as Kigurumi may be used. That is, when a person is used as an object as in each of the above embodiments, the same object (same person) is extracted, and color correction is performed using the color of the skin area, clothes, and the like. On the other hand, if an object other than a person is used as the object, color correction can be performed with high accuracy even if the same type of object is selected. For example, in the case of red cars of the same type, the color of the exterior of the car is generally the same, and by using such color information, it is possible to perform color correction with high accuracy even if the objects are not the same.

When using an object other than such a person, it is desirable to use an object suitable for the object as the feature amount of the object used in the object collation unit 205. For example, it is conceivable to use SIFT feature amount or the like as the image feature amount (corresponding to the upper left of FIG. 3) extracted from the luminance image.

The attributes extracted from the luminance image (corresponding to the upper right of FIG. 3) are determined by the object to be used. For example, in the case of a car or the like, it is conceivable to use a character string of a license plate or the like. Also, if it is a car, the type of car can be considered. In addition, if it is a poster or a signboard, it is conceivable to use a character string or the like that appears. When these objects are used at the same time, it is conceivable to use the type of the object.

Depending on the object, there may be objects with the same shape in different colors. For example, there are the same models of different colors. Therefore, when using such an object, it should be judged that they are similar by matching the attribute information. For example, in the case of a car, it is judged that they are similar only when the numbers on the license plates match.

Further, as the image feature amount extracted from the color image (corresponding to the lower left of FIG. 3), it is conceivable to use the color histogram of the object or the like. Alternatively, the color histogram for each region may be used. For example, in the case of a car or the like, since areas such as the bonnet, door, and tire can be defined as areas, it is conceivable to obtain color histograms for each of these areas. Similarly, it is conceivable to use a mascot or the like by defining areas such as the head and torso.

As the attribute extracted from the color image (corresponding to the lower right of FIG. 3), for example, in the case of a car, it is conceivable to use the information of the vehicle type in which the color is specified as the attribute.

As the feature amount of the object, a feature amount other than these may be used, and the present invention is not limited to these feature amounts to be used.

The color correction parameter calculation unit 208 obtains the color correction parameters so that the colors of the objects match each other. For humans, the skin area and clothing area were used. For objects other than the above person, the area to be used may be determined in advance. For example, in the case of a mascot, if the type of the mascot is specified, and if there is a part with abundant colors, it is conceivable to use that part. Alternatively, the color correction parameter may be obtained using the entire area.

The color correction parameters may be calculated by methods other than these, and the present invention is not limited to the calculation methods of these color correction parameters.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

201 Input section 202 Storage section 203 Matching condition input section 204 Matching result display section 205 Matching section 206 Query list creation section 207 Specific section 208 Color correction parameter calculation section 209 Corrected image generation section

Claims (7)

A detection means for detecting at least one object included in a plurality of captured images captured by a plurality of imaging devices and having a similarity equal to or higher than a threshold value.
A selection means for selecting an object having an attribute that appears less frequently than a predetermined frequency from the detected objects, and a selection means.
A determination means for determining color correction information used for color matching of the plurality of captured images so that the colors corresponding to the objects of the plurality of captured images including the selected object match.
A correction means for correcting the colors of the plurality of captured images using the color correction information, and
An image processing device characterized by having. The selection means selects an object in which the number of achromatic pixels is less than a predetermined ratio with respect to the total number of pixels in the image data of the object , based on the color histogram of each pixel in each of the detected objects. The image processing apparatus according to claim 1. It said object is a person, the determining means, based on the color information of the face area or torso region of the person in the image of the person, according to claim 1 or 2, characterized in that to determine the color correction information The image processing apparatus according to. Any one of claims 1 to 3, wherein the object is a person, and the attribute is at least one of the person's age, gender, the presence or absence of glasses, the presence or absence of a mustache, or the presence or absence of a mustache. The image processing apparatus according to the section. Said determining means among the selected object, based on the color information number of the largest object in the photographed image pickup device, one of claims 1 to 4, characterized in that to determine the color correction information The image processing apparatus according to item 1. A detection step of detecting at least one object included in a plurality of captured images captured by a plurality of imaging devices and having a similarity equal to or higher than a threshold value.
A selection step of selecting an object having an attribute that appears less frequently than a predetermined frequency from the detected objects, and a selection step.
A determination step of determining color correction information to be used for color matching of the plurality of captured images so that the colors corresponding to the objects of the plurality of captured images including the selected object match.
A correction step for correcting the colors of the plurality of captured images using the color correction information, and
An image processing method characterized by having. A program for causing a computer to function as the image processing device according to any one of claims 1 to 5.

Images