JPH08202733A - Method and device for retrieving image - Google Patents

Abstract

PURPOSE: To provide the method and device for retrieving image with which high-speed image retrieval can be performed by utilizing information on the area of the closed area of a retrieval instructed retrieving image. CONSTITUTION: When the desired image to be retrieved including the closed area including a color, position and shape or the like is inputted from a retrieving image input part 3, the color of the desired image to be retrieved is compared with that of the closed area while referring to the color indexes of respective stored images, for which the pairs of the colors and areas of areas in a color index storage part 23 are mapped on a color space, and the total of the areas of the areas of stored images having an approximate color is compared with the area of the closed area so that the image to be the retrieving object can be narrowed down. An area similarity calculating part 25 calculates the degree of similarity between the stored image narrowed down like this way and the retrieving image so that the image can be retrieved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval method and apparatus for retrieving stored images based on images for which retrieval is instructed.

[0002]

2. Description of the Related Art Conventionally, there have been the following methods for retrieving a desired image from an image database.

First, when an image is input to the database, the image is divided into a number of uniform areas, and each area is stored in an image file in the form of data that describes it. Then, when the searcher (operator) gives an instruction to search for an image from the image file, the image or graphic indicated by the searcher and the data describing each area stored in the image file are displayed. Check the matching between them and calculate the similarity such as area and second moment. Then, there was a method of selecting an image having a large value as a similar image (Material 1: Hirata Hara, “Image Search Using Outline Image”, Information Processing Society of Japan Study Group Material, DE92-
2).

When an image is searched by using the above method, all the stored images of the image file are matched with each area of the image and a designated image or figure to be searched (hereinafter, a searched image). Is required, and when the amount of accumulated images is large, the amount of calculation becomes enormous, and the time required for the search becomes long. Therefore, there is already a method of reducing the amount of images to be calculated for checking the matching and speeding up the search processing by narrowing down the stored images using the stored images and the color information of the search images. Introduced (Document 2: Hirata /
Hara, "Media Navigation in Hypermedia", Information Processing Society of Japan Human Interface Study Group HI-47-10 (1993)).

This method will be described in detail. (1) When registering an image in a database, area division is performed by a known method, and a representative color is obtained for each area. (2) This representative color is mapped for each area in a color space (for example, HSV) as shown in FIG. Here, mapping is performed for each image and each region. (3) At the time of search, the color information used by the searcher in the search image shown in the form of one or more closed regions (see FIG. 3).
Then, C1, C2 and C3) are similarly mapped to the color space. (4) An image that is not mapped in the vicinity of the color information used in the search image (for example, a spherical region centering on each color. The hatched portion in FIG. 3) is excluded from the search candidates, and the image in the vicinity is excluded. Only for the above, the narrowing-down process of performing the matching process between the subsequent search image and each region of the accumulated image is performed. In the example of FIG. 3, the image 3 is excluded from the candidates. Thus, the speed of search and the flexibility are adjusted by changing the size of the neighborhood of the color information.

[0006]

However, in the above search method, when an accumulated image contains a large number of fine areas having various colors, which image is designated as the search image. However, it is highly possible that the stored image includes a color similar to the color of the closed figure specified in the search image. As a result, there is a drawback in that an image including such a region becomes an object of distance calculation even if the image cannot be a search target, and the effect of the narrowing down process does not appear.

The present invention has been made in view of the above-mentioned conventional example, and an image search method and a method therefor capable of performing a high-speed image search by utilizing the information on the area of the closed region of the search image for which the search is instructed. The purpose is to provide a device.

Another object of the present invention is to provide an image search method and apparatus capable of performing an image search at high speed by narrowing down the images that can be searched and then determining the similarity only to the image. The purpose is to provide.

[0009]

In order to achieve the above object, the image retrieval apparatus of the present invention has the following configuration.
That is, an image retrieval device for retrieving a desired image from a plurality of accumulated images accumulated, an input means for inputting an image to be retrieved including a closed region, a region obtained by dividing each of the accumulated images and the retrieval desired. Comparing the color of the closed area of the image, comparing means for comparing the total area of the areas of the accumulated image having an approximate color and the area of the closed area, and the result of the comparison by the comparison means, the area of the closed area Determination means for determining an accumulated image whose total area is smaller than
A similarity determining unit that determines the similarity between the stored image determined by the determining unit and the image to be searched for, and a similarity determination unit that determines whether the similarity is greater than or equal to a predetermined value. Display means for displaying the determined image as a search result.

In order to achieve the above object, the image retrieval method of the present invention comprises the following steps. That is, an image retrieval method for retrieving a desired image from a plurality of accumulated images accumulated, a step of inputting an image to be retrieved including a closed region, a region obtained by dividing each of the accumulated images and the image to be retrieved. Comparing the color of the closed region, the step of comparing the total area of the region of the accumulated image having an approximate color and the area of the closed region, the result of the comparison, the area of the area rather than the area of the closed region. A step of setting a stored image having a small total as a search target; a step of determining a similarity between the stored image determined to be the search target and the image to be searched to obtain a similarity; and an image having a similarity of a predetermined value or more. Is displayed as a search result.

[0011]

In the above structure, when an image to be searched including a closed region is input, the colors of the divided regions of the stored image and the closed region of the image to be searched are compared, and the stored image having an approximate color is compared. Comparing the total area of the area and the area of the closed area. As a result of the comparison, the accumulated image whose total area is smaller than the area of the closed region is set as the search target, and the similarity between the accumulated image determined as the search target and the image to be searched is determined to obtain the similarity. , And operates so as to display an image with the calculated degree of similarity being a predetermined value or more as a search result.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

(First Embodiment) FIG. 1 is a block diagram showing the functional arrangement of an image retrieval apparatus according to an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes an image registration processing unit, which performs pre-processing described below before registering an image in the image database system 2. Two
Is an image database system, 3 is an example image (search image) drawn by a searcher and presented to the image search apparatus of this embodiment.
The input units 4 and 4 are display units for displaying search results.

Next, the configuration of the image registration processing section 1 will be described. Reference numeral 11 denotes a registration image input unit for inputting an image to be registered in the database. An area division processing unit 12 divides the input image into areas having similar colors. An area feature amount extraction unit 13 extracts various feature amounts (color, shape, etc.) for each area divided by the area division processing unit 12. A color index creation unit 14 creates a color index of the registered image from the color data for each area obtained by the area feature amount extraction unit 13.

Next, the structure of the image database system 2 will be described.

Reference numeral 21 denotes an image storage unit, which is a registered image input unit 11
The registered image input by the user is accumulated. Reference numeral 22 denotes a region feature amount storage unit that divides the registered image into regions and uses the region feature amount extraction unit 1
The feature amount of each area extracted in 3 is accumulated for each area of each registered image. A color index storage unit 23 stores the color index of each registered image created by the color index creation unit 14. A screening processing unit 24 compares the search image presented by the searcher to the image search apparatus of this embodiment with the color index, and whether or not the region similarity calculation unit 25 in the subsequent stage calculates the region similarity. Has been decided. Area similarity calculator 25
Performs matching between each region of the search image and each region of the accumulated image with respect to the image that has not been excluded from the processing target by the screening process in the screening processing unit 24, and calculates the similarity between them. An intermediate result buffer 26 temporarily stores the calculation result of the similarity for each image in the area similarity calculation unit 25. Reference numeral 27 denotes a candidate ranking determination unit, which refers to the similarity of each image stored in the intermediate buffer 26 to select an image having a high similarity and displays it on the display unit 4.

In this embodiment, as an image retrieval method from an image database, a retrieval image is created by a searcher (operator) designating one or more closed regions by specifying a position, a shape, and a color, and the closed regions are defined. An example of searching for an image stored in the image database using the position, shape, and color of is shown.

FIG. 4 shows the image retrieval apparatus of this embodiment.
6 is a flowchart showing a process when an image is registered in the image database system 2.

First, in step S1, an image to be registered in the image database system 2 is input from the registered image input unit 11 and stored in the image storage unit 21. Next, in step S2, the input image is divided into equal color areas by the area division processing unit 12. Although there are various methods for this division, for example, the method disclosed in Document 1 may be used.

Next, in step S3, various feature quantities are extracted for each area obtained by dividing the input image into the color matching areas in step 2, and the result is used as the area feature quantity to accumulate the area feature quantity for each input image. It is stored in the unit 22. As the area feature amount, the representative color of the area (HSV value), the area of the area, and the shape of the area (coordinates of each vertex of the polygon when the area is polygonally approximated) are used. Next, in step S4, a color index is created based on the area division result of the input image.

The method of creating a color index in the color index creating unit 14 of this embodiment will be described below.

The representative color of each area extracted in step 3 is mapped in the HSV color space as in FIG. At that time, for each mapped coordinate, the area of the corresponding region (the value refers to the ratio to the entire image, and hereinafter, "area" refers to this ratio) is stored. The mapped color space is used as a color index and stored in the color index storage unit 23 for each stored image. This is the end of the image registration process.

Next, a process for searching for images registered in the image database system 2 will be described.

FIG. 5 is a flow chart showing the image search processing in the image search apparatus of this embodiment. First, in step S11, a searcher inputs a search image or a figure (search image) by handwriting with a drawing editor or the like or using a scanner or the like. For example, when a searcher wants to search for an image as shown in FIG. 2 (a), based on the memory of the image, the color matching area in the image is changed as shown in FIG. 2 (c). As one closed figure 200, its shape and color are designated and input. This figure is called a search image.

Next, in step S12, the area Sc of the specified closed figure 200 from the search image of FIG.
Is calculated. Similar to the above, this area is calculated by the ratio of the area of the closed figure 200 to the entire search image. Next, in step S13, the color index storage unit 23 sets the already created color index for one image with the stored image stored in the image storage unit 21.
Read from.

Next, in step S14, the color of the search image is converted into an HSV value, and the HSV value is mapped in the same color space as the color index loaded in step S13, as shown in FIG.

FIG. 6 is a diagram for explaining the comparison between the color of each area in the accumulated image and the color of the search image. In FIG. 6, Pi indicates coordinates in the color space of the color of the region i in the accumulated image, and C indicates coordinates in the color space of the color of the search image. Further, Si represents the area of the region i.

Next, in step S15, a spherical surface 600 having a radius r (predetermined value) centered on C is drawn in this color space, and Pi contained inside the spherical surface is searched. Then, the area Si is obtained for all the coordinates Pi on the spherical surface 600, and the total Sp thereof is calculated.

Next, in step S16, Sp (total of Si) / Sc (area of closed figure 200) is calculated, and when the value is smaller than a predetermined value x (<1), that is, "in the stored image. , "The area of the search image" is considerably larger than "the total area of the areas having colors close to the search image", "the color matching area of the search image intended by the searcher is not in the accumulated image", that is, It is considered that "the stored image is obviously different from the image intended by the searcher", and it is determined that it is excluded from the target of the subsequent search processing, and the process proceeds to step S17 to load the color index of the next stored image. It returns to step S14.

If not, the process proceeds to step S18,
For the accumulated image that has not been excluded from the processing target in step S16, the regional characteristic amount of the image is calculated as the regional characteristic amount storage unit 2
2, the area similarity calculation unit 25 performs matching determination processing between each area and the search image. The value of the degree of similarity between the search image and the accumulated image obtained as a result of this matching determination processing is set to the intermediate buffer 26 for each accumulated image.
Save to.

Then, the process proceeds to step S19, and it is checked whether or not the above retrieval processing is completed for all the accumulated images accumulated in the image accumulating section 21. If not completed, the process proceeds to step S17 to start the process for the next accumulated image. When the processing for all the stored images is completed in this way, the process proceeds to step S20, where the candidate ranking determination unit 27 sorts the similarities (stored in the intermediate buffer 26) between the stored images and the search image, and the area matching processing. Among the images that have been subjected to the above, those having a high degree of similarity are displayed on the display unit 4, and the entire search processing is terminated.

By performing the above processing, it is possible to efficiently narrow down the number of accumulated images to be subjected to the area matching processing that requires a long processing time, and as a result, the search time is shortened.

(Second Embodiment) In the first embodiment described above, a sphere centering on the color of the search image is drawn in the color space of the color index, and a region in the accumulated image having a color contained in the sphere is drawn. However, since it takes time for the search, the search time can be further shortened by simplifying the process as follows.

When creating a color index for each stored image, the HSV color space is divided into appropriate blocks as shown in FIG. 7A, and the area of the area having the color contained in the block is divided in the color space. For each block, and the value (Tj for block j) is stored in the color index storage unit 23 as a color index in block units.

At the time of searching for an image, the color of the searched image is mapped on the color space as in the first embodiment described above,
As shown in FIG. 7B, for all blocks included in a distance within a predetermined value r2 from the coordinates, the value obtained by adding the values of Tj corresponding to the block is set as Tp. And Tp
If the value of / Sc is smaller than the predetermined value x2 (<1), it is considered that the stored image is clearly different from the image intended by the searcher, and the subsequent processes are excluded from the target as in the above.

(Third Embodiment) In the first and second embodiments described above, the search image is input as a graphic having one closed region, but the present invention is not limited to this, and a plurality of images may be input. You may input as a closed area.

Even in such a case, the above-mentioned (step S17 in the flowchart of FIG. 5 of the first embodiment) is performed for each closed region.
The processing similar to the above) is performed, and the processing from step S18 may be performed only on the accumulated images that have not been excluded from the search target for all closed regions.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. The present invention can also be applied to the case where it is achieved by supplying a program for implementing the present invention to a system or an apparatus.

As described above, according to the present embodiment, by using the data of the area of the closed region designated as the search image, the screening work for narrowing down the amount of the image which is the target of the region matching process in the subsequent stage. Can be performed more effectively, and as a result, the time required for search can be significantly reduced.

[0040]

As described above, according to the present invention, there is an effect that a high-speed image search can be performed by using the information regarding the area of the closed region of the search image instructed to search.

Further, according to the present invention, there is an effect that an image can be searched at high speed by narrowing down the images which can be searched and then determining the similarity only to the image.

[0042]

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of an image search device according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining image retrieval in the present embodiment, in which (a) is a diagram showing accumulated images that a searcher wants to retrieve,
(B) is a figure which shows the example which divided the accumulated image into a color matching area | region, and (c) is a figure which shows an example of the search image which the searcher produced.

FIG. 3 is a diagram for explaining a conventional search process.

FIG. 4 is a flowchart showing a process of registering an image in an image database system in the image search device according to the present embodiment.

FIG. 5 is a flowchart showing an image search process in the image search apparatus of this embodiment.

FIG. 6 is a diagram showing a color of each area in an accumulated image according to the present embodiment,
It is a figure for demonstrating comparison with the color of a search image.

FIG. 7 is a diagram for explaining a second embodiment of the present invention,
(A) is a diagram showing a state in which the HSV color space is divided into blocks, and (b) is a diagram for explaining a process of searching for a color close to a search image in the block unit.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 image registration processing unit 2 image database system 3 search image input unit 4 display unit 11 registered image input unit 12 region division processing unit 13 region feature amount extraction unit 14 color index creation unit 21 image storage unit 22 region feature amount storage unit 23 colors Index storage unit 24 Screening processing unit 25 Similarity calculation unit 26 Candidate ranking determination unit

Claims (8)

[Claims] 1. An image retrieval method for retrieving a desired image from a plurality of accumulated images, which comprises: inputting an image to be retrieved including a closed region; Comparing the color of the closed area of the image to be searched, comparing the total area of the area of the accumulated image having an approximate color and the area of the closed area, the result of the comparison, than the area of the closed area A step of setting a stored image having a small total area as a search target; a step of determining a similarity between the stored image determined to be the search target and the image to be searched to obtain a similarity; And a step of displaying the above image as a search result. 2. The stored image is stored by dividing each of the plurality of stored images into a plurality of regions and mapping a combination of a color and an area of each of the plurality of regions in a color space. The image retrieval method according to claim 1, characterized in that 3. The image search method according to claim 1, wherein the image to be searched includes a closed region having a color, a position, and a shape. 4. The image search method according to claim 1, wherein the search results are sorted and displayed in descending order of similarity. 5. An image retrieval apparatus for retrieving a desired image from a plurality of accumulated images, comprising input means for inputting an image to be retrieved including a closed region, and a region obtained by dividing each of the accumulated images. Comparing the color of the closed region of the image to be searched, the total area of the accumulated image having an approximate color and the area of the closed region, the comparison result by the comparison unit, the closed Judgment means for judging an accumulated image in which the total area of the areas is smaller than the area of the area as a search target, and a judgment of matching between the accumulated image judged by the judgment means with the image to be searched is judged. And a display unit for displaying an image whose similarity is determined to be greater than or equal to a predetermined value by the similarity determination unit as a search result. Apparatus. 6. The accumulated image is stored by dividing each of the accumulated images into a plurality of regions and mapping a combination of a color and an area of each of the plurality of regions in a color space. The image retrieval device according to claim 5, wherein the image retrieval device is an image retrieval device. 7. The image input by the input means is
The image search device according to claim 5, further comprising a closed region having a color, a position, and a shape. 8. The image search apparatus according to claim 5, wherein the display unit displays images searched by sorting in descending order of similarity.