KR100319153B1 - Image query system using authority relevance feedback - Google Patents

Abstract

The present invention relates to a self-learning feedback image retrieval system according to a user's authority level, and the image retrieval system makes the characterization of the image more accurate as the image is used for retrieval. Image retrieval method and its data with self-grown weight information that enables the image to develop intelligently by reflecting its own characteristics in a way that restricts the access rights or the degree of the image information according to the user's grade To a production method.

The present invention provides a space in which an image retrieval system can have weighted information reflecting its own characteristics for each image, and updates the weighted information each time it is used for retrieval. Self-growth image retrieval method and image data structure that is developed into an image having more precisely searched characteristics, and the degree of updating the weight is different depending on the user's rating when updating the weight information in the image retrieval system. To have the reliability value of the image characterization information by having a reliability code that can indicate the reliability of the characteristic value of the image in the image data in the image retrieval system. Image retrieval and its data generation Provided.

Description

Image search system based on user's permission level {IMAGE QUERY SYSTEM USING AUTHORITY RELEVANCE FEEDBACK}

The present invention relates to a self-learning feedback image retrieval system according to a user's authority level, and the image retrieval system makes the characterization of the image more accurate as the image is used for retrieval. Image retrieval method and its data with self-grown weight information that enables the image to develop intelligently by reflecting its own characteristics in a way that restricts the access rights or the degree of the image information according to the user's grade To a production method.

In a conventional image retrieval method, a similarity search is performed using features such as a color, a texture, and a shape of an image.

In this case, the importance of each feature varies according to the characteristics of the image to be searched for, and the importance of one feature differs depending on the partial elements existing therein, for example, in the color elements, such as red and green. Can be.

However, because different importance characteristics are not known for each image, we used a method that allows the user to directly indicate the importance when searching (US PATENT 5,579,471; Image Query System and Methhod) (Virage Image Search Engine). The disadvantage is that it is difficult to directly determine the importance of each low-level feature such as color, texture, and shape.

Meanwhile, a method of automatically calculating weights for each feature type by calculating similarity between them by registering images similar to the reference image and images similar to the reference image at the time of search (Relevance Feedback Techiniques in Interactive, YongRui, SPIE Vol. .3312).

However, since this method does not maintain weight information once a search is completed, even if the same image is found as a reference image, the weight calculation must be repeated each time.

The present invention provides a self-growth image retrieval method and an image data generation method that have an image retrieval characteristic as the image is used for retrieval in the image retrieval system.

The present invention also provides a space in which an image retrieval system can have weighting information reflecting its own characteristics for each image, and updates the weighting information each time it is used for retrieval. The present invention provides a self-growth image retrieval method and image data generation method that develops into an image having a more precise retrieval characteristic.

In addition, the present invention is to ensure that the image retrieval system has the information (Authority Code) to vary the degree of updating the weight according to the user's rating when updating the weight information in the image retrieval system, thereby increasing the reliability An image retrieval method and its data generation method are provided.

In addition, the present invention provides an image retrieval method and the data so that the image has a reliability value of the image characterization information by placing a reliability (Reliability Code) that can represent the reliability of the characteristic value of the image in the image data in the image retrieval system Provides a creation method.

1 illustrates a partial structure of an image in the image retrieval system of the present invention.

2 is a diagram showing the structure of image data in the image retrieval system of the present invention;

3 is a flowchart showing an image retrieval method according to the present invention.

4 is a diagram showing a data structure of an embodiment in which reliability can be variably applied in the image retrieval system of the present invention.

FIG. 5 is a diagram showing an example of updating the weight information in the image by FIG. 4. FIG.

6 is a diagram showing a data structure of an embodiment to which reliability can be fixedly applied in the image retrieval system of the present invention.

FIG. 7 is a table showing an example of updating the weight information in the image by FIG. 5. FIG.

According to the present invention, a weight space (data area) of a feature used in a search is included in an image (data) itself, and a predetermined initial value is given in advance at the initial generation of the image.

The feature weight that an image has may be a weight for each feature type such as a color, texture, or shape, a weight for each element in a feature for each type, such as an i-th color element, or a weight for a feature for each type. It can also be a weight for both and elemental features.

In addition, the present invention includes the weight space of the feature used in the search itself in the image itself, the value of the weight space is updated each time the user uses the image for the search and maintains the updated value.

In addition, the present invention is to limit the right of using the information according to the user grade level or to differentiate the degree of influence on the update to the user grade level in order to limit the possibility of updating the weight information updated each time the image is used for image search This information is included with the weight information in the image itself as the authority code, or the image does not have this authority code, but instead of the authority code in the search system. ) To determine the user's privilege level.

Therefore, an image (data) having the structure of the present invention can be developed to have a weight that is more appropriate to itself as it is used for a search by changing the weight of itself correctly whenever the image is used for a search.

As such, the image itself contains weight information of the feature used for the search. Therefore, regardless of the place or application (image search engine), regardless of the place or application, the current weight of the self-grown image is used. You will be able to perform more effective searches.

In addition, whenever a weight is used in a search, the weight can be adaptively changed more appropriately as an effective response to the subjective and objective query of the search.

Therefore, the present invention can be used as an image format that can use the concept of 'Relevance Feedback' that adjusts the weight according to the feedback of the user. The feedback of the user is to provide the user's review information according to the search result of the image. Refers to.

Meanwhile, in the present invention, as described above, whenever the weights having initial values are used as the reference image when the image is searched or used as the correct search result, the similarity between the result and the reference image is calculated by calculating the similarity. The image is self-grown in such a way that the weight of the feature type or element that increases is further increased (updated) and the weight of the feature type or element that is not further increased (updated).

Alternatively, the present invention adapts to a user by constructing a list of similar images that are not similar to a list of similar images according to a user's needs, and intentionally learning the weight values based on the similar image list and the dissimilar image list. The image can be grown in a positive direction.

On the other hand, in the present invention, since the image itself has weight information of the feature used for the search, and the weight is updated every time the user gives feedback, there is a fear that the image may grow in the wrong direction if the update is allowed indefinitely. do.

For example, if two different images are regarded as the same image due to a user's error, the image used for the search by the user is updated in a different direction and the reliability of the result of self-growth of the image is updated. Will be lacking.

In consideration of such a case, in the present invention, the authority code is included in the image (data).

Alternatively, the image does not have an authority code, but has an authority code on the search system side, and the user has the authority to use a predetermined image (weight update) or update the weight when using the image. It can also limit and differentiate the extent of the impact on the environment.

In this case, by using the authority code that determines whether or not to update the weight of the image, the user mis-learns or incorrectly judges the dissimilar image as the result of the correct search. The weight of the image can be prevented from developing to an incorrect value.

Meanwhile, in the present invention, the reliability information (Reliability Code) that determines how well the image is characterized is included in the image (data) itself, and as described above, the image has a weight space of the feature used for the search and at the same time the weight thereof. Have confidence in

In this way, if the image has self-growth and confidence in the weight, it has the following advantages.

It is assumed that one image is generated, copied into a plurality of images, and then characterized (weighted update), respectively, through different applications.

The image 1 characterized by the application program 1 will then have a different weight from the image 2 characterized by the application program 2.

If you want to retrieve these images 1 and 2 from application 3 and use them for image retrieval, image 1 and image 2 are characterized by different weights, so you have to decide which of these two images (data) to use.

In this case, if the image 1 and the image 2 have the reliability of the weights that are each characterized by their own application 1 and application 2, you can select the image with high reliability and use it in the search.

Alternatively, a new weight obtained by combining two weights of the image 1 and the image 2 in proportion to the reliability may be obtained.

In this case, the credibility may be a method of processing the image with higher credibility as the image having been trained by a user having a higher grade level is used based on the authority code.

Alternatively, a method of processing the reliability with higher reliability may be used for a higher frequency image trained by a user from a higher grade level to a certain number of grade levels.

Alternatively, it may be adaptively determined according to the characteristics of the image retrieval system, the viewpoint of the user, and the like.

Such reliability can be maintained in the image (data) itself together with the weight information and authority code as described above. In the case of an image that does not have authority code, only the weight information and the reliability image You can also keep on.

In addition, the reliability may be set variably, or the number of grades and the degree of influence that the grades may have may be set in advance (to be described later with reference to FIGS. 4 to 7).

As described above, in the present invention, the image data has a weight space for each feature of the image, has reliability of the weight, and whenever the image is used for the search, the weight is updated to self-grow into a more accurate search element. do.

Therefore, the image retrieval system gradually adapts to the execution of the image retrieval and makes intelligent development in a state optimized for the retrieval system (or user).

Hereinafter, the operation of the present invention will be described in more detail with reference to the embodiments of FIGS. 1 to 7.

First, Fig. 1 shows a substructure of the image (data) according to the present invention (except information on a feature).

Image description for relevance feedback (101) for reflecting user response information is performed as follows.

first; Image feature information 103a used for the actual search and their updateable weight information 103b.

This information is updated each time the user gives feedback, and the weight information 103b is weighted by feature of each type or by feature element.

second; When an image is used for a search, it has an authority code 104 for determining whether or not to update the weight information or affecting the update. The authority code is an authority code. It has a number of authority levels (105) and an authority level (106) of authority levels.

Here, the number of the privilege levels indicates the number of the grade levels, for example, when the image 1 has five grades from grade 1 to grade 5, and the privilege grade level permits weight update of the image and As the information indicating the degree, the update is allowed only to the users of the 1st and 2nd class, and each update degree is set such as 0.95 for the 1st class user and 0.85 for the 2nd class user.

The authority code may have the image itself, but the search system has the authority code and the image data does not have this information; Only reliability information 107 (Reliability Code) may be included together with the image characteristic information 102.

The reliability information 107 may be represented by the number of uses by Level i of which the weight of the image is updated by the user at the i level.

The image as described above may appear in the format shown in Fig. 2 in the actual image format.

That is, the weight information indicating the weight of each feature type or element, the number of grade levels indicating the number of grade levels, the grade level password having a password of each level, and the frequency of use for each grade level A data format with a reliability that represents

Here, the password is given as a different password for each level that is allowed to access the image based on the authority level information (Authority Code), and functions as a security key that allows the user to access the image by inputting the password.

This prevents access to image data by lower-level users.

When having an image data structure as shown in Fig. 1 and Fig. 2, the image creator (creator) can divide the permissions n at the time of characterization of the image, assign a password for each level, and the image provider gives the user the user's credentials Password will be supplied according to

The image user can use the image by receiving the corresponding password according to the qualification (permission level) to characterize the image, and the image also changes the characterization information according to the user's level as described above.

As the user of each grade uses the image, image characterization is performed, and the number of times of each grade level is recorded as the reliability.

As described above, when two identical images are learned and self-grown by different histories, the reliability of their characterization values (updated weights) is calculated as follows using the reliability of each image.

That is, the product of the number of weight updates (Nk) of the corresponding image of the k-class user and the coefficient (ak) assigned to the k-user class (depending on the update, which can be differentially applied according to the user's rating) (ak * Nk) is summed by the number of user grades (1 to n) and multiplied by a predetermined reliability coefficient (a) to obtain the reliability of differently characterized images.

The reliability can be used to determine which image to use.

3 shows the procedure of image characterization according to the results of image retrieval and retrieval with images of the data structure described above.

When the reference image is selected, the user searches for an image identical or similar to the reference image to obtain an image of the search result, and the user feeds it back.

Based on the structure of Figs. 1 and 2, the retrieved image is compared with the authority level of the current user and the authority level (an authority level that can modify (update) weight) of the image.

If the current user's permission level is lower than the image's permission level, the weight update of the image is not allowed. If the current user's permission level is a level that allows the image to be modified, the similarity between the reference image and the search result is measured.

The weight of the reference image and the search result is calculated and the weight of the existing reference image and the search result is modified in proportion to the user's rating.

Here, the weighting method is updated by reflecting the difference between the weighted value of the k-th feature obtained by the current image search and the average of the calculated weights obtained by the current image search to the weight before the k-th feature is corrected.

For example;

Wk (t) = Wk (t-1) + ΔWk, (ΔWk = a (Wnew_k-M (Wnew))

Wk (t): Modified weight of k th feature

Wk (t-1): weight before modification of the k th feature

Wnew_k: Calculated weight of k th feature obtained from current image search

M (Wnew): Average of calculated weights obtained by local image search

a: coefficient indicating the degree of weight correction according to user rating

On the other hand, as another method for modifying the weight value in the present invention, in addition to the method of using the reference image and the result image described above, it may be performed using a similar image list or dissimilar image list obtained during the search process.

For example, if we search for similarity, we measure the similarity between the images, determine the similarity in the similarity, give more weight to the feature that has high contribution, measure the similarity between dissimilar images, The weight can be modified in such a way as to further increase the weight for the feature that contributes to determining that it is not similar.

In addition, the reliability can be used as a method for modifying the weight in the present invention.

The method of updating the weight using the reliability includes the reliability (Rc) of the trained image up to now, the weight (Wc) set for the reliability, and the degree of influence on the weight correction corresponding to the rating of the current feedback user ( Rt) and weights (Wt) for its grade,

The weight to be updated = (RcWc + RtWt) / (Rc + Rt).

In this way, the weight of each feature of the image (data) can be updated while reflecting the user's grade in consideration of reliability.

4 shows a data structure in the case of variably setting the reliability code in the image structure of the present invention described above.

The reliability information 401 (Reliability Descriptor) is obtained by the number of levels (402), the degree of effect of the i level to the weight correction (403), by the user of the i level. It has a number of updates 404.

As shown in Fig. 4, Fig. 5 shows how the information of the present invention in the image can be made when the reliability information is variably configured.

In FIG. 5, there are five levels of authority level, in the case of the first level, the update rate is recorded 8 times by the user of the first level with the weight update degree of 1.0. In the case of the second level, the weight update is 0.7. 14 updates were recorded by the second-level users with the degree of accuracy. In the third grade, 30 updates were recorded by the third-level users with a weighted update rate of 0.5. In case of a weight update of 0.3, the number of updates was recorded 23 times by the user of the fourth grade. In case of the grade 5, the number of updates was recorded four times by the user of the fifth grade with a weight of 0.1. It was.

Here, the number of each privilege level and the value that can be updated by the user of each privilege level are freely set.

In the case of Figs. 6 and 7, the authority level is fixed to n (n = 10 in this example), and the update impact level value corresponding to each level is pre-assigned, and then the grade used by the system is set to an appropriate update impact level value. This is an example to use after assigning selection.

As shown in Fig. 6, how the information of the present invention in the image can be made when the reliability information is fixed is illustrated in Fig. 7 as an example.

Referring to FIG. 7, the number of grade levels is preset to 10, and the degree of influence that can be updated by the user of each grade is equally divided into values between 0.1 and 1. FIG.

In this way, the system may be used for all ten levels, or only a predetermined number may be partially used.

This is, of course, effective when the privilege level is retrieved from the system to control access to the image (weight update permit / prohibit or update).

In FIG. 7, the eighth update frequency was recorded at the update degree of 1.0 for the first grade, the 14 update frequency was recorded at the update degree of 0.7 for the second grade, and the 30 update rate was 0.5 for the third grade. The number of renewals was recorded. In the case of the fourth grade, 23 renewals were recorded with an update of 0.3. In the case of the fifth grade, four updates were recorded with an update of 0.1.

When the reliability information structure of FIG. 7 is used, this is used as update information for five grades of users as shown in FIG. 5 in the system.

In FIG. 5, the user of the first grade corresponds to the grade corresponding to 1.0 in FIGS. 7 and 7 because the update degree is 1.0. In FIG. 5, the user of the second grade corresponds to the grade corresponding to 0.7 in FIG. In FIG. 5, the third grade user corresponds to a grade corresponding to 0.5 in FIG. 7 because the update degree is 0.5. In FIG. 5, the fourth grade user corresponds to a grade corresponding to 0.3 in FIG. In FIG. 5, the user of the fifth grade has an update degree of 0.1, so in FIG. 7, the number corresponding to the lowest 0.1 is recorded in the corresponding position.

In this case, in the reliability information structure of FIGS. 6 and 7, only 5 grade levels are used out of the 10 fixed grade levels, and whether or not to update each grade level is as currently described in the system. You can do this by searching for the user's permission level.

According to the present invention, by including feature weights in an image, the more the image is used for a search, the more the image is characterized to have a better search result, thereby enabling an effective search.

In addition, the present invention prevents side effects due to image characterization by including an authority code that can give a user's authority to image characterization in an image to prevent indiscriminate image characterization.

In addition, the present invention enables the user to determine the characterization state of the current image by including reliability information in the image, which can know which image is more effectively characterized when the same images are characterized in different environments.

In addition, the present invention is intelligently developed so that the weight of each feature of the image is updated every time the image is searched to have a feature that allows a more accurate search, thereby improving the search speed of the image search system and improving the reliability of the search results. Bring.

Claims (24)

(One-time correction) An image format in which the weight of image features is adjusted according to the feedback of the user who provides the user's review information according to the search results of the image, and the weight of the information representing the characteristics of the image can be recorded. An area is included in the image data, and the weight is updated each time the user feedback is made, and is maintained in the image data itself. (1 time correction) The image according to claim 1, further comprising an authority code for determining whether the user can update the weight and whether the weight can be updated. How to generate data. (1 time correction) The reliability information according to claim 1, wherein the weight is characterized according to whether or not the user has authority to update the weight and the degree to which the weight can be updated. (Reliability Code) further comprising the image data generation method. (1 time correction) The method according to claim 2, wherein the authority code divides the authority into n grade levels, and the number of divided grade levels and a user of each level receive permission to update the weight information. Image data generation method comprising a password that can be made. (Revision once) The image data generation according to claim 3, wherein the reliability code is recorded as a value differentially applied by the user according to the authority to update the weight and the grade according to the degree. Way. (Revision once) The image according to claim 3, wherein the reliability code comprises an arbitrary set value which is variable according to a user's authority to update the weight and a grade according to the degree. How to generate data. (Revision once) The reliability code according to claim 3, wherein the reliability code is selectively made from a fixed value in consideration of a user's authority to update the weight and a grade according to the degree. How to create image data. (One time correction) The image data generating method according to any one of claims 3 to 7, wherein the reliability code is recorded as an update frequency of the weight information updated by a user. (One-time correction) An image format that adjusts weights of image features according to a user's feedback that provides a user's review information according to a search result of an image. An image retrieval system, characterized by having authority level information (Authority Code) for determining authority and degree of renewal thereof. 10. The image retrieval system of claim 9, wherein the authority code is included in image data. The image retrieval system according to claim 9, wherein the authority code is included in the image retrieval system separately from the image data. 10. The system of claim 9, wherein the authority code includes an authority divided into n grade levels, the number of divided grade levels, and a password that allows users of each level to receive the update permission of the weight information. Image retrieval system, characterized in that made by. (Single correction) Reliability indicating the reliability of the weight of information representing the characteristics of the image in the form of an image format in which the weight of the image features is adjusted according to the feedback provided by the user who provides the user's review information according to the search result of the image. Image data generation method characterized in that it further comprises a Reliability Code. (Single correction) The image data generating method according to claim 13, wherein the reliability information is recorded at an update frequency of the weight by a user. An image retrieval system in which a weight of an image feature is adjusted according to a user's feedback that retrieves an image and provides the user's review information on the search result. An area capable of recording weights for information representing characteristics of an image has an image of a structure included in image data, and authority level information for determining the authority to update the weight and the degree of the update. Has a Reliability Code indicating the reliability of the weight, and records the image by modifying the weight according to the user grade by the authority code whenever the image is used. system. The image retrieval system according to claim 15, wherein the weight is measured by measuring similarity between the reference image and the retrieval image, and correcting and recording the similarity according to the similarity. The image retrieval system of claim 15, wherein the authority code is included in an image. 16. The image retrieval system according to claim 15, wherein the weight is limited by the user grade according to the authority code. 16. The method of claim 15, wherein the authority code (Authority Code) divides the authority into n class levels, and includes a number of divided grade levels and a password that allows users of each level to receive the update permission of the weight information. And image retrieval is restricted according to the user grade by the password. 16. The image retrieval system according to claim 15, wherein the weight is updated by the difference between the average of the calculated weights of the image features obtained by the current image search and the calculated weight of the k th feature. The image retrieval system according to claim 15, wherein the weight is set according to reliability. The image retrieval system according to claim 15, wherein the weight is differentially applied to the reliability according to the user's rating and set according to the reliability by the rating. 16. The image retrieval system according to claim 15, wherein the reliability is set in accordance with the grade of the user. The image retrieval system according to claim 15, wherein the reliability is set in proportion to a user's rating.