JPH10320401A - Key word giving method and automatic key word giving device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily give a key word in a short time. SOLUTION: An automatic key giving function deriving process part 51 derives an automatic key giving function from the image quantity of a key word given to a specified reference image and the feature quantity of the reference image and stores it in an automatic key giving function storage part 52. A feature quantity calculation part 53 of an image data registering process part 5 calculates the feature quantity of a registered image. Further, a key word estimation part 54 finds an image quantity from the above-mentioned calculated feature quantity and automatic key giving function, estimates an key word on the basis of the found image quantity, and gives it to the registered image data. Thus, the key word is only registered in a key word storage part 3 in advance together with the image quantity and then the derivation of the automatic key giving function and the addition of the key word to the registered image can automatically be performed, so that the key word can easily be given in a short time.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a keyword assigning method for automatically assigning a keyword to an object when the object is stored together with the keyword, and an automatic keyword assigning apparatus using the keyword assigning method.

[0002]

2. Description of the Related Art With the rapid spread of computers, electronic information handled by people is rapidly increasing. Such a large amount of electronic information is generally stored together with a keyword, and by specifying this keyword, desired electronic information can be searched.

As a method of assigning a keyword to such electronic information, there is a method described in Japanese Patent Application Laid-Open No. 6-295318. In this keyword assignment method, for example, when a keyword is assigned to image data of a human face photograph and registered, the following is performed.

[0004] In advance, "face length" and "round face" related to human faces
Are registered in advance. In addition, for each keyword candidate, matching degree calculation information and a threshold used when calculating the matching degree are registered. For example, when the keyword candidate is “surface length”, the adaptability calculation information is “aspect ratio of contour”. Next, one keyword candidate “surface length”, the fitness calculation information and the threshold value for the keyword candidate “surface length” are read, and the fitness of the keyword candidate “surface length” is determined from the feature amount of the image data to which the keyword is to be assigned. Computation degree is calculated based on the calculation information. Then, if the obtained matching degree is equal to or larger than the threshold value of the keyword candidate “surface length”, it is determined that the keyword candidate “surface length” is a keyword to be added to the image data. Hereinafter, the same operation is performed for all of the registered keyword candidates, and the keyword to be assigned is determined. Thus, an objective keyword is assigned irrespective of the subjectivity of the registration worker.

[0005]

However, in the above-described conventional keyword assignment method, the calculation of the fitness based on the fitness calculation information and the comparison with the threshold must be repeated by the number of registered keyword candidates. Without
There is a problem that it takes time to register image data.
In addition, it is necessary to set and register appropriate relevance calculation information and a threshold value for each keyword candidate in advance, and the keyword assignment operation itself is performed automatically, but it is troublesome and difficult to construct a keyword assignment system. There is also a problem that it involves complicated work.

An object of the present invention is to provide a keyword assigning method capable of assigning a keyword easily and in a short time, and an automatic keyword assigning apparatus using the keyword assigning method.

[0007]

In order to achieve the above object, a keyword assigning method according to the first aspect of the present invention relates to a method for assigning a keyword and an image quantity obtained by quantifying the keyword's feeling on a predetermined scale to each other. The keyword is stored in the keyword storage unit, and the object to which the keyword is assigned is referred to. The image amount of the keyword assigned to the referenced object and the feature amount of the referenced object are used to calculate the image. A relational expression between the amount and the characteristic amount is derived, a characteristic amount of the registration target object is calculated, an image amount is obtained based on the calculated characteristic amount and the relational expression, and based on the obtained image amount. The keyword to be assigned to the object to be registered is estimated with reference to the keyword storage unit, and the estimated keyword and the registered And object elephant, is characterized in that stored in the object storage unit in association with each other.

According to the above configuration, the relational expression between the keyword image amount and the feature amount is derived with reference to the object to which the keyword has already been assigned, and the derived relational expression and the feature amount of the registered object are obtained. A keyword to be assigned to the registered object is estimated based on the keyword storage unit. Therefore, simply by preliminarily registering the keyword and the image amount of the keyword in the keyword storage unit in association with each other, the keyword is easily and automatically assigned to the registered object. Further, the processes of deriving the relational expression and estimating the keywords do not need to be performed only for the keywords registered in the keyword storage unit. A grant is made.

According to a second aspect of the present invention, in the keyword assigning method according to the first aspect of the present invention, the relational expression is characterized in that p feature amounts obtained from the n referenced objects are used as explanatory variables. , And is derived by multivariate analysis using the image quantities of the n keywords as objective variables.

According to the above configuration, the relational expression is easily derived by performing the multivariate analysis by the microcomputer processing.

A keyword automatic assigning device according to a third aspect of the present invention is a keyword automatic assigning device, comprising: a keyword storage unit for registering a keyword and an image amount obtained by quantifying the word feeling of the keyword on a predetermined scale in association with each other; Is referenced, and the relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned to the referenced object and the feature amount of the referenced object. A relational expression deriving unit that stores the derived relational expression in the relational expression storage unit, a characteristic amount calculating unit that calculates the characteristic amount of the registration target object, and an image based on the calculated characteristic amount and the relational expression. Then, the keyword to be assigned to the object to be registered is estimated by referring to the keyword storage based on the obtained image amount. And keyword estimator for, and characterized in that the said estimated keyword and the object to be registered, with the object registration unit that registers the object storage unit in association with each other.

According to the above configuration, the relational expression deriving unit derives the relational expression between the image amount and the feature amount by referring to the object to which the keyword is already assigned.
When a new object is registered, first, the feature amount of the registration target object is calculated by the feature amount calculation unit. Next, the keyword estimator
An image amount is obtained based on the calculated feature amount and the relational expression, and a keyword to be assigned is estimated by referring to the keyword storage based on the image amount. In this way, simply by preliminarily registering the keyword image amount and the keyword in the keyword storage unit in association with each other, the keyword is easily and automatically assigned to the registered object. Further, the processes of deriving the relational expression and estimating the keywords do not need to be performed only for the keywords registered in the keyword storage unit. A grant is made.

According to a fourth aspect of the present invention, in the keyword automatic assigning apparatus according to the third aspect of the present invention, the relational expression deriving unit is configured to calculate the p from the referenced n objects.
It is characterized in that the above relational expression is derived by multivariate analysis, using the feature amounts obtained for each of them as explanatory variables and the image amounts of n keywords as objective variables.

According to the above configuration, the relational expression is easily derived by performing the multivariate analysis by the microcomputer processing.

In the program storage medium according to the present invention, a keyword and an image amount obtained by quantifying the word feeling of the keyword on a predetermined scale are registered in the keyword storage unit in association with each other. A relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned to the referenced object and the feature amount of the referenced object by referring to the object to which the keyword is assigned. Steps to
Calculating the feature amount of the object to be registered; obtaining the image amount based on the calculated feature amount and the relational expression; and referring to the keyword storage unit based on the obtained image amount, Estimating a keyword to be assigned to the object to be registered;
An information processing program for storing the estimated keyword and the object to be registered in an object storage unit in association with each other is stored.

According to the above configuration, the relational expression between the keyword image amount and the feature amount is derived with reference to the object to which the keyword has already been assigned, and the derived relational expression and the feature amount of the registered object are obtained. A keyword to be assigned to the registered object can be estimated based on the keyword storage unit. Therefore, simply by preliminarily registering the keyword and the image amount of the keyword in the keyword storage unit in association with each other, the keyword is easily and automatically assigned to the registered object.

According to a sixth aspect of the present invention, in the program storage medium according to the fifth aspect of the present invention, the step of deriving the relational expression is characterized in that the step of deriving the relational expression is performed by p pieces of the n referenced objects. The method is characterized by including a step of performing multivariate analysis using the quantity as an explanatory variable and the image quantity of n keywords as a target variable.

According to the above configuration, the relational expression between the image quantity and the feature quantity can be easily derived by multivariate analysis.

According to a seventh aspect of the present invention, a keyword and an image amount obtained by quantifying the sensation of the keyword on a predetermined scale are registered in the keyword storage unit in association with each other, and the keyword is assigned. The relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned to the referenced object and the feature amount of the referenced object, and Calculating the feature amount of the object, obtaining an image amount based on the calculated feature amount and the relational expression, and referring to the keyword storage unit based on the obtained image amount, to register the object to be registered. The keyword to be assigned to the object is estimated, and the estimated keyword and the object to be registered are stored in association with each other. The information processing program stored in the section is stored.

According to the above configuration, the relational expression between the image amount and the feature amount of the keyword is derived by referring to the object to which the keyword has already been assigned, and the derived relational expression and the characteristic amount of the object to be registered are obtained. A keyword to be assigned to the registered object can be estimated based on the keyword storage unit. Therefore, simply by preliminarily registering the keyword and the image amount of the keyword in the keyword storage unit in association with each other, the keyword is easily and automatically assigned to the registered object.

According to an eighth aspect of the present invention, in the program storage medium according to the seventh aspect of the present invention, the relational expression is characterized in that the feature amount obtained from each of the n referenced objects is p as an explanatory variable. It is characterized in that it is derived by multivariate analysis using the image amount of n keywords as a target variable.

According to the above configuration, the relational expression between the image quantity and the feature quantity can be easily derived by multivariate analysis.

Here, the "object" is
This means electronic information (eg, text data, image data, and audio data) that is to be stored by an information processing device such as the present search device, or a folder in which these are collected.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments. FIG. 1 is a functional block diagram of a search device equipped with the automatic keyword assigning device of the present embodiment. In the present embodiment, image data is used as the "object".

From the input unit 1, image data to be stored in the image data storage unit 6, search conditions, various instructions, and the like are input. The keyword registration processing unit 2 registers a keyword, which is one of the search conditions input from the input unit 1, and an image amount described later in the keyword storage unit 3 in association with each other. The automatic keying function derivation processing unit 51, based on the keyword assigned to the image data of the reference image specified from the input unit 1, as described later in detail,
A function (hereinafter, referred to as an automatic keying function) for automatically performing keyword assignment (that is, for performing an automatic keying process) is derived and stored in the automatic keying function storage unit 52.

The image data registration processing unit 5 includes the input unit 1
Automatically assigns a keyword to the image data input from and registers the keyword in the image data storage unit 6. The image data registration processing unit 5 includes a feature amount calculation unit 53 and a keyword estimation unit 5
Four. The feature amount calculation unit 53 calculates a feature amount of the image based on the image data input from the input unit 1. Then, the keyword estimating unit 54 estimates a keyword of the image based on the calculated feature amount and the automatic keying function.

The image data search processing unit 7 includes the input unit 1
The image data registered in the image data storage unit 6 is searched for image data that satisfies the above search condition, using the keyword set from the search condition as a search condition. The candidate image is displayed on the display unit 9 based on the image data of the searched candidate image.
To be displayed. The display of the menu screen other than the candidate images is performed by a display processing unit (not shown) different from the image data display processing unit 8.

That is, in this embodiment, the image data registration processing section 5 constitutes the object registration section, and the automatic keying function derivation processing section 51 constitutes the relational expression deriving section, and stores the automatic keying function. The section 52 constitutes the relational expression storage section.

FIGS. 2 and 3 are diagrams showing a hardware configuration for realizing the functions of FIG. As shown in FIG. 2, the search apparatus mainly includes a CRT (cathode ray tube) 12, a keyboard 13, a mouse 14 and a control unit 11 having a CPU (Central Processing Unit) mounted thereon and controlling the operation of the entire search apparatus. , Floppy disk drive 15b, hard disk drive 16, printer 17, scanner 18, CD-R
It is composed of an OM drive 19b, a speaker 20, a microphone 21, and the like.

The CRT 12 displays the searched image,
Various information required for operations such as character information is displayed. The keyboard 13 and the mouse 14 are used for various input operations and instruction operations. The floppy disk 15a is mounted on the floppy disk drive 15b of the control device 11, and data is stored and reproduced. Hard disk drive 1
Reference numeral 6 stores image data and keywords. Printer 1
Reference numeral 7 outputs an image based on image data, a drawing created by image organization, and the like on a sheet. The scanner 18 reads an image on a sheet-shaped document and outputs image data. The CD-ROM 19a stores image data in a predetermined format.
The image data is reproduced by being attached to b. Speaker 20
Performs voice output, and the microphone 21 performs voice input. Here, the image data read by the scanner 18 and the CD-ROM drive 19b is stored in the hard disk device 16. The scanner 18, CD-R
The OM drive 19b, the speaker 20 and the microphone 21 may be built in the control device 11 and integrally formed.

FIG. 3 is a block diagram mainly showing the CPU mounted on the control device 11 of FIG. The CPU 25 is, for example, a product number i8 manufactured by Intel Corporation.
0486DX or the like is used. The CPU 25 includes, via a data bus 26, a ROM (read only memory) 28 in which a program for controlling the search device is stored, and a RAM (random access memory) in which various data and programs are stored. 29, CR for images or characters
A display control circuit 30 for displaying at T12, a keyboard control circuit 31 for controlling transfer of input from the keyboard 13, a mouse control circuit 32 for controlling transfer of input from the mouse 14,
A floppy disk drive control circuit 33 for controlling the floppy disk drive 15b; a hard disk control circuit 34 for controlling the hard disk device 16; a printer control circuit 35 for controlling the output operation of the printer 17; a scanner control circuit for controlling the image input operation of the scanner 18 3
6. CD-RO for controlling the CD-ROM drive 19b
M drive control circuit 37, speaker control unit 38 for controlling the audio output of speaker 20, and microphone 2
Microphone control circuit 3 for controlling voice input from 1
9 is connected.

The CPU 25 is connected to a clock 27 for generating a reference clock required for operating the search device, and further to an extension for connecting various extension boards via a data bus 26. The slot 40 is connected. Here, the SCSI is
A floppy disk drive 15b and a hard disk drive 1 are connected via the SCSI board.
6. The scanner 18, the CD-ROM drive 19b, etc. may be connected.

That is, in the present embodiment, the input unit 1 is connected to the keyboard 13, mouse 14, hard disk drive 16, scanner 18, and CD-ROM drive 1.
9b, the image data storage unit 6 is composed of the floppy disk 15a and the hard disk device 16, and the keyword storage unit 3 and the automatic keying function storage unit 52 are R
The display unit 9 is composed of a CRT 12, the keyword registration processing unit 2, the image data registration processing unit 5, the image data search processing unit 7, and the automatic keying function derivation processing unit 51 are composed of C29.
It consists of PU25.

In the present embodiment, as described above,
Although the image data storage unit 6 is constituted by the floppy disk 15a and the hard disk device 16, other storage media such as a magneto-optical disk device may be used. Although the input section of the input section 1 for inputting image data is constituted by the hard disk drive 16, the scanner 18, and the CD-ROM drive 19b, other input apparatuses such as a still video camera and a digital camera may be used. Further, another output device such as a digital copying machine may be used instead of the printer 17.

In the present retrieval apparatus, programs for image data registration processing, keyword registration processing, automatic keying function derivation processing, and the like are stored in the ROM 28. However, part or all of the program may be stored in an external storage medium such as the floppy disk 15a or the hard disk device 16, and the program may be read into the RAM 29 as needed.

FIG. 4 is a flowchart of a basic processing operation executed under the control of the CPU 25 based on the program stored in the ROM 28. Hereinafter, the basic processing operation of the search device will be described with reference to FIGS.

When the power of the search apparatus is turned on and the above program is started, the basic processing operation starts. In step S1, initialization of flags and registers required in the following processing, and initial setting processing such as initial screen display on the display unit 9 are performed. If it is determined in step S2 that the job menu "search" has been selected from the initial screen displayed on the display unit 9, the process proceeds to step S3, and if not, the process proceeds to step S4.

In step S3, a search process for searching desired image data from the image data storage unit 6 and displaying it on the display unit 9 is executed according to the selected job menu "search". After that, the process proceeds to step S13. Note that the search processing is performed by the image data search processing unit 7. If it is determined in step S4 that the job menu "data registration" has been selected from the initial screen displayed on the display unit 9, the process proceeds to step S5, and if not, the process proceeds to step S6. In step S5, image data input from the input unit 1 is fetched in accordance with the selected job menu "data registration", a keyword is assigned to the image data, and the image data is registered in the image data storage unit 6. Is executed. After that, the process proceeds to step S13. Note that the image data registration processing is performed by the image data registration processing unit 5.

If it is determined in step S6 that the job menu "keyword registration" has been selected from the initial screen displayed on the display unit 9, the process proceeds to step S7, and if not, the process proceeds to step S8. In step S7, according to the selected job menu "keyword registration", a keyword registration process for registering the keyword input from the input unit 1 together with the image amount in the keyword storage unit 3 is executed. After that, the process proceeds to step S13. Note that the keyword registration processing is performed by the keyword registration processing unit 2.

In step S8, if it is determined that the job menu "automatic keying function derivation" is selected from the initial screen displayed on the display section 9, the process proceeds to step S9.
Otherwise, go to step S10. In step S9,
An automatic keying function deriving process for deriving the automatic keying function is executed according to the selected job menu “automatic keying function derivation”. After that, the process proceeds to step S13. Note that the automatic keying function derivation processing is performed by the automatic keying function derivation processing unit 51.

If it is determined in step S10 that the job menu "other menu" has been selected from the initial screen displayed on the display section 9, the process proceeds to step S11, and if not, the process proceeds to step S12. In step S11, other menu processing such as printout of the search image is executed according to the selected job menu "other menu". After that, the process proceeds to step S13.

In step S12, it is determined whether or not "end" has been selected from the initial screen displayed on the display unit 9. As a result, if "end" is selected, the basic processing operation is ended. Otherwise, the process proceeds to step S13. In step S13, other processing is executed. After that, the process returns to step S2 and shifts to the next menu process. Then, in the above step S12, "end"
Is selected, the basic processing operation ends.

Hereinafter, the "keyword registration processing", "automatic keying function derivation processing", and "image data registration processing" will be described in detail. It should be noted that “search processing” and “other menu processing” are not directly related to the present invention, and thus detailed description thereof is omitted.

FIG. 5 is a flowchart of a keyword registration processing subroutine executed by the keyword registration processing unit 2 in step S7 in the basic processing operation shown in FIG. Hereinafter, the keyword registration processing operation will be described in detail with reference to FIG. When it is determined in step S6 of the main routine of the basic processing operation shown in FIG. 4 that the job menu "keyword registration" is selected, a keyword registration processing subroutine starts.

In step S21, the keyword input from the input unit 1 is fetched. In step S22, the image amount input from the input unit 1 is fetched. Here, the image amount is a value obtained by digitizing the word feeling of the keyword using a certain scale. For example, FIG. 6 shows a result of digitizing the word feeling of 16 keywords on a scale of “hard: 0 ← → soft: 15”. The image amount of the keyword “romantic” is “12”, and the keyword is “deep”. Is numerically expressed as “4”. In step S23, the input keyword is registered in the keyword storage unit 3 in association with the input image amount. In step S24, it is determined whether or not the keyword registration process is to be ended by a dialog with the user. As a result, if not finished, the process returns to the step S21 and shifts to registration of the next keyword. On the other hand, if it is finished, the keyword registration processing subroutine ends, and the process returns to the main routine of the basic processing operation shown in FIG.

In this manner, by defining an image amount for a keyword registered in the keyword storage unit 3, the feature amount of an image and the image amount of a keyword are associated with each other as will be described in detail later. A keyword appropriate for the registered image can be estimated from the feature amount. Note that the image amounts shown in FIG. 6 are just examples, and similarly, the image amounts of various keywords are set by various scales.

FIG. 7 is a flowchart of an automatic keying function deriving process subroutine executed by the automatic keying function deriving unit 51 in step S9 in the basic processing operation shown in FIG. Hereinafter, the automatic keying function derivation processing operation will be described in detail with reference to FIGS. When it is determined in step S8 of the main routine of the basic processing operation shown in FIG. 4 that the job menu "automatic keying function derivation" is selected, an automatic keying function deriving process subroutine starts.

In step S31, an image to be referred to (hereinafter, referred to as a reference image) is designated from among the image data registered in the image data storage section 6 through dialogue with the user. The method of designating the reference image is not particularly limited. For example, a list of image data names of all the image data registered in the image data storage unit 6 is displayed on the display unit 9 and the input unit 1 You can specify it. In step S32, the keyword assigned to the designated reference image is read out with reference to the image data storage unit 6. Then, referring to the keyword storage unit 3, the image amount of the read keyword is obtained.

In step S33, the feature amount of the designated reference image is obtained. In this case, various characteristic amounts such as characteristic amounts relating to colors such as lightness, hue, and saturation, and characteristic amounts relating to contour characteristics and shape characteristics such as a linear content ratio and a circular content ratio are considered. However, there is no particular limitation.

In step S34, for each keyword read in step S32, the image amount obtained in step S32 and the keyword
An automatic keying function, which is a relational expression with the feature quantity obtained in 33, is derived by multivariate analysis. The multivariate analysis is not particularly limited, and for example, a multiple regression analysis or the like can be considered. However, this automatic keying function derivation is
This is performed for each image amount of the same scale. In step S35,
The derived automatic keying function is stored in the automatic keying function storage unit 52. After that, the automatic keying function derivation processing subroutine ends, and the process returns to the main routine of the basic processing operation shown in FIG.

Hereinafter, a specific example in which an automatic keying function is derived by the multiple regression analysis will be described. FIG. 8 shows the image amount obtained from the n reference images in step S32 of the automatic keying function derivation processing subroutine and p (= 5) image amounts obtained from the same n reference images in step S33. Are shown. For example, 1
The keyword “powerful” is assigned to the image data of the second reference image, and the image amount of the keyword “powerful” is “2.00 (see FIG. 6)”. The five feature amounts of the reference image obtained based on the image data are “8.00”, “5.00”, “10.00”,
"4.00" and "20.00".

Then, the n image quantities obtained in this manner are set as objective variables y, and the n first feature quantities are set as explanatory variables x
_1, and the n second feature quantity of pieces as explanatory variable x _2, the third feature amount of n as explanatory variables x _3, the fourth feature value of n is an explanatory variable x _4, the n fifth performing a multiple regression analysis feature quantities as explanatory variable x _5, and automatic key pricing function seeking multiple regression equation _{y = a 0 + a 1 x} 1 + a 2 x 2 + a 3 x 3 + a 4 x 4 + a 5 x 5 It is. Note that a ₀ , a
_{1, a 2, a 3,} a 4, a 5 are regression coefficients. Here, if the correlation coefficient γ is bad (for example, γ <0.75), a warning to that effect is displayed on the display unit 9 to urge the change of the reference image or to refrain from deriving the automatic keying function. It may be displayed.

FIG. 9 is a flowchart of an image data registration processing subroutine executed by the image data registration processing section 5 having the feature amount calculating section 53 and the keyword estimating section 54 in step S5 in the basic processing operation shown in FIG. is there. Hereinafter, the image data registration processing operation will be described in detail with reference to FIG. When it is determined in step S4 of the main routine of the basic processing operation shown in FIG. 4 that the job menu "data registration" has been selected, the image data registration processing subroutine starts.

In step S41, the input unit 1 is input by the user.
It is determined whether or not "automatic keying mode" has been designated. As a result, if specified, step S42
On the other hand, if not specified, the process proceeds to step S43. In step S42, the automatic keying mode flag is set. After that, the process proceeds to step S44. In step S43, the automatic keying mode flag is reset.

In step S44, it is determined whether or not "new input" for newly registering image data has been designated from the input unit 1 by the user. As a result, if specified, the process proceeds to step S45, and if not specified, the process proceeds to step S46. In step S45, an image data capturing process described in detail later is executed. After that, step S
Go to 47. In step S46, existing image data is selected and read.

In step S47, it is determined whether or not the automatic keying mode flag has been set. As a result, if set, the process proceeds to step S48, and if not set, the process proceeds to step S51. In step S48,
The new image data captured in step S45 or the above-described step S45
A feature amount is obtained from the existing image data read in 46. The above-mentioned feature is the same as the feature obtained in step S33 in the automatic keying function deriving process subroutine. In step S49, the keyword estimating unit 54 causes the automatic keying function storage unit 52
Is read, and the feature amount obtained in step S48 is substituted for the read automatic keying function to obtain the image amount. Then, an appropriate keyword is estimated based on the obtained image amount. In the above estimation,
This is performed by selecting a keyword in which the image amount closest to the image amount (image amount on the same scale) obtained from the automatic keying function is defined. In step S50, the estimated keyword is set as one of the additional information of the image data to be registered, and is stored in, for example, a register (not shown).

In step S51, it is determined whether or not the manual input of the additional information has been specified by the user from the input unit 1. As a result, if specified, step S
The process proceeds to 52, and if not specified, the process proceeds to step S53.
In step S52, an additional information input process, which will be described in detail later, is executed by a dialog with the user. In step S53, the additional information held in the register is read, and the image data to be registered is stored in the image data storage unit 6 together with the read additional information. Thereafter, the image data registration processing subroutine ends, and the process returns to the main routine shown in FIG.

In this way, by simply designating the "automatic keying mode" from the input unit 1, a keyword suitable for the feature amount of the image to be registered is automatically estimated and added to the image data. Because In the keyword estimation in step S49, if the difference between the estimated image amount of the keyword and the image amount obtained from the automatic keying function is equal to or greater than a predetermined value, the keyword to be assigned is It may be determined that there is no such information, and the fact is displayed on the display unit 9, and the step S50 may be skipped.

FIG. 10 is a flowchart of the image data fetch processing subroutine executed in step S45 in the image data registration processing subroutine shown in FIG. Hereinafter, the image data capturing processing operation will be described in detail with reference to FIGS. Step S in the image data registration processing subroutine shown in FIG.
If it is determined at 44 that "new input" is designated, the image data fetching subroutine starts.

In step S61, new image data is fetched from the input unit 1 (an image input device such as the scanner 18 or the CD-ROM drive 19b or an image data storage medium such as the hard disk device 16). Step S6
In step 2, a storage location such as a working memory for storing the captured image data is designated. In step S63, the fetched image data is stored in the work memory with the image data name set by the user from the input unit 1. After that, the image data import processing subroutine ends, and the process returns to the image data registration processing subroutine shown in FIG.

FIG. 11 is a flowchart of the additional information input processing subroutine performed in step S52 of the image data registration processing subroutine shown in FIG. Hereinafter, the additional information input processing operation will be described in detail with reference to FIGS. If it is determined in step S51 of the image data registration processing subroutine shown in FIG. 9 that "manual input of additional information" has been designated, the additional information input processing subroutine starts.

In step S71, the additional information input menu is displayed on the display unit 9. If it is determined in step S72 that the additional information input menu "keyword addition" has been selected, the process proceeds to step S73; otherwise, the process proceeds to step S74. In step S73, a keyword addition process, which will be described in detail later, is executed according to the selected additional information input menu “keyword addition”. After that, the process proceeds to step S76. If it is determined in step S74 that the additional information input menu "additional information" has been selected, the process proceeds to step S75; otherwise, the process proceeds to step S75.
Go to 76. In step S75, the additional information input from the input unit 1 is fetched in accordance with the selected additional information input menu "addition of other information", and held in the register as one of the additional information of the image data. In step S76, it is determined whether or not to terminate the additional information input process through a dialogue with the user. As a result, if the processing is completed, the additional information input processing subroutine is completed and the process returns to the image data registration processing subroutine shown in FIG. On the other hand, if not finished, the process returns to step S72 and shifts to the next menu process.

In this way, keywords other than the keyword estimated in step S49 in the image data registration processing subroutine, and additional information other than keywords such as a data creator name, a creation date, a registration date, a size, and a comment are stored in the user data. Manually entered by

FIG. 12 is a flowchart of the keyword addition processing subroutine executed in step S73 of the additional information input processing subroutine shown in FIG. Hereinafter, the keyword addition processing operation will be described in detail with reference to FIGS. If it is determined in step S72 of the additional information input processing subroutine shown in FIG. 11 that the additional information input menu "keyword addition" has been selected, the keyword addition processing subroutine starts.

In step S81, the keyword storage unit 3
Are listed on the display unit 9. In step S82, a specific keyword is selected from the displayed keywords by dialogue with the user. In step S83, the selected keyword is set as one of the additional information, and is stored in the register. In step S84, through dialogue with the user,
It is determined whether to end the keyword addition processing. As a result, if not ended, the process returns to the step S82 and shifts to the selection of the next keyword. On the other hand, if completed, the keyword addition processing subroutine ends, and the process returns to the additional information input processing subroutine shown in FIG.

As described above, in the present embodiment, the automatic keying function derivation processing unit 51 uses the image data storage unit 6
Then, the automatic keying function is derived from the image amount of the keyword assigned to the reference image and the obtained characteristic amount. The automatic keying function thus derived is stored in the automatic keying function storage unit 52. On the other hand, the image data registration processing unit 5 has a feature amount calculation unit 53 and a keyword estimation unit 54. When the image data to be registered is registered in the image data storage unit 6, the feature amount of the registration target image is calculated by the feature amount calculation unit 53. Then, the keyword estimating unit 54 obtains an image amount by substituting the calculated feature amount into the automatic keying function, estimates a keyword of the registration target image based on the obtained image amount, and adds the keyword to the image data. You do it.

As described above, according to the present embodiment, the derivation of the automatic keying function and the derivation of the derived automatic keying function are performed simply by setting the image amount of the keyword to be registered at the time of keyword registration. The keyword assignment based on the function is automatically performed. Therefore, unlike the conventional keyword assignment method disclosed in Japanese Patent Application Laid-Open No. 6-295318, it is not necessary to perform a troublesome operation of previously setting optimum fitness calculation information and a threshold value for each keyword candidate. It is.

Further, when estimating keywords to be assigned, it is necessary to perform a calculation for determining whether all keywords registered in the keyword storage 3 are appropriate as keywords to be assigned to registered image data. In addition, even if a large number of keywords are registered in the keyword storage unit 3, keyword assignment can be performed in a short time. Also,
The automatic keying function used at the time of keyword assignment is derived based on the feature amount of the reference image and the image amount of the keyword. Therefore, by designating a reference image having the same field of use or application as the reference image, it is possible to assign an appropriate keyword according to the field of use or application.

In the above embodiment, a case where a keyword is added to image data has been described as an example. However, the present invention is not limited to this, and audio data,
The present invention is also applicable to a case where a keyword is added to multimedia data in which audio data is combined with image data. In short, any data can be used as long as it is possible to extract a characteristic amount representing a characteristic of the content of the data.

In the above embodiment, when one keyword is assigned to the reference image, an automatic keying function is derived by applying a multiple regression analysis technique. Even when a keyword is assigned, an automatic keying function can be derived by applying another multivariate analysis technique such as canonical correlation analysis. Alternatively, when a plurality of keywords are assigned to the reference image, the average value of the image amounts of the plurality of keywords can be adopted as the image amount for the reference image.

[0071]

As is apparent from the above description, in the keyword assigning method according to the first aspect of the present invention, the keyword and the image amount of the keyword are registered in the keyword storage unit in association with each other, and the feature amount of the reference object is registered. The relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned and the feature amount of the registered object is calculated. The image amount obtained from the calculated feature amount and the relational expression is calculated. The keyword to be assigned is estimated by referring to the keyword storage unit based on the keyword. Therefore, simply registering the keyword and the image amount of the keyword in advance, the keyword is easily and automatically stored in the registered object. Can be granted. Therefore, unlike the conventional keyword assignment method, it is not necessary to perform a troublesome operation of previously setting the optimum matching degree calculation information and the threshold value for each keyword candidate.

Further, the processes of deriving the relational expression and estimating the keywords do not need to be performed only for the keywords registered in the keyword storage unit. The application can be performed in a short time. Further, since the relational expression is derived based on the feature amount of the reference object and the image amount of the assigned keyword, by specifying an object having the same field of use and application as the reference object, Appropriate keywords can be assigned accordingly.

Further, in the above-mentioned relational expression in the keyword assigning method of the invention according to claim 2, the feature quantity obtained from each of n reference objects by p is used as an explanatory variable, and the image quantity of n keywords is used as an objective variable. Since the relational expression is derived by multivariate analysis, the above relational expression can be easily derived by microcomputer processing or the like.

The keyword automatic assigning device according to the third aspect of the present invention registers the keyword and the image amount of the keyword in the keyword storage, and determines the feature amount of the reference object and the image amount of the assigned keyword. The relational expression deriving unit derives the relational expression between the image amount and the feature amount, calculates the characteristic amount of the registered object by the feature amount calculating unit, and estimates the keyword based on the calculated characteristic amount and the relational expression. Since the keyword to be assigned is estimated by referring to the keyword storage unit, the image amount of the keyword and the keyword are registered in advance in the keyword storage unit in association with each other in a simple manner. In addition, a keyword can be assigned to a registered object. Therefore, unlike the conventional keyword assignment method, it is not necessary to perform a troublesome operation of previously setting the optimum fitness calculation information and the threshold value for each keyword candidate.

Further, the processes of deriving the relational expressions and estimating the keywords need not be performed for the keywords registered in the keyword storage unit. The application can be performed in a short time. Further, since the relational expression is derived based on the feature amount of the reference object and the image amount of the assigned keyword, by specifying an object having the same field of use and application as the reference object, Appropriate keywords can be assigned accordingly.

Further, the relational expression deriving unit in the automatic keyword assigning apparatus according to the fourth aspect of the invention uses the feature quantity obtained from the n referenced objects as p as explanatory variables, Since the relational expression is derived by multivariate analysis using the image amount of the target as a target variable, the relational expression can be easily derived by microcomputer processing.

According to a fifth aspect of the present invention, there is provided the program storage medium, wherein the keyword and the image amount of the keyword are registered in the keyword storage unit in association with each other, and the characteristic amount of the reference object and the assigned keyword are stored. A step of deriving a relational expression between the image amount and the feature amount from the image amount; a step of calculating a feature amount of the registered object; and a step of calculating the characteristic amount of the registered object based on the image amount obtained from the calculated feature amount and the relational expression. Since an information processing program having a step of estimating a keyword to be given with reference to the keyword storage unit is stored,
By simply registering the keyword and the image amount of the keyword in advance, the keyword can be easily and automatically assigned to the registered object.

Further, in the step of deriving the relational expression in the program storage medium according to the invention of claim 6, the step of deriving the relational expression from the n referenced objects as p explanatory variables, Since the method includes a step of performing multivariable analysis using the image amount of the keyword as a target variable, the relational expression between the image amount and the feature amount can be easily derived by multivariable analysis.

Further, in the program storage medium according to the present invention, the keyword and the image amount of the keyword are registered in the keyword storage unit in association with each other, and the characteristic amount of the reference object and the assigned keyword are stored. A relational expression between the image amount and the characteristic amount is derived from the image amount, a characteristic amount of the registered object is calculated, and the keyword storage unit is calculated based on the calculated characteristic amount and the image amount obtained from the relational expression. Since it stores an information processing program for estimating a keyword to be given by referring to
By simply registering the keyword and the image amount of the keyword in advance, the keyword can be easily and automatically assigned to the registered object.

The relational expression in the program storage medium according to the eighth aspect of the present invention is such that the feature quantity obtained from each of the n reference objects by p is used as an explanatory variable, and the image quantity of the n keywords is used as an objective variable. Therefore, the relational expression between the image quantity and the feature quantity can be easily derived by multivariable analysis.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing an embodiment of a search device equipped with an automatic keyword assigning device of the present invention.

FIG. 2 is a diagram showing an apparatus configuration for realizing the functions shown in FIG.

FIG. 3 is a hardware CP realizing the functions shown in FIG. 1;
It is a block diagram centering on U.

FIG. 4 is a flowchart of a basic processing operation executed under control by a CPU in FIG. 3;

FIG. 5 is a flowchart of a keyword registration processing subroutine executed by a keyword registration processing unit in FIG. 1;

FIG. 6 is an explanatory diagram of an image amount.

FIG. 7 is a flowchart of an automatic keying function derivation processing subroutine executed by the automatic keying function derivation processing unit in FIG. 1;

8 is a diagram illustrating an example of an image amount and a feature amount obtained in the automatic keying function derivation processing of FIG. 7;

FIG. 9 is a flowchart of an image data registration processing subroutine executed by the image data registration processing unit in FIG. 1;

FIG. 10 is a flowchart of an image data capturing processing subroutine executed during the image data registration processing of FIG. 9;

FIG. 11 is a flowchart of an additional information input processing subroutine executed during the image data registration processing of FIG. 9;

FIG. 12 is a flowchart of a keyword addition processing subroutine executed during the additional information input processing of FIG. 11;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Input part, 2 ... Keyword registration processing part, 3 ... Keyword storage part, 5 ... Image data registration processing part, 6 ... Image data storage part,
7 image data search processing unit 9 display unit
51 ... automatic keying function derivation processing unit, 52
... Automatic keying function storage unit 53 ... Feature amount calculation unit
54 ... Keyword estimation unit.

Claims (8)

[Claims] 1. A keyword and an image amount obtained by quantifying the word feeling of the keyword on a predetermined scale are registered in a keyword storage unit in association with each other, and an object to which the keyword is assigned is referred to. A relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned to the referenced object and the feature amount of the referenced object, and the feature amount of the registration target object is calculated. An image amount is obtained based on the calculated feature amount and the relational expression, and a keyword to be assigned to the registration target object is estimated by referring to the keyword storage unit based on the obtained image amount. The estimated keyword and the object to be registered are stored in the object storage unit in association with each other. Keyword assignment method according to claim. 2. The keyword assigning method according to claim 1, wherein the relational expression is such that a feature amount obtained from each of the referenced n objects by p is used as an explanatory variable, and an image amount of the n keywords is calculated. Is derived by multivariate analysis using as a target variable. 3. A keyword storage unit for registering a keyword and an image amount obtained by quantifying the word feeling of the keyword on a predetermined scale in association with each other, and an object to which the keyword is assigned is referred to. A relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned to the object and the feature amount of the referenced object,
A relational expression deriving unit that stores the derived relational expression in a relational expression storage unit, a feature amount calculating unit that calculates a feature amount of an object to be registered, and based on the calculated feature amount and the relational expression. A keyword estimating unit for obtaining an image amount, referring to the keyword storage unit based on the obtained image amount, and estimating a keyword to be assigned to the object to be registered; and An automatic keyword assigning device, comprising: an object registering unit for registering an object with an object storing unit in association with each other. 4. The automatic keyword assigning device according to claim 3, wherein the relational expression deriving unit sets n feature values obtained from the n referenced objects as explanatory variables, and sets the n keyword keywords. An automatic keyword assigning apparatus characterized in that the above relational expression is derived by multivariate analysis using the image quantity of (1) as a target variable. 5. A step of registering a keyword and an image amount obtained by quantifying the feeling of the keyword on a predetermined scale in a keyword storage unit, and referring to an object to which the keyword is assigned, Deriving a relational expression between the image amount and the feature amount from the image amount of the keyword assigned to the object to be added and the feature amount of the referenced object, and calculating the feature amount of the registration target object Calculating an image amount based on the calculated feature amount and the relational expression, and referring to the keyword storage unit based on the obtained image amount, a keyword to be assigned to the registration target object. And estimating the estimated keyword and the object to be registered with each other. Program storage medium characterized by storing an information processing program according to the step of storing in the object storage portion in association response. 6. The program storage medium according to claim 5, wherein the step of deriving the relational expression includes, as explanatory variables, p feature amounts obtained from the n referenced objects, and n A program storage medium including a step of performing a multivariate analysis using an image amount of a keyword as an objective variable. 7. A keyword and an image amount obtained by quantifying the vocabulary of the keyword on a predetermined scale are registered in the keyword storage unit in association with each other, and the object to which the keyword is assigned is referred to. A relational expression between the image amount and the feature amount is derived from the image amount of the keyword assigned to the referenced object and the feature amount of the referenced object, and the feature amount of the registration target object is calculated. An image amount is obtained based on the calculated feature amount and the relational expression, and a keyword to be assigned to the registration target object is estimated by referring to the keyword storage unit based on the obtained image amount. Information that stores the estimated keyword and the object to be registered in the object storage unit in association with each other. Program storage medium characterized by storing a processing program. 8. The program storage medium according to claim 7, wherein the relational expression is such that the feature amount obtained from each of the referenced n objects is p as an explanatory variable, and the image amount of the n keywords is A program storage medium characterized by being derived by a multivariate analysis, using as a target variable.