JP2004287829A - Image data classifying device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for classifying image data more efficiently while reducing a user's load. <P>SOLUTION: This image data classifying device 1 has a storage device 11 for storing image data and the feature amount of the image data, an operation input part 14a, and a display part 14b. When an instruction is given through the operation input part 14a to the effect that the image data stored in the storage device 11 is to be included in a certain group, whether it is proper to include the image data in the group is determined by comparing the feature amount of the image data with the feature amount corresponding to the group. In the case it is proper, the image data is included in the group and stored in the storage device 11, but in the case it is not proper, a message to the effect that it is not proper is outputted from the display part 14b. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for classifying image data.
[0002]
[Prior art]
In recent years, it has become possible to read a document written on paper with a scanner and digitize it and manage it as image data with a computer device, or to store and manage image data representing an image captured by a digital camera with a computer device. Is being done. As a method of managing image data, there is a method of classifying and managing each type in the same manner as a method of managing a document written on paper. A method of classifying image data for each type is as follows, for example. The user grasps in advance the type of an image represented by image data such as a landscape, a person, and a document, and stores the image data in a folder corresponding to the type. When the user performs such a classification operation for each image data, a plurality of image data are classified for each type.
However, when there is a large amount of image data to be classified, it is quite troublesome for the user to perform such classification work for each image data. Further, there is a high possibility that the user makes a mistake in the classification.
Techniques that can solve such a problem are described in Patent Literature 1 and Patent Literature 2. Patent Documents 1 and 2 disclose devices for automatically classifying and storing image data. The device disclosed in Patent Literature 1 analyzes the layout of image data and classifies the image data based on the analysis result. The layout analysis result includes information such as the number of blocks of image data, the number of text attributes, the degree of overlap with other attribute blocks, the number of table attributes and their sizes, and the number of picture attributes and their sizes. In the device disclosed in Patent Literature 2, when image data is input, a feature amount representing a feature of an image represented by the image data is calculated, and the image data and the device are already stored in the device using the feature amount. The relevance with the stored image data is evaluated by a clustering technique, and the image data is classified and stored for each group of highly relevant image data. The characteristics of the image include, for example, characteristics such as color tone, aspect ratio, luminance and color distribution of the entire image.
[0003]
[Patent Document 1]
JP 2001-101213 A [Patent Document 2]
JP 2001-256244 A
[Problems to be solved by the invention]
However, since the image data is automatically classified in any of the apparatuses, the classification result does not always match the user's intention. Therefore, the user has to check how it was classified. Therefore, the user needs to display the classification destination of the image data on a computer device, for example, and check the classification result by looking at the display. If the classification destination is contrary to the user's intention, the user must eventually perform the classification work.
[0005]
The present invention has been made in view of the circumstances described above, and an object of the present invention is to provide a technique for reducing the load on a user and more efficiently classifying image data.
[0006]
[Means for Solving the Problems]
The present invention provides a storage device that stores image data and a feature amount of the image data, an operation input unit, an output unit, and an instruction that image data stored in the storage device should be included in a group. When given through the section, the feature amount of the image data is compared with the feature amount associated with the group to determine whether it is appropriate to include the image data in the group. A classification registration unit that stores the image data in the storage device in a case where the image data is appropriate, and outputs a message indicating that the image data is invalid if the image data is invalid. To provide an image data classifying device.
[0007]
According to the present invention, when an instruction to include image data stored in the storage device in a certain group is given through the operation input unit, the feature amount of the image data is associated with the group. It is determined whether or not it is appropriate to include the image data in the group by comparing the obtained feature amount, and if so, the image data is included in the group and stored in the storage device. If not, a message indicating that the message is not valid is output from the output unit.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In all the drawings for describing the embodiments, components having the same function are denoted by the same reference numerals, and repeated description thereof will be omitted.
(1. Configuration)
<Overall configuration>
FIG. 1 is a block diagram showing a hardware configuration of an image data classification device 1 according to the present invention.
The user interface 14 includes an operation input unit 14a and a display unit 14b. The operation input unit 14a includes a plurality of keys for inputting numbers, characters, commands, and the like, and outputs an operation signal corresponding to the operation of these keys. The display unit 14b has a display panel (not shown) and a drive circuit for driving the display panel. Note that an audio output unit that outputs information in audio may be provided instead of the display unit 14b.
[0009]
The network interface 15 connects the control unit 10 to another device via a network, and controls data transfer between the two.
The image input device 12 has a scanner that reads an image from a document and outputs an analog image signal, and an A / D conversion circuit that converts an analog image signal output from the scanner into a digital image signal and outputs the digital image signal. In a preferred embodiment, the image input device 12 has an ADF (Automatic Document Feeder) that takes out documents one by one from a bundle of documents set in a document receiver and outputs the documents to a scanner.
[0010]
The storage device 11 includes a large-capacity storage device such as a hard disk drive or a DVD-RAM (Digital Video Disc-Random Access Memory) drive. The control unit 10 is connected to each unit of the apparatus described above via the bus 16. The control unit 10 is a unit that controls the entire image data classification device 1 and includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) (all not shown). . Here, the CPU controls each unit of the image data classification device 1 by executing a program stored in the ROM.
[0011]
FIG. 2 is a block diagram illustrating a functional configuration according to the control unit 10. The control unit 10 includes a feature amount extraction unit 10b, a similarity evaluation unit 10c, and a classification registration unit 10a. These entities are software modules executed by the CPU of the control unit 10.
The feature amount extraction unit 10b performs a feature amount extraction process on the image data, and calculates a feature amount of an image represented by the image data. The characteristics of the image include, for example, image density, aspect ratio, luminance and color distribution, edge distribution, and flat area distribution.
A procedure for calculating such a feature amount is, for example, as follows. First, the feature amount extraction unit 10b divides an image area represented by image data into X in the vertical direction and Y in the horizontal direction. That is, an image represented by one piece of image data is divided into X * Y areas. Next, a feature value is calculated for each of the regions by a predetermined calculation method. Although various types of feature amounts are conceivable, in this embodiment, R (red), G (green), and B (blue) components of the color of the region are obtained. Then, the R, G, and B components are converted into three quantities representing color components in an L * a * b space (characteristic color space). These three quantities are set as feature quantities corresponding to one region. As a result, (X * Y * 3) feature amounts are calculated from the image data. FIG. 5 shows a case where X = 6 and Y = 4 as an example, and shows that a total of 72 feature amounts have been calculated from this image data. These 72 feature amounts constitute a 72-dimensional vector representing the features of the image represented by the image data.
[0012]
The similarity evaluation unit 10c compares a feature amount of a certain image data with a feature amount obtained from another image data or a group of image data, and resembles each corresponding image to the extent that the images of the same type are recognized as the same type of image. It is a module that determines whether or not it is performing. The feature amount to be compared is provided by the control unit 10 to the similarity evaluation unit 10c.
The judgment performed by the similarity evaluation unit 10c is as follows. First, n feature amounts obtained from each image data form an n-dimensional vector representing a feature of a certain image. Thus, when n pieces of feature amounts corresponding to two pieces of image data to be compared are given, the similarity evaluation unit 10c calculates an n-dimensional vector having one n pieces of feature amounts as components and the other n pieces of n pieces of feature amounts. The distance in the n-dimensional space from the n-dimensional vector having the feature amount of the component as a component is obtained. Then, based on this distance, it is determined whether or not the images represented by the two image data are similar to the extent that they are recognized as belonging to the same type. FIG. 6 and FIG. 7 are diagrams for explaining the processing by the similarity evaluation unit 10c. In these figures, for convenience, three-dimensional vectors α and β are shown as two vectors each having a feature amount corresponding to two pieces of image data as components. FIG. 6 shows a state in which the Euclidean distance between vector α and vector β is short, and FIG. 7 shows a state in which the Euclidean distance between vector α and vector β is long. The similarity evaluation unit 10c regards the Euclidean distance between these two vectors α and β as the similarity between the two images to be compared, and the similarity is equal to or more than a predetermined value (the Euclidean distance is equal to or less than a predetermined length). In this case, it is determined that the two images belong to the same type, and when the similarity is less than a predetermined value (the Euclidean distance is longer than a predetermined length) as shown in FIG. It does not belong to.
Note that the distance to be calculated is not limited to the Euclidean distance, and may be a distance by another calculation method such as a Mahalanobis distance.
[0013]
The classification registration unit 10c uses the feature amount extraction unit 10b and the similarity evaluation unit 10c described above to assist the user in classifying the image data stored in the storage device 11 into appropriate folders. I do. This processing will be clarified in the description of the operation of the present embodiment in order to avoid redundant description.
[0014]
(2. Operation)
FIG. 8 is a flowchart showing the operation of the image data classification device 1 when the user classifies the image data stored in the storage device 11.
The operation described below is performed in the following situation.
First, various types of image data such as image data input via the image input device 12 or image data received via the network interface 15 are stored in a certain folder and stored in the storage device 11.
Then, the user gives an instruction from the operation input unit 14a, and displays the contents of this folder on the display unit 14b. FIG. 3 illustrates the display contents of the display unit 14b at this time. As shown in this figure, the display unit 14b is created for storing an unclassified folder 11a for storing unclassified image data and image data of a specific type such as a landscape folder, a person folder, and a still life folder. The displayed various classified folders 11b and icons representing image data not stored in any of these folders are displayed.
In this state, when the user performs a predetermined operation using the operation input unit 14a, in the control unit 10, the classification registration unit 10a starts operating.
Then, when the user performs an operation of inserting an icon of certain image data into an icon of a certain classification folder 11a with a mouse in the operation input unit 14a, the routine shown in FIG. 8 is performed by the classification registration unit 10a.
[0015]
First, the classification registration unit 10a reads target image data represented by the icon designated by the mouse from the storage device 11 (step S10), and determines whether or not a feature amount corresponding to the target image data is stored in the storage device 11. Is determined (step S11). If the determination result is negative, the classification registration unit 10a delivers the target image data to the feature amount extraction unit 10b, and calculates the feature amount (Step S12). Then, the calculated feature amount is stored in the storage device 11 in association with the target image data. Next, the user obtains an icon of the classification folder 11b in which the user attempts to insert the icon of the target image data, and stores the feature amount associated with the classification folder 11b (target folder) represented by the icon in the storage device 11. It is determined whether or not it has been performed. If the corresponding feature amount is not stored in the storage device 11 (step S13: YES), the target image data is stored in the target folder (step S16), and the feature amount of the target folder is determined based on the feature amount of the target image data. Update (step S17). Here, since there is no feature amount associated with the target folder, the classification registration unit 10a stores the feature amount itself of the target image data in the storage device 11 in association with the target folder, and ends the routine shown in FIG. I do.
[0016]
When the feature amount associated with the target folder is stored in the storage device 11 (step S13: NO), the classification registration unit 10a sends the feature amount and the feature amount calculated in step S12 to the similarity evaluation unit 10c. Deliver (step S14). Then, a result of determination that the images represented by the two feature amounts are similar to the extent that they are recognized as belonging to the same type is obtained from the similarity evaluation unit 10c, and it is appropriate to store the target image data in the target folder. (Step S15: YES), the classification registration unit 10a stores the target image data in the target folder (step S16). Next, the feature amount associated with the target folder is updated using the feature amount related to the target image data (step S17). Various forms of the processing in step S17 are conceivable. One specific example is as follows. First, after storing the target image data in the target folder, the classification registration unit 10a associates all the image data (including newly stored target image data) stored in the target folder at that time with each other. The stored characteristic amounts are read from the storage device 11. Here, as described above, the feature amount associated with one piece of image data forms a vector representing a feature of the image. The classification registration unit 10a obtains each feature amount associated with each image data in the target folder, that is, the center of gravity of a plurality of vectors representing the features of the image, and associates this with the target folder to the storage device 11. Store it. Note that the feature amount associated with the target folder may be obtained by another calculation method. Further, the timing of obtaining the feature amount may be any time from immediately after the image data is stored in the classification folder 11b to immediately before the new image data is stored in the classification folder 11b.
[0017]
On the other hand, if it is determined in step S15 that storing the target image data in the target folder is not appropriate, a warning message “This image data is not suitable for this folder. Is displayed on the display unit 14b (step S18). When the user inputs an instruction to respond positively to the warning message (step S19: YES), the classification registration unit 10a performs the processing of steps S16 to S17.
However, when the user inputs an instruction to respond negatively to the warning message displayed in step S18 (step S19: NO), the classification registration unit 10a stores the target image data in the unclassified folder 11a (step S18). S20).
[0018]
Although not shown, the classification registration unit 10a performs a routine shown in FIG. 8 for the image data stored in the unclassified folder 11a when the instruction to store the image data in the classification folder 11b is issued. Is configured to execute. Therefore, the user can move the image data stored in the unclassified folder 11a to the appropriate classified folder 11b.
The same applies to a case where an instruction to store image data already stored in the classification folder 11b in another classification folder 11b is issued.
[0019]
As described above, when the user attempts to store the image data in the desired folder, the similarity between the image data already stored in the folder and the image data to be stored is determined. It is determined whether it is appropriate to store the image data to be stored in a desired folder. If the result of the determination is negative, a warning message is issued to the user. Therefore, the user can prevent the image data from being stored in an inappropriate storage destination without paying careful attention to the storage destination of the image data.
[0020]
(3. Modified example)
As described above, the embodiments of the present invention have been described, but the present invention can be embodied in various other forms without departing from the main features. Note that, for example, the following modifications are possible.
[0021]
<Modification 1>
In the present modification, when attribute information is input from the operation input unit 14a, the control unit 10 generates the attribute information when creating a job package based on image data input via the image input device 12. Is included in the job package and stored in the storage device 11. The attribute information is, for example, information on the name, size, creation date, type of data format, and the like of the document data. Then, in the above-described step S27 or step S61, the control unit 10 causes the display unit 14b to display the attribute information in the job package instead of displaying the thumbnail image on the display unit 14b based on the found image data in the job package.
Further, attribute information associated with image data may be used as a feature amount.
[0022]
<Modification 2>
The image input device 12 may include a read / write device that reads and writes information from and to a medium such as a magnetic disk and a memory card, and a control unit therefor. The image input unit 12 may be a device that generates bitmap image data from data created by application software.
[0023]
<Modification 3>
The image data classification device 1 may be configured to further include an image output device. The image output device may include a printer and a printer control circuit that controls the driving of the printer, and may output an image represented by image data to paper. Further, the image output device 13 may be constituted by a read / write device for reading / writing information from / to a medium such as a magnetic disk or a memory card, and a control unit therefor. Such an image output device 13 outputs an image not to paper but to a medium such as a magnetic disk or a memory card.
Alternatively, the image output device 13 may be a data transfer device that exchanges data via the network interface 15. The image output device 13 may output data converted into a data format such as HTML (Hyper Text Markup Language) instead of outputting an image.
Further, the image output device 13 may be configured by, for example, a CRT (Cathode Ray Tube) and its control means.
[0024]
<Modification 4>
A configuration in which a plurality of image data are collectively stored in one classification folder may be employed. In such a configuration, the control unit 10 acquires each feature amount related to a plurality of image data, and associates each of these feature amounts with a classification folder (target folder) selected by the user as a classification destination folder. The feature amounts are compared with each other. Then, for image data determined to be inappropriate to be stored in the target folder, a warning message is displayed in the same manner as described above, and for image data determined to be valid to be stored in the target folder, Store in the target folder. According to such a configuration, it is possible to further reduce the labor and time required for the user's classification work.
[0025]
<Modification 5>
The feature amount calculated from the image data may be any one of the color tone, the aspect ratio, the distribution state of the luminance and the color, the distribution state of the edge, and the distribution state of the flat area, or may be plural. . For example, when the area of the image represented by the image data is divided into 24, the color tone and the edge state may be extracted from one area, and the feature amount may be represented as a 144-dimensional vector. In addition, the number of feature values may be reduced as much as possible without impairing the features of the image data by using a technique such as principal component analysis. Thus, the amount of calculation performed by the similarity evaluation unit 10c can be reduced. Therefore, the number of types of feature amounts to be calculated does not matter. Further, the number of regions to be divided does not matter.
[0026]
<Modification 6>
In the above-described embodiment, the features related to the image data are the color tone, the aspect ratio, the distribution state of luminance and color, the distribution state of edges, and the distribution state of flat areas of the entire image. Specifying image data using such feature amounts is very effective when the image represented by the image data occupies all or almost all of the document. However, when characters occupy almost all parts of the manuscript, such as a document such as a contract, it may not be very effective. Therefore, in the present modification, when the image represented by the image data includes a document, the features related to the image data are related to the document format such as character height, character spacing, line spacing, and vertical / horizontal writing. Treat as a feature. Hereinafter, a method of obtaining the feature amount related to the document format will be described.
Various conventional techniques can be applied to determine the characteristics related to the document format from the image read by the image input device 12 such as a scanner. An example of such a prior art is disclosed in paragraphs 0009 to 0012 of JP-A-5-108793. In this technology, a line spacing b and a character spacing d are obtained on the assumption that the writing is horizontal. However, in some cases, it is not known whether the original is written horizontally or vertically. Therefore, it is usual to utilize the fact that the line spacing is larger than the character spacing, and to compare b obtained as the line spacing and d obtained as the character spacing according to the technology, the larger one is taken as the line spacing, and the smaller one is taken as the character spacing. It is determined whether the document is written vertically or horizontally by a method of writing horizontally when the size is larger and writing vertically when the size d is larger.
[0027]
Further, another method for obtaining the characteristics related to the document format is as follows. The image represented by the image data is divided into a plurality of regions. Next, for each of the divided areas, a feature amount related to a document format including information such as character height, character spacing, line spacing, and vertical / horizontal writing is calculated. For example, values such as a character height "10 pt", a character spacing "15 pt", a line spacing "10.5 pt", and a vertical / horizontal writing "0" are obtained. Here, “0” is associated with vertical writing, and “1” is associated with horizontal writing.
[0028]
Next, a value that appears most frequently among the calculated feature amounts of all the regions is determined, and is set as the feature amount of the page. For example, if the character height is "10 pt" in 20 of the 24 areas, "12 pt" in 3 areas, and "16 pt" in one area, the page is "10 pt" is calculated as a feature amount related to the height of the character. The same applies to character spacing, line spacing, vertical / horizontal writing.
[0029]
Generally speaking, titles and headings occupy a smaller percentage of the entire page than the text that characterizes the document. Therefore, the most frequently occurring value can be estimated to be a quantity representing the feature of the text. As a result, even if a title or a heading part exists in the document, a feature amount can be accurately obtained from the image data.
As described above, one feature vector is obtained from one image data. An example of a feature vector f represented by a four-dimensional vector having four components is shown below.
f = (character height, character spacing, line spacing, [vertical writing: 0, horizontal writing: 1])
[0030]
【The invention's effect】
According to the present invention, the load on the user can be reduced, and the image data can be classified more efficiently.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a hardware configuration of an image data classification device 1 according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a functional configuration of a control unit 10 according to the embodiment.
FIG. 3 is a diagram for explaining a function of a classification registration unit 10a according to the embodiment.
FIG. 4 is a diagram for explaining functions of a classification registration unit 10a according to the embodiment.
FIG. 5 is a diagram for explaining a function of a feature amount extraction unit 10b according to the embodiment.
FIG. 6 is a diagram for explaining a function of a similarity evaluation unit 10c according to the embodiment.
FIG. 7 is a diagram for explaining a function of a similarity evaluation unit 10c according to the embodiment.
FIG. 8 is a flowchart showing a flow of an operation of the image data classification device 1 according to the embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image data classification apparatus, 10 ... Control part, 10a ..., Classification registration part, 10b ... Feature amount extraction part, 10c ... Similarity evaluation part, 11 ... Storage device, 12 image input device, 14 user interface, 14a operation input unit, 14b display unit, 15 network interface, 16 bus.

Claims (4)

A storage device for storing image data and a feature amount of the image data,
An operation input unit;
An output unit;
When an instruction to include the image data stored in the storage device in a certain group is given via the operation input unit, the feature amount of the image data and the feature amount associated with the group are determined. By comparing, it is determined whether or not it is appropriate to include the image data in the group. If the image data is appropriate, the image data is included in the group and stored in the storage device. An image data classification device, comprising: a classification registration unit that outputs a message indicating that the message is invalid by the output unit. After outputting the message to the output unit, the classification and registration unit, when an instruction to include the image data in the group is given via the operation input unit, the image data is output to the group. The image data classification device according to claim 1, wherein the image data is stored in the storage device. A feature amount extracting unit that extracts a feature amount of the image data,
The classification registration unit transfers the image data specified by the operation input unit to the feature amount extraction unit, and stores the feature amount extracted by the feature amount extraction unit in the storage device in association with the image data. The image data classification device according to claim 1, wherein a feature amount associated with a certain group is calculated from a feature amount stored in the storage device in association with image data belonging to the group. Receiving, via the operation input unit, an instruction indicating that the image data stored in the storage device should be included in a certain group;
Determining whether the image data is appropriate to be included in the group by comparing the feature amount of the image data with the feature amount associated with the group;
Storing the image data in the storage device by including the image data in the group when it is appropriate to include the image data in the group;
Outputting a message to the effect that the image data is not to be included in the group by the output unit.

Images