JP4650293B2 - Image classification display device and image classification display program - Google Patents

Description

  The present invention relates to an image classification display device and an image classification display program, and more particularly to a technique for searching for an image.

Conventionally, there has been proposed an image display device that calculates the degree of similarity of image data of a plurality of thumbnails and displays the thumbnails as a group (Patent Document 1).
JP 2005-236646 A

  However, in Patent Document 1, when the number of images to be grouped is enormous, the number of images belonging to the similarity group increases, and it is difficult to easily classify and search for images. there were. In Patent Document 1, image grouping is performed only once for images to be grouped. Therefore, the number of images belonging to each group cannot be adjusted, and an enormous number of images can be adjusted. There was a problem that an effective search was not possible for images.

  The present invention has been made in view of such circumstances, and provides an image classification display device and an image classification display program capable of easily classifying and searching images even when the number of images is enormous. Objective.

In order to achieve the above object, an image classification display device according to a first aspect of the present invention includes an image input means for inputting an image, and the image based on at least first image information and second image information. Clustering means for clustering, and the image in a coordinate space defined by at least two coordinate axes of a first coordinate axis corresponding to the first image information and a second coordinate axis corresponding to the second image information. an image display unit that displays a cluster icon corresponding to clusters generated are clustered, in the coordinate space, first coordinate axis icon indicating the first coordinate of the first coordinate axis, and the second coordinate axis A coordinate axis icon display means for displaying a second coordinate axis icon indicating the second coordinate of the coordinate system, and a coordinate axis icon selection for selecting the first and second coordinate axis icons. And when the first coordinate axis icon is selected, the clustering unit clusters the images of the cluster corresponding to the selected first coordinate axis icon again, while the second coordinate axis icon Is selected, the cluster image corresponding to the selected second coordinate axis icon is clustered again, and the image display means displays the cluster generated by the second clustering in the coordinate space. Features.

According to the first aspect , images are automatically clustered on the basis of image tag information (for example, date and time information, position information indicating a shooting location, etc.), image similarity, etc. Displayed in coordinate space. Thereby, even if the number of target images for search (clustering) is enormous, a desired image can be easily extracted. Furthermore, according to the first aspect, image clustering can be performed on the basis of desired image information of the first and second image information. Thereby, it is possible to easily search for an image based on desired image information.

In the image classification display device according to a second aspect of the present invention, in the first aspect , the clustering means calculates the similarity of the input image as the first image information, and the similarity The images are clustered based on the second image information, and the images are clustered based on date and time information when the image was taken or updated as the second image information. In the second aspect , the reference for image clustering is limited to the date / time information and similarity of the image.

The image classification display device according to a third aspect of the present invention provides the cluster corresponding to the designated cluster icon by indicating the cluster icon displayed in the coordinate space in the first or second aspect . The image display means further displays a list of images belonging to the selected cluster.

According to the third aspect , in addition to the effects described above, it is possible to easily search for a desired image by displaying a list of images in the cluster.

The image classification display device according to a fourth aspect of the present invention is the image classification display device according to the first or second aspect , wherein the cluster corresponding to the indicated cluster icon is indicated by indicating the cluster icon displayed in the coordinate space. A cluster selection unit that selects a cluster, and when the number of images belonging to the selected cluster is equal to or greater than a predetermined number, the clustering unit clusters the images belonging to the selected cluster again, and the image display unit Is characterized in that clusters generated by re-clustering are displayed in the coordinate space.

According to the fourth aspect , images belonging to the selected cluster can be further clustered when the number of images belonging to the selected cluster is large and search by list display is not suitable. Thereby, the search of an image can be performed easily.

The image classification display device according to a fifth aspect of the present invention is the image classification display device according to the fourth aspect , wherein the image display means is selected when the number of images belonging to the selected cluster is less than a predetermined number. A list of images belonging to a cluster is displayed.

According to the fifth aspect, when the clustering of the images belonging to the selected cluster is repeated and the number of images belonging to the selected cluster is reduced to a level suitable for the search by the list display, the desired images can be displayed by displaying the images as a list. Can reach the image.

An image classification display program according to a sixth aspect of the present invention includes: an image input function for inputting an image; a clustering function for clustering the images based on at least first image information and second image information; A cluster generated by clustering the images in a coordinate space defined by at least two coordinate axes of a first coordinate axis corresponding to one image information and a second coordinate axis corresponding to the second image information. the shown image display function that displays the corresponding cluster icon in the coordinate space, first coordinate axis icon indicating the first coordinate of the first coordinate axis, and a second coordinate of the second coordinate axis A coordinate axis icon display function for displaying two coordinate axis icons, a coordinate axis icon selection function for selecting the first and second coordinate axis icons, and the first A function of clustering the cluster image corresponding to the selected first coordinate axis icon again when the target axis icon is selected, and the selected second coordinate axis icon when the second coordinate axis icon is selected. The computer is realized with a function of clustering the cluster images corresponding to the two coordinate axis icons again and a function of displaying the clusters generated by the second clustering in the coordinate space .

An image classification display program according to a seventh aspect of the present invention, in the sixth aspect , selects a cluster corresponding to the indicated cluster icon by indicating the cluster icon displayed in the coordinate space. A computer is realized with a cluster selection function and a function of displaying a list of images belonging to the selected cluster.

An image classification display program according to an eighth aspect of the present invention, in the sixth aspect , selects a cluster corresponding to the indicated cluster icon by indicating the cluster icon displayed in the coordinate space. A cluster selection function, a function of re-clustering images belonging to the selected cluster when the number of images belonging to the selected cluster is greater than or equal to a predetermined number, and a cluster generated by re-clustering is represented in the coordinate space And a function of displaying on the computer.

An image classification display program according to a ninth aspect of the present invention provides a list of images belonging to the selected cluster when the number of images belonging to the selected cluster is less than a predetermined number in the eighth aspect . A display function is realized by a computer.

In addition to personal computer (PC), software and firmware including the image classification display program according to each aspect described above , video playback device (video deck, television), digital camera, personal digital assistant (PDA), mobile photo player, mobile phone The image classification display device of the present invention can be realized by applying to a device having image storage and playback functions such as the above.

  According to the present invention, images are automatically clustered on the basis of image tag information (for example, date and time information, position information indicating a shooting location, etc.), image similarity, and the like to form a two-dimensional or higher coordinate space. Is displayed. Thereby, even if the number of target images for search (clustering) is enormous, a desired image can be easily extracted.

  Preferred embodiments of an image classification display device and an image classification display program according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the main configuration of an image classification display device according to an embodiment of the present invention. An image classification display device 1 shown in FIG. 1 is composed of, for example, a personal computer (PC) and the like, and is a device for performing image storage, retrieval, classification, print order, and the like. As shown in FIG. 1, a central processing unit (CPU) 10 is connected to each block in the image classification display device 1 via a bus 12, and controls the operation of each block. The main memory 16 includes a storage area for storing the control program and a work area for executing the program. The hard disk device 18 stores an operating system (OS) of the image classification display device 1, an image classification display program according to the present invention, various application software, and the like. The CD-ROM device 20 reads data from a CD-ROM (not shown). The image classification display program according to the present invention can be installed in the image classification display device 1 by setting a CD-ROM in which the image classification display program is recorded in the CD-ROM device 20. The card interface unit (card I / F) 22 reads image data from the memory card 24. The display memory 26 temporarily stores display data. The monitor 28 is, for example, a CRT (Cathode Ray Tube) monitor or a liquid crystal monitor, and displays images, characters, and the like based on image data, character data, and the like output from the display memory 26. The keyboard 30 and the mouse 32 receive an operation input from the operator and input a signal corresponding to the operation input to the CPU 10. In the image classification display device 1 of FIG. 1, a touch panel, a touch pad, or the like can be used as a pointing device in addition to the mouse 32. The mouse controller 34 detects the state of the mouse 32 and outputs signals such as the position of the mouse pointer (reference C1 in FIG. 3) on the monitor 28 and the state of the mouse 32 to the CPU 10. A microphone 38 and a speaker 40 are connected to the audio input / output circuit 36, and various audio signals are input, and various operation sounds are reproduced and output in response to operation inputs from the keyboard 30 and the like. The communication interface unit (communication I / F) 42 performs communication with the network 44. A camera connection interface unit (camera connection I / F) 46 transmits and receives data to and from an imaging device (electronic camera, digital camera) 48.

  FIG. 2 is a functional block diagram showing the main configuration of the image classification processing unit 14. As shown in FIG. 2, the image classification processing unit 14 includes a similarity calculation unit 50, a tag information acquisition unit 52, a clustering unit 54, and a coordinate drawing unit 56.

  When the image classification (clustering) is performed on all of the images stored in the hard disk device 18 or the images in the designated folder, the similarity calculation unit 50 analyzes the images and characterizes the images. The amount (for example, image brightness and color, subject identification) is calculated. The tag information acquisition unit 52 acquires date and time information when an image is captured and updated from the tag information of the image stored in the hard disk device 18. The clustering unit 54 classifies the images stored in the hard disk device 18 into predetermined clusters based on the image similarity and date / time information (clustering). The coordinate drawing unit 56 calculates the average value of the cluster similarity of the images classified by the clustering unit 54. Then, the coordinate drawing unit 56 calculates coordinates in a coordinate space using the calculated image similarity and date / time information as coordinate axes based on the average value of the similarities. Then, the cluster is plotted in a coordinate space from the calculated cluster coordinates and output to the display memory 26. As a result, an image classification display screen is displayed on the monitor 28.

  FIG. 3 is a diagram showing an image classification display screen. The image classification display screen shown in FIG. 3 is used as a GUI (Graphical User Interface) for displaying a classified image and selecting a desired image, and includes an operation button display area 60 and an image cluster classification display area 62. have. Reference numeral 64 in the figure denotes a title bar, which displays the name of the image classification display program according to the present invention, the name of the current folder, and the like. The title bar 64 includes a minimize button for minimizing (iconizing) the image classification display screen, a maximize button for maximizing, an end button for closing, and the like. The configuration of the title bar 64 may differ depending on the operation system (OS) of the PC that constitutes the image classification display device 1.

  In the operation button display area 60, a “Back” button for returning to the screen displayed before the current screen, a “Forward” button for proceeding to the screen displayed later, and a screen for closing the image classification display screen are displayed. Various operation buttons such as an “end” button are displayed.

  In the image cluster classification display area 62, a coordinate space defined by the time axis corresponding to the date and time information when the image was taken or updated and the image similarity axis corresponding to the degree of similarity of the images is displayed. In this coordinate space, cluster icons 1 to 20 corresponding to the cluster of images are displayed. In the cluster icon, for example, thumbnails of representative images among images belonging to each cluster are displayed. Here, the representative image is, for example, the newest image or the oldest image, an image whose similarity is closest to the average value of the similarities of images in the cluster, and the like.

  In the coordinate space shown in FIG. 3, a total of 20 cluster icons are displayed in four columns in the time axis direction and five columns in the image similarity axis direction. Note that the number of columns in each coordinate axis direction can be arbitrarily set by the user.

  In FIG. 3, a time axis icon corresponding to the coordinate in the time axis direction is arranged on the left side of the time axis. The time axis icon indicates the date and time (year) when the image corresponding to the coordinate in the time axis direction is taken or updated. For example, the time axis icon “˜1994” is before 1994, the time axis icon “1995-1998” is from 1995 to 1998, the time axis icon “1999-2002” is from 1999 to 2002, The time axis icon “2003-2006” corresponds to the period from 2003 to 2006. When one of the time axis icons, for example, the time axis icon “1995-1998” is clicked with the mouse cursor C1, clusters 11 to 15 corresponding to the time axis icon “1995-1998” are selected. Then, as shown in FIG. 4, the images belonging to the selected clusters 11 to 15 are displayed in the coordinate space after being segmented in the time axis direction as “1995”, “1996”, “1997”, and “1998”. The Note that when the “Back” button is clicked on the image classification display screen shown in FIG. 4, the screen can return to the image classification display screen shown in FIG. 3.

  On the other hand, image similarity axis icons 1 to 5 are arranged below the image similarity axis. The image similarity axis icons 1 to 5 correspond to the degree of image similarity. When one of the image similarity axis icons is clicked, a cluster corresponding to the clicked image similarity axis icon is selected. The images belonging to the selected cluster are further subdivided and displayed in the similarity axis direction.

  Further, when a cluster icon is clicked on the image classification display screen of FIG. 3 or FIG. 4, a cluster corresponding to the cluster icon is selected. Then, the number of images belonging to the selected cluster is counted. When the number of images belonging to the selected cluster is greater than or equal to a predetermined number, the images belonging to the selected cluster are subdivided in the time axis direction and the image similarity axis direction and displayed in the coordinate space of the image classification display screen. . On the other hand, if the number of images belonging to the selected cluster is less than a predetermined number, a list of images belonging to the selected cluster is displayed as shown in FIG. When the “Back” button is clicked on the image list display screen shown in FIG. 5, the screen can return to the image classification display screen. In addition, the code | symbol B1 in FIG. 5 is a scroll bar.

  Next, the flow of image classification and display processing in the image classification display device 1 will be described with reference to the flowchart of FIG. First, when image classification processing is executed on all of the images stored in the hard disk device 18 of the image classification display device 1 or images in the designated folder (step S10), the images to be classified (clustered). The degree of similarity is calculated, the images are clustered for each degree of similarity (step S12), and the images are clustered based on the date and time information when the images are taken or updated (step S14).

  Next, a cluster icon corresponding to the cluster of the image is displayed in the coordinate space shown in FIG. 3 (step S16). Then, when it is detected that the cluster icon is clicked by the mouse 32, a cluster corresponding to the clicked cluster icon is selected (Yes in step S18). If the number of images belonging to the selected cluster is equal to or greater than the predetermined number (No in step S20), the process returns to step S12, and the images belonging to the selected cluster are clustered again based on the similarity and date / time information. (Steps S12 and S14) are displayed on the coordinate space (Step S16). On the other hand, when the number of images belonging to the selected cluster is less than the predetermined number (No in step S20), as shown in FIG. 5, a list of images belonging to the selected cluster is displayed (step S22).

  If the mouse 32 detects that the time axis icon or the image similarity axis icon has been clicked (Yes in step S24), as shown in FIG. 4, the column corresponding to the clicked coordinate axis icon is displayed. The cluster is further subdivided (step S26) and displayed in the coordinate space (step S16).

  According to the present embodiment, images are automatically clustered according to image tag information such as date and time information and image similarity and are displayed in a two-dimensional coordinate space, so the number of search target images is enormous. Even if it exists, a desired image can be taken out easily.

  In the present embodiment, for example, when the tag information of the image includes location information (GPS information) of the shooting location, information of the photographer, and the subject, the display is performed using these information as coordinate axes. It may be. Further, as information that can be used as coordinates, for example, image similarity based on luminance, color, and subject (identifying a human face area) can be used. Further, the coordinate space displayed on the image classification display screen is not limited to the two-dimensional shown in FIG. 3 or FIG. 4, and may be one-dimensional or three-dimensional. In addition, the type of each coordinate axis may be changed on the image classification display screen. For example, in FIG. 3, the time axis may be changed to a coordinate axis indicating position information.

  In addition to a personal computer (PC), software and firmware for executing the above processing are image storage such as a video playback device (video deck, television), a digital camera, a personal digital assistant (PDA), a portable photo player, and a cellular phone. By applying to a device having a reproduction function, the image classification display device of the present invention can be realized.

1 is a block diagram showing the main configuration of an image classification display device according to an embodiment of the present invention. Functional block diagram showing the main configuration of the image classification processing unit 14 Figure showing the image classification display screen Figure showing the image classification display screen Figure showing the image list display screen A flowchart showing the flow of image classification and display processing in the image classification display device 1

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image classification display apparatus, 10 ... CPU, 12 ... Bus, 14 ... Image classification process part, 16 ... Main memory, 18 ... Hard disk apparatus, 20 ... CD-ROM apparatus, 22 ... Card I / F, 24 ... Memory card , 26 ... display memory, 28 ... monitor, 30 ... keyboard, 32 ... mouse, 34 ... mouse controller, 36 ... audio input / output circuit, 38 ... microphone, 40 ... speaker, 42 ... communication I / F, 44 ... network, 46 ... Camera connection I / F, 48 ... Imaging device, 50 ... Similarity calculation unit, 52 ... Tag information acquisition unit, 54 ... Clustering unit, 56 ... Coordinate drawing unit

Claims (9)

An image input means for inputting an image;
Clustering means for clustering the images based on at least the first image information and the second image information;
The image is clustered and generated in a coordinate space defined by at least two coordinate axes of a first coordinate axis corresponding to the first image information and a second coordinate axis corresponding to the second image information. and image display means that displays a cluster icon corresponding to the cluster,
A coordinate axis icon display means for displaying, in the coordinate space, a first coordinate axis icon indicating a first coordinate of the first coordinate axis and a second coordinate axis icon indicating a second coordinate of the second coordinate axis;
Coordinate axis icon selection means for selecting the first and second coordinate axis icons,
When the first coordinate axis icon is selected, the clustering unit clusters the image of the cluster corresponding to the selected first coordinate axis icon again, while the second coordinate axis icon is selected. And clustering the cluster image corresponding to the selected second coordinate axis icon again,
The image classification display device , wherein the image display means displays a cluster generated by re-clustering in the coordinate space .   The clustering means calculates the similarity of the input image as the first image information, clusters the images based on the similarity, and captures or updates the image as the second image information The image classification display device according to claim 1, wherein the images are clustered based on the obtained date and time information. Further comprising cluster selection means for selecting a cluster corresponding to the indicated cluster icon by indicating the cluster icon displayed in the coordinate space;
3. The image classification display device according to claim 1, wherein the image display means displays a list of images belonging to the selected cluster. Further comprising cluster selection means for selecting a cluster corresponding to the indicated cluster icon by indicating the cluster icon displayed in the coordinate space;
The clustering means re-clusters the images belonging to the selected cluster when the number of images belonging to the selected cluster is a predetermined number or more;
3. The image classification display device according to claim 1, wherein the image display means displays a cluster generated by re-clustering in the coordinate space.   5. The image classification display device according to claim 4, wherein the image display means displays a list of images belonging to the selected cluster when the number of images belonging to the selected cluster is less than a predetermined number. . An image input function for inputting images,
A clustering function for clustering the images based on at least the first image information and the second image information;
The image is clustered and generated in a coordinate space defined by at least two coordinate axes of a first coordinate axis corresponding to the first image information and a second coordinate axis corresponding to the second image information. and image display function that displays a cluster icon corresponding to the cluster,
A coordinate axis icon display function for displaying a first coordinate axis icon indicating the first coordinate of the first coordinate axis and a second coordinate axis icon indicating the second coordinate of the second coordinate axis in the coordinate space;
A coordinate axis icon selection function for selecting the first and second coordinate axis icons;
When the first coordinate axis icon is selected, a function of reclustering the cluster image corresponding to the selected first coordinate axis icon;
When the second coordinate axis icon is selected, a function of re-clustering the cluster image corresponding to the selected second coordinate axis icon;
A function for displaying the cluster generated by the clustering again in the coordinate space;
An image classification display program for causing a computer to realize the above. A cluster selection function for selecting a cluster corresponding to the indicated cluster icon by indicating the cluster icon displayed in the coordinate space;
A function of displaying a list of images belonging to the selected cluster;
7. The image classification display program according to claim 6 , wherein the image classification display program is realized by a computer. A cluster selection function for selecting a cluster corresponding to the indicated cluster icon by indicating the cluster icon displayed in the coordinate space;
A function of re-clustering images belonging to the selected cluster when the number of images belonging to the selected cluster is a predetermined number or more;
A function for displaying the cluster generated by the clustering again in the coordinate space;
7. The image classification display program according to claim 6 , wherein the image classification display program is realized by a computer. 9. The image classification display program according to claim 8 , wherein when the number of images belonging to the selected cluster is less than a predetermined number, the computer realizes a function of displaying a list of images belonging to the selected cluster. .

Images