JP3484234B2 - Image management map expression method and apparatus - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video database utilization system, etc.
The present invention relates to a video management map representation method and apparatus that provides a visual interface for management, search, and use.

[0002]

2. Description of the Related Art Conventionally, when a metaphor is used as a user interface for grasping and accessing the contents of a video database, etc., it is managed as a file system of a computer and visualized two-dimensionally or three-dimensionally. There was something.

As an example of a use environment using a metaphor as an intuitive interface, there is one that realizes video management / search by a graphic interface simulating a library or a video shop. It is designed so that it can be used in a natural sense by bringing it closer to the actual usage environment.

On the other hand, there has been an example in which many files are expressed three-dimensionally for a file system such as Unix, and normal file management, retrieval and access are possible (G. George and et al., "Information Visualizatio
n Using 3D Interactive Animation ", ACM Communicati
ons Magazine, Vol.36, No.4, pp.57-71, (April 199
3)). In order to include all files in the management range in the visual field of the interface, the peripheral visual field is made smaller in perspective projection, and the one displayed behind is also introduced.

[0005]

Although these conventional techniques provide a visual intuition interface, they have some limitations when used as a general video management interface.

[0006] First, in the interface using the metaphor, there is a taxonomy that is premised even in the example of the library or the example of the video shop. In other words, it is based on an orderly classification with a hierarchical structure mainly for each field. In such an interface, it is necessary for the user to grasp the classification structure as prior knowledge or through familiarization with usage. In addition, when accessing, it is often impossible to determine for certain which category the video stored in the user's memory belongs to.

Further, when a new image is newly stored and added to the management data, which of the classifications to include becomes a big problem, and a more complicated device such as multiple heading items to a plurality of possible classifications is added. There is a problem that is required. Moreover, even if the taxonomy is a general taxonomy that is determined by factors outside of this taxonomy, or even if the user decides for himself, it is only based on the judgment at a certain point, and is flexible for new images. It is difficult to deal with, and there is a limit from the viewpoint of practical use, except for the application to a fixed, general-purpose database whose entire set is fixed in advance.

On the other hand, in the example of visualization of the file system, it is important for management to be able to intuitively grasp the entire image of the file, but this is actually a hierarchical classification when data is put in the file system. It has the same problem as the library metaphor in that it is supposed to be done. Also, since the existence of many files is visualized, there are naturally limits to the visualization method, and it is difficult to intuitively visualize some attributes of the image.

Therefore, an object of the present invention is to automatically arrange video information on the user interface as a whole in order to facilitate the management, retrieval, and use of the video data accumulated by the user. It is to be possible to provide a visual interface that can be grasped by a user and an access environment to individual images.

[0010]

According to the present invention, in order to manage, search, and use video data accumulated by a user, the attribute information about the video itself is used, and the attribute information selected by the user is used. By setting the absolute position in the feature space and the relative positional relationship with other videos on a map that is a projection representation of the feature space by attribute information, the overall layout of the video information can be grasped on the user interface. It provides a visual interface and access environment to individual images.

[0011]

In the present invention, since the existence position of each video material is displayed by, for example, an icon or other expression on a map in which the attribute space expressed by the specific selected attribute information is projected and expressed on the computer screen plane, The user can grasp the layout focusing on the attribute of the video information from the overall viewpoint, and can easily access it when searching for a specific video content.

When displaying the presence of the video material, the representative image of the video material information to be displayed is reduced in size, or the attribute information is visualized and displayed, so that the visual image is displayed on the map in which the attribute space is projected. It is possible to add an index function of video information that is easy to understand.

Since the projection condition of the attribute space on the map projected on the computer screen plane can be changed, the user can see the existing position of the image material in the attribute space in an easy-to-see manner focusing on the required attribute. be able to.

By filtering the attribute information of the video material according to a specific condition, and highlighting the representation of the video material that matches the result on the computer screen, the video material having the specific attribute information can be displayed. The search can be facilitated.

As the attribute information, time, geographical position, color or the like can be used. As a result, it is possible to intuitively understand when and where the video material was shot or created, and it is possible to easily find an event related to the video due to geographical, temporal proximity, or color proximity. It will be possible.

[0016] by operating on an icon or the like of the video material on the map the attribute space by projecting representations, appropriate image reproduction display, by performing <br/> video Display for video materials Hayami, the video information A user interface that is easy to access and use can be provided.

When inputting / updating a video material, not only the user inputs the attribute information, but also the information obtained by analyzing the video material is used as the attribute information to automate the input of the attribute information as the management information. You can also do it.

By handling the real-time video directly input from the camera or the real-time video accessed through the network as one of the video materials to be managed, the live video can be managed in the same manner.

As described above, according to the present invention, by using the attribute information obtained by analyzing the image itself and the attribute information input by the user, the absolute position in the feature space by the attribute information and the relative position with respect to other images can be obtained. The positional relationship can be automatically expressed.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. 1 shows a system configuration diagram of an embodiment of the present invention, FIG. 2 shows a processing procedure relating to the embodiment of the present invention, and FIG. 3 shows a processing procedure relating to input of video attribute information. It is a thing.

The processing procedure will be described below in order with reference to FIGS. First, a processing procedure by the video management control unit 2 when the attribute information regarding the video material is already stored in the video storage unit 3 or can be obtained through the line interface 6 will be described. When the management / access processing of the processing procedure 10 is started, in step 13, first, which attribute of the image attribute information is to be used for which range of view is registered in advance or is interactively input and registered. Data view attribute 1
Read out 1 and set.

Next, in step 14, for each video material within the target range set in the data view, the target attribute set in the data view is read in order to determine the mapping position in the attribute feature space. . At the next step 15, the icon to be actually displayed at the mapping position is constructed by reading out the necessary attribute information, and at step 16, the video icon is mapped and displayed at the set position calculated in the attribute feature space.

In step 17, the above steps 14-
It is determined whether the processing of 16 has been completed for all the processing targets, and if it is judged that the processing has been completed, the process waits for an event in step 18. Various processes can be applied to an event, but a typical example is a video access instruction from the user. When this video access instruction is issued, in step 19, the instruction video is accessed to the video storage unit 3, or if the material is not in the video storage unit 3, the material to be accessed via the line is accessed via the line interface 6. Access and display the video in the video display area or window.

If there is an instruction to change the attributes relating to the mapping display such as the viewpoint for three-dimensional representation and the condition for two-dimensional projection of three-dimensional data as an event, step 20 The recorded mapping attribute 12 is changed according to, and the processing for changing the mapping attribute is performed.

If there is an event of an instruction to change the type or range of the attribute information of interest, step 21
Then, the recorded data view attribute 11 is changed, and the process from step 13 is performed again. For the event instructing the end, move to step 22 and manage /
The access process ends.

The input registration process for newly registering management video material in the video storage unit 3 or an external storage unit is as follows.
This is performed as shown in FIG. When the input registration process of the processing procedure 30 is started, in step 31, a video is input from the video input unit 7 or a video is input via the line interface 6 and the video content is analyzed. As the analysis content, any data that can be automatically analyzed can be used. As usable analysis target data, for example, average image brightness, or image cut point information can be obtained according to Japanese Patent Application No. 5-317663 "Image cut point detection method and apparatus". According to the “Video shooting state detection method” of Japanese Patent Application No. 5-311922, information on the video shooting state such as camera operation can be obtained, and such information may be used.
Further, for example, representative color information on the target video can be obtained by the "video feature processing method" of Japanese Patent Application No. 5-1678889, but it is not limited to these.

In step 32, the analysis result information thus obtained is described and recorded as attribute information by the management information storage unit 4. Further, not only the automatically obtained attribute information but also the attribute information can be manually registered. In this case, in step 33, a manual input from the registrant is accepted and recorded in the management information storage unit 4. The information obtained by video analysis originally obtains information that changes with time, but when the video is mapped in the attribute feature space, it is represented in some way, such as by taking an average or taking an average evaluated by a certain weighting. Use attribute information.

The flow of the processing is as described above. Next, the mapping display in the attribute space of the image by the interface image will be concretely described. FIG. 4 shows an example in which an attribute space based on a representative time of a video and a representative geographical position is used as an interface, and FIG. 5 shows an example of an interface using hue attribute information of the video and representative time information. In the figure, 40 is a display screen, 41 is a video browsing window, 42 is a two-dimensional map, 43 is a material icon, 44 is a map x-axis, 45 is a map y-axis, 46 is a three-dimensional map, 47 is a variable lever, and 48 is Indicates the material mark. Further, 50 is a time axis and 51 is a hue axis.

The three-dimensional map 46 is a map in which the geographical position is expressed in a three-dimensional attribute space by two axes of x and y and three axes including a time axis.
Is a projected two-dimensional map projected on the xy plane under the filtering condition set in the three-dimensional map 46.
For example, when the xy plane is replaced with a map, a material mark 48 indicating the existence of the image is present at a corresponding position in the three-dimensional map 46 corresponding to the point on the map where the image was taken and the time when the image was taken. Is displayed.

The orientation of the coordinate axes in the three-dimensional map 46 can be set in a direction that is easy to see by rotating it by a conversational instruction such as a mouse operation. Two of the three axes (map x
When a plane is set by the axis 44 and the map y-axis 45) and the data is projected on the plane, the two-dimensional map 42 is obtained, and the material icon 43 showing the material image is displayed in the map. The plane position by setting the two axes is given by the variable lever 47 or the like as the remaining variable one-axis value. As for the target range to be projected, by setting the range centered on the plane position on the variable one axis, it is possible to project the data of only the range of interest. In addition to this, by performing conditional filtering of attributes including time and geographical position, it is also possible to perform emphasis processing on the mark for the corresponding data.

A figure projected two-dimensionally within the range designated by the three-dimensional map 46 is displayed in the two-dimensional map 42. The specific mark corresponding to each image material is, for example, simply a colored spot. An image obtained by reducing one representative image or various other display modes is conceivable. For example, an identifier as a video name may be displayed. A map image is attached to the two-dimensional map 42, and it is possible to intuitively know at which place and when the image was taken. In this case, when the variable lever 47 for changing the time axis is changed, the target time range is changed. Here, when the material icon 43 of the video material of interest is pointed to, the video browsing window 41 displays a reproduction display or a summary video of the video.

In the example of FIG. 5, the attributes used are hue information and time. The two-dimensional map 42 in the display screen 40 has a hue axis (HUE) 51 and a time axis (t) 50.
The coordinate system whose position is determined by is defined, and the material icon 43 represented by the representative image is displayed at the position determined by the representative hue and representative time of the material image. This makes it possible to know at what time (day, month, year, etc.) the image with a sense of color hue exists.

When the video to be accessed is directly connected to a remote camera input through a line and the time axis is used as one of the attributes, the live video is displayed at the most recent position. Will be done.

[0034]

As described above, according to the present invention,
For the video data accumulated by the user, managing them,
By using the attribute information obtained by automatically analyzing the image or manually inputting the attribute information about the image itself for searching and using, the absolute position in the feature space and other images by the attribute information are used. Since the relative positional relationship with and is automatically expressed visually on the user interface, the user has the effect of being able to grasp the arrangement of the video information from the overall viewpoint. Moreover, there is an effect that it is possible to immediately access a specific video content.

Further, when inputting a video, it is not necessary to consider a specific classification and it is not necessary to care about the classification when using it. In particular, when the geographical position of the image and the absolute time axis are used, there is an effect that it is possible to visually discover an event based on the geographical and temporal proximity of the image material.

[Brief description of drawings]

FIG. 1 is a system configuration diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing a processing procedure according to an embodiment of the present invention.

FIG. 3 is a diagram showing a processing procedure relating to input of video attribute information.

FIG. 4 is a diagram showing an interface example using a representative time of a video and a representative geographical position as attribute information.

FIG. 5 is a diagram showing an example of an interface using hue attribute information of a video and representative time information.

[Explanation of symbols]

1 Interface section 2 Video management controller 3 video storage 4 Management information storage 6 line interface 10 Management / Access processing 30 Input registration processing 40 display screen 41 Video browsing window 42 two-dimensional map 43 material icons 44 map x-axis 45 map y-axis 46 3D map 47 Adjustable lever 48 material mark 50 time axis 51 Hue axis

Continuation of the front page (56) References JP-A-6-180661 (JP, A) JP-A-6-130892 (JP, A) JP-A-5-61632 (JP, A) JP-A-5-30464 (JP , A) JP-A-6-141005 (JP, A) JP-A-4-348474 (JP, A) JP-A-5-207408 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB) Name) G06F 17/30

Claims (6)

(57) [Claims] 1. A method for expressing a video management map in a system for managing, accessing, and utilizing accumulated video materials or video materials accessible via a line, wherein time information and geo-position information are provided for each video material. , Or a first step of retaining at least three types of attribute information as management information, which includes one or more pieces of image-related information obtained by analyzing the image material or one of the image-related information input by the user. And data defining which selection attribute information is used as the target range among the selection attribute information of two or three selected from the attribute information included in the management information about the above-mentioned video material Second to set view attributes
And, for each video material within the target range set by the data view attribute, an expression indicating the existence of each video material is arranged in the two-dimensional or three-dimensional attribute space expressed by the selection attribute information. In order to display the attribute space on the computer screen plane, the selection attribute of the image material on the map in which the attribute space is projected on the computer screen plane according to a predetermined mapping attribute related to the projection condition when the attribute space is projected on the computer screen plane. A third step of determining a position based on information, an expression indicating the presence of the image material at the position determined by the third step, and a projection expression of the two-dimensional or three-dimensional attribute space including the expression 4 to display the completed map
And when there is a change instruction regarding the mapping attribute,
The process of changing the mapping attribute and repeating the processes of the third process and the fourth process, and changing the data view attribute when there is a change instruction regarding the data view attribute, process,
An image management map representation method comprising: repeating the processes of the third step and the fourth step. 2. The representative image representation of the image material information to be displayed or the visualization representation of the attribute information is displayed when the expression indicating the presence of the image material is displayed.
Video management map expression method described. 3. The attribute information of the video material is filtered according to a preset condition or a condition set by the user, and an expression indicating the presence of the video material on the map that matches the result is highlighted. The image management map expression method according to claim 1, wherein 4. Reproduction display of an image or display of an outline image for quick viewing of the image material by a user operation for an expression indicating the presence of the image material on the map displayed on a screen of a computer. The image management map expression method according to claim 1, wherein 5. The image management map representation method according to claim 1, wherein the image material includes a real-time image directly input from a camera or a real-time image accessible through a network. 6. A system for managing, accessing, and utilizing stored video material or video material accessible via a line, a video storage unit for storing video material, or a line interface for accessing a remote video material, Or, both of them are provided, and for each video material, time information, geolocation information, or one or more pieces of video-related information obtained by analyzing the video material or video-related information input by the user is provided. A management information storage unit for storing at least three types of attribute information including management information as management information, an interface unit for displaying information to the user and allowing the user to input information, and management information regarding the video material. It is selected from the attribute information included, among two or three selected attribute information, using any selection attribute information, etc. A data view attribute that determines whether or not to display the video material as the target range is set, and for each video material within the target range set by the data view attribute, each video material within the target range set by the data view attribute For projecting the attribute space onto a computer screen plane in order to display and display the presence of each of these image materials in the two-dimensional or three-dimensional attribute space represented by the selected attribute information. A position based on the selected attribute information of the video material on a map in which the attribute space is projected and expressed on a computer screen plane is determined according to a predetermined mapping attribute related to the projection condition, and the existence of the video material is indicated at the determined position. Make a representation and display a map that is a projection representation of the two-dimensional or three-dimensional attribute space containing the representation, If there is an instruction to change the mapping attribute, the mapping attribute is changed, and if there is an instruction to change the data view attribute, the data view attribute is changed to change the data view attribute and the mapping attribute. And a video management control unit that performs the display processing based on the above.