JP2010171798A - Video processing device and video processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize more efficient viewing by extracting only a scene with a high viewing value from among digest videos and presenting it to a user. <P>SOLUTION: In a feature amount extraction unit 16, a feature amount vector of scene information on a steady scene extracted by a specific scene extraction unit 13 is extracted from the scene information on a digest video and in a discriminator learning unit 18, a discriminator is produced from the feature amount vector. In the feature amount extraction unit 16, the feature amount vector of the scene information on the discrimination object scene extracted by a discrimination object scene extraction unit 14 is extracted from the scene information on the digest video of an extraction object. In a non-steady scene detection unit 19, the feature amount vector of the discrimination object scene is supplied to the discriminator for comparison, and whether the discrimination object scene is a non-steady scene is discriminated and the discriminated non-steady scene is displayed in a result display unit 22. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video processing apparatus and method, and more particularly, to a video processing apparatus and method for electronically processing video and providing only a characteristic scene to a user when playing back video content.

  Due to the increase in recording capacity of recording devices, a viewing style in which a television program is recorded and viewed later has become common. In addition, with the increase in the number of programs due to the increase in the number of channels, the number of programs that an individual views is also increasing greatly. Furthermore, due to an increase in the use of large-capacity video content via the Internet due to the improvement of the communication environment, it has become common to store video content on a personal computer through the Internet and enjoy it.

  With such an increase in the number of video contents, there is a demand for efficiently viewing only the scenes that the user wants to view when viewing the video contents.

  In order to respond to such a request, various video processing functions are added to a television program recorder or a computer. The following digest video creation techniques are known as conventional techniques for video processing functions for realizing efficient viewing.

  First, between frame images constituting a certain video, the video is divided by regarding a point where the change in the image feature amount is a certain level or more as a cut switching. Each of the divided videos is called a scene. That is, the video is divided into a plurality of scenes. Here, the image feature amount is, for example, the ratio of the color of each pixel included in the image, the distribution of the average value of the pixels for each area obtained by dividing the image in a grid pattern, or the like.

  Next, for example, the feature amount of the first frame image of each scene is calculated using the first frame image of each scene as the feature amount of the scene. A digest video can be created by clustering scenes with similar feature quantities into the same group. By displaying such a digest video to the user and specifying and viewing only a specific group of scenes, the user can efficiently view the video content (see Patent Document 1).

  However, with the conventional digest creation function, it is not possible to extract and present to the user an unusual scene that is highly worth viewing in a specific scene, and it is possible to realize more efficient viewing. There is a problem that you can not.

  For example, considering an example in which a specific scene in a baseball broadcast is an infield defensive scene, the conventional digest creation function can create a digest video by extracting only the infield defensive scene from the video content of the baseball broadcast. there were. However, the digest video that is created is a scene that is not suitable for the majority, that is, a scene that is not worth watching, and a very small number of fine play scenes and error scenes that are unusual and particularly worth watching. Therefore, it has not been considered to distinguish between these scenes with low viewing value and scenes with high viewing value. Accordingly, there is a problem that a digest video composed only of scenes that are particularly worth watching is not created, and more efficient viewing cannot be realized.

Japanese Patent No. 40673326

  The present invention provides a video processing apparatus and method capable of realizing more efficient viewing by extracting only scenes having high viewing value from digest video and presenting them to the user. With the goal.

  In order to solve the above-described problems, a video processing apparatus according to the present invention includes a video database for storing video content information, and digest information for storing digest information for specifying a digest video composed of a specific scene included in the video content. A specific scene extraction unit that extracts scene information that specifies a plurality of stationary scenes composed of a plurality of frame images from the digest information stored in the digest information database, and the database extracted by the specific scene extraction unit Extracting a plurality of stationary scenes corresponding to scene information from the video database, and extracting a feature vector for each frame image of a predetermined time interval constituting each of the plurality of stationary scenes; The plurality of features extracted by the feature amount extraction unit A classifier learning unit that creates a classifier based on a set of feature vectors for each frame image corresponding to a stationary scene, and an identification target scene composed of a plurality of frame images are identified from the extraction target digest information stored in the digest information database. An identification target scene extraction unit for extracting scene information to be extracted, and the frame constituting the identification target scene by extracting the identification target scene corresponding to the scene information extracted by the identification target scene extraction unit from the video database. The feature amount extraction unit that extracts a feature amount vector for each image and the feature amount vector extracted by the feature amount extraction unit are supplied to the discriminator for comparison, whereby the scene to be identified is an unsteady scene The non-stationary scene detection unit for identifying whether or not It is characterized in that it comprises and a result display section for displaying the other has been the unsteady scene.

  In the video processing method of the present invention, the specific scene extraction unit stores a plurality of frame images from the digest information that is stored in the digest information database and specifies the digest video that includes the specific scene included in the video content. Extracting the scene information identifying the stationary scene of the image, and in the feature amount extraction unit, extracting the plurality of stationary scenes corresponding to the scene information extracted in the step from the video database, A step of extracting a feature vector for each frame image of a predetermined time interval constituting each stationary scene, and a step of extracting the feature vector in the discriminator learning unit correspond to the plurality of stationary scenes. Classifier based on feature vector set for each frame image A step of creating, a step of extracting scene information for identifying a scene to be identified consisting of a plurality of frame images from the extraction target digest information stored in the digest information database in the scene to be extracted of the identification target, and the feature amount extraction In the section, the identification target scene corresponding to the scene information extracted by the step of extracting scene information for identifying the identification target scene is extracted from the video database, and for each frame image constituting the identification target scene. A step of extracting a feature quantity vector; a step of extracting whether or not the identification target scene is a non-stationary scene by a step of extracting a feature quantity vector of the identification target scene in a non-stationary scene extraction unit; and a result Whether or not the display unit is the unsteady scene A method characterized by comprising the step of displaying the non-stationary scene identified by identifying.

  According to the present invention, it is possible to provide a video processing apparatus and method that can realize more efficient viewing by extracting only a scene worth watching from a digest video and presenting it to the user. be able to.

It is a block diagram which shows the structure of the video processing apparatus which concerns on 1st Embodiment. It is a table figure which shows the data of the video content recorded on video DB. It is a schematic diagram which shows a digest image | video. It is a table figure which shows the data of the digest image | video recorded on digest information DB. It is explanatory drawing explaining a mode that a feature-value vector is extracted from a video unit. It is explanatory drawing which shows the relationship between a feature-value vector and an identification surface. It is explanatory drawing explaining the method to produce a discriminator. It is the schematic diagram which showed the method of identifying that an identification object scene is a stationary scene. It is the schematic diagram which showed the method of identifying that an identification object scene is an unsteady scene. It is the schematic diagram which showed the method of identifying that an identification object scene is an unsteady scene. It is a flowchart which shows operation | movement of the video processing apparatus which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of a specific scene extraction part. It is a flowchart which shows operation | movement of a feature-value extraction part. It is a flowchart which shows operation | movement of a discriminator learning part. It is a flowchart which shows operation | movement of the identification object scene extraction part. It is a flowchart which shows operation | movement of an unsteady scene detection part. It is a block diagram which shows the structure of the video processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows operation | movement of the video processing apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the video processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows operation | movement of the video processing apparatus which concerns on 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the video processing apparatus according to each embodiment described below takes out normal scenes from existing digest video, and causes the discriminator to learn feature quantities of these scenes as learning data. And it is a video processing apparatus which takes out and displays a scene different from usual from the digest video to view using this discriminator. In the following description, a digest video to be viewed is referred to as an extraction target digest video.

(First embodiment)
First, the video processing apparatus according to the first embodiment will be described with reference to FIGS. In the video processing apparatus according to the present embodiment, the number of device configurations is the smallest.

  First, a configuration for creating learning data from an existing digest video and causing a discriminator to learn will be described. This configuration includes specific scene extracting means for extracting scene information for specifying a plurality of stationary scenes composed of a plurality of frame images from the digest information stored in the digest information DB 12, and a plurality of stationary information corresponding to the extracted scene information. A means for extracting a scene from the video DB 11 and extracting a feature vector for each frame image constituting each of the plurality of stationary scenes, and a feature vector set for each frame image corresponding to the extracted plurality of stationary scenes. A classifier having a reference vector. Hereinafter, these configurations will be described in detail with reference to FIGS. 1 to 5.

  FIG. 1 is a block diagram showing the configuration of the video processing apparatus according to the first embodiment. As shown in FIG. 1, first, in the video processing device according to the first embodiment, a plurality of video contents are stored in the form of a table shown in FIG. It is recorded.

  Each record constituting the table shown in FIG. 2 includes a video number for identifying video content, a video name that is the name of the video content, a type that is the type of video content, a creation date and time that is the date and time when the video content was created, and a video It consists of a video data link indicating link destination information to a recording area (not shown) where video data is actually stored, which is the video time of the content. In the above example, the video DB 11 stores link destination information to the storage area where the video content is actually stored, but the video content may actually be stored.

  In the digest information DB 12 shown in FIG. 1, digest information for specifying a digest video composed of a specific scene included in the video content recorded in the video DB 11 is recorded.

  FIG. 3A is a schematic diagram showing a digest video created from the “professional baseball all-star game first game” recorded in the video DB 11 as an example of the digest video described above. The digest video shown in FIG. 3A has a batter-to-pitcher scene. When the batter hits, it moves to the infield defense scene, then the batter player's face up scene, and the pitcher-to-batter scene. It is an image.

  The digest video shown in FIG. 3A is recorded in the digest information DB 12 in the form of a table shown in FIG. 3B. In each record constituting this table, scene information is recorded.

  The scene information has a continuous scene number for each scene and a scene label, and further indicates position information indicating a position in the video content of each scene, and whether each scene is a low-value stationary scene to be viewed, And information for distinguishing whether the scene is a non-stationary scene that is highly worth viewing. The steady scene and the unsteady scene will be described in detail later.

  Here, the scene label is a video name of each scene. For example, in the example of the digest video of the baseball broadcast described above, the scene labels are “pitcher vs batter”, “defense scene”, “player up”, and the like.

  The position information corresponds to the video number corresponding to the video number of the video content recorded in the video DB 11 and the start frame number indicating the start frame of each scene corresponding to the frame number of the video content recorded in the video DB 11. And an end frame number indicating the end frame of the scene.

  Information for distinguishing whether the scene is a stationary scene or an unsteady scene includes additional information 1 to N. The additional information is N = 2 in the example of FIG. 3B. For example, in the additional information 1, information indicating how many times the front or back play scene is recorded is recorded. Further, in the additional information 2, for example, scorebook information such as “1go” (meaning 1 塁 goro) and “3 lost” (meaning that 3rd hand has an error) for the defensive scene is recorded. That is, whether each scene is a stationary scene or an unsteady scene is distinguished by this additional information.

  Here, the stationary scene is a scene that has a high probability of being normally generated and that is particularly low in view value. For example, a stationary scene in a baseball broadcast video is a scene that is out in a defensive scene. That is, among scenes having a scene label of “defense scene”, the scene has additional information such as “1go”.

  On the other hand, a non-stationary scene is a scene that has a low probability of being normally generated and that is particularly worth watching. For example, an unsteady scene in a baseball broadcast video is an error scene in a defensive scene. That is, among scenes having a scene label of “defense scene”, the scene has additional information such as “3 lost”.

  The additional information may be input by a viewer or the like, for example, or may be information acquired from the outside via the Internet or the like as in a third embodiment to be described later. Good.

  Next, a specific scene extracting means for extracting scene information for specifying a plurality of stationary scenes composed of a plurality of frame images from the digest information stored in the digest information DB 12 will be described.

  As shown in FIG. 1, a specific scene extraction unit 13 is connected to the digest information DB 12 described above. The specific scene extraction unit 13 extracts only the scene information of the steady scene from the scene information of at least one designated digest video among the existing digest videos recorded in the digest information DB 12. The extracted scene information of the steady scene is recorded in the specific scene information RAM 15.

  The stationary scene is extracted by, for example, a viewer or the like specifying a scene label and additional information.

  For example, it is extracted when the viewer designates scorebook information of an out scene such as “defensive scene” as a scene label and “1go” as additional information.

  In the specific scene information RAM 15 shown in FIG. 1, the scene information of the steady scene extracted by the specific scene extracting unit 13 is given a new continuous scene number in the order of extraction and is recorded in the form of a table. ing.

  Next, a plurality of stationary scenes corresponding to the scene information specifying the plurality of stationary scenes extracted by the above-described means are extracted from the video DB 11, and frame images at predetermined time intervals that respectively constitute the plurality of stationary scenes. A means for extracting the feature vector for each will be described.

  As shown in FIG. 1, a feature quantity extraction unit 16 is connected to the specific scene information RAM 15 described above. The feature quantity extraction unit 16 refers to the scenes of the stationary scene recorded in the specific scene information RAM 15 and extracts the stationary scenes corresponding to the scene information from the video DB 11 connected to the feature quantity extraction unit 16. . Then, a feature vector in units of video at predetermined time intervals constituting each of the steady scenes is calculated. Further, the calculated feature vector is recorded in the feature vector RAM 17 connected to the feature extraction unit 16.

  Here, the video unit is a time-series frame image obtained by sampling the extracted scene at a predetermined time interval such as a frame unit or several frame units. FIG. 4 is an explanatory diagram schematically showing how a feature vector is calculated for each video unit from N video units along a time series obtained by sampling from one of the extracted scenes. As shown in FIG. 4, assuming that the number of video units is N frames, N feature vectors v1 to vN are calculated. Note that the sampling time interval may be constant, or may be sampled at different time intervals.

Also, the feature quantity vector is a vector display of the feature quantity calculated for each video unit. For example, if video content is recorded in the video DB 11 as data in the Mpeg-1 format, the video unit of one scene constituting this video content is defined as the I-th frame image (I frame) and P from the beginning of the scene. It is assumed that this is the th frame image (P frame). At this time, a motion vector is calculated from the I frame and the P frame. Further, a parameter indicating the motion of the relay camera is estimated from this motion vector. The parameter indicating the motion of the relay camera is defined as a feature vector of the I frame. That is, from the j-th video unit sampled from a certain scene _{, K} parameters c _{j, 1} , c _{j, 2} ,..., C _j, K are set as parameters indicating the movement of the relay camera. When calculated, the feature quantity vector Vj of the j-th video unit is Vj = (c _{j, 1} , c _{j, 2} ,..., C _{j, K} ).

  In the feature quantity vector RAM 17 shown in FIG. 1, the feature quantity vectors of the stationary scene extracted by the feature quantity extraction unit 16 are displayed in the form of a table together with the scene number of the stationary scene and the video unit number of the video unit constituting each scene. It is recorded. This video unit number is a consecutive number assigned in order from the beginning of each scene.

  Next, a classifier that is learned from learning data using a feature vector set of frame images corresponding to a plurality of stationary scenes extracted by the above-described means will be described.

  As shown in FIG. 1, a classifier learning unit 18 is connected to the feature vector RAM 17 described above. The discriminator learning unit 18 creates a discriminator from the set of feature amount vectors of the stationary scene recorded in the feature amount vector RAM 17. Here, as will be described later, the discriminator means an identification plane learned from learning data that is a set of feature quantity vectors. And the information of the discriminator which consists of the discriminator number provided for every discriminator and a discriminating surface is recorded on discriminator DB20.

Here, as schematically shown in FIG. 5A, the identification surface 181 is the same (for example, the same unit among the video units constituting each stationary scene extracted from the existing digest video in the feature vector space (for example, This is a plane learned using the feature vector (V _i ¹ , V _i ² ,..., V _i ^M ) of each i-th video unit as learning data. That is, the reference vectors for comparing the feature vector ^{_{(V i 1, V i 2}} , ···, V i M) set of later a feature quantity vector of the identification target scene at unsteady scene detection unit 19 The surface formed in the feature vector space by this reference vector is the identification surface.

It should be noted that, in the formation of the identification surface, if there is a vector indicating a point that is completely different from other vectors among the feature amount vectors (V _i ¹ , V _i ² ,..., V _i ^M ), the vector is excluded. Thus, the identification surface is learned. The range of the feature vector to be excluded can be arbitrarily designated according to the identification accuracy for identifying the unsteady scene from the classification target scene by the unsteady scene detection unit 19 described later.

Figure 5B, when the learning data is a set of N _M-number of feature vectors in total of ordered along the sequence shows how to learn the classifier fi (i = 1~N _M) schematically described FIG. As shown in FIG. 5B, from the feature quantity vector RAM 17, the i-th video unit has each of the video units constituting each stationary scene having scene numbers 1, 2,... the amount vector ^{_{V i 1, V i 2,}} ···, when extracting _V ^{i M,} the feature amount vector space, the feature vector ^{_{V i 1, V i 2,}} ···, consisting of all the _V ^{i M} By surrounding the set 182, an identification surface 181 as shown in FIG. 5A is formed. The discriminator fi is created by learning the discriminating surface 181. Note that the identification surface 181 as shown in FIG. 5A may be formed by a set of feature amount vectors within a range arbitrarily designated according to identification accuracy from all extracted feature amount vectors.

  For example, the classifier to learn is 1-class SVM. When the parameter indicating the motion of the relay camera is used as the feature vector that is an element of the learning data as described above, the 1-class SVM has a plurality of relay cameras that indicate the motion of each relay scene in the feature vector space. A discriminant plane formed by enclosing the feature vector is learned.

  With the above configuration, learning data can be created from an existing digest video and can be learned by a discriminator.

  Next, a configuration for extracting and displaying a scene different from normal, that is, an unsteady scene from the digest video to be extracted using the discriminator that has learned learning data will be described. This configuration includes means for extracting scene information for identifying an identification target scene composed of a plurality of frame images from extraction target digest information stored in the digest information DB 12, and identification corresponding to the scene information of the extracted identification target scene. Means for extracting the target scene from the video DB 11 and extracting a feature vector for each frame image of a predetermined time interval constituting the identification target scene; and a feature vector for each frame image constituting the extracted identification target scene Is supplied to the discriminator, and a reference vector that is learning data of the discriminator is compared with a feature vector of the scene to be discriminated to output a non-stationary scene. Hereinafter, these configurations will be described with reference to FIGS. 1 and 6A to 6C.

  It should be noted that the extraction of the non-stationary scene can be executed in the same manner as the above-described specific scene extraction unit 13 refers to the additional information to distinguish whether the scene is a stationary scene or an unsteady scene. However, for example, if the digest video to be extracted is a digest video that has just been newly recorded in the digest information DB 12, no additional information is attached to each scene, and it is a steady scene or an unsteady scene. Cannot be distinguished. Also, even if the digest video to be extracted is an existing digest video and additional information is added to each scene, scenes that are not represented on the score of fine play scenes or the like but are worth watching are added. It cannot be distinguished only by referring to the information. Therefore, it is necessary to identify whether each scene of the extraction target digest video is a stationary scene or an unsteady scene using the above-described classifier.

  First, a means for extracting scene information for identifying an identification target scene composed of a plurality of frame images from extraction target digest information stored in the digest information DB 12 will be described.

  First, for example, the viewer designates the digest video to be extracted from the digest video specified by the digest information recorded in the digest information DB 12 described above. As described above, when the extraction target digest video is specified, the identification target scene is specified from the digest information for specifying the extraction target digest video in the identification target scene extraction unit 15 shown in FIG. 1 connected to the digest information DB 12. The scene information to be extracted is extracted. The identification target scene specified by this scene information is a scene having the same scene label as the scene label specified by the specific scene extraction unit 13. For example, in the above-described baseball broadcast example, when the scene label specified by the specific scene extraction unit 13 is a defensive scene, the identification target scene is a defensive scene in the extraction target digest video.

  Such a scene to be identified is extracted by designating the same scene label as the scene label designated by the specific scene extraction unit 13. The extracted scene information of the identification target scene is recorded in the specific scene information RAM 15 connected to the identification target scene extraction unit 15.

  In the specific scene information RAM 15, scene information of the identification target scene extracted by the identification target scene extraction unit 15 is recorded along with the scene information of the stationary scene extracted by the specific scene extraction unit 13.

  Next, means for extracting an identification target scene corresponding to the scene information of the identification target scene extracted by the above-described means from the video DB 11 and extracting a feature vector for each frame image constituting the identification target scene will be described. .

  As described above, the feature amount extraction unit 16 is connected to the specific scene information RAM 15, and the feature amount vector of each unit video constituting the identification target scene is obtained in the same manner as the calculation of the feature amount vector for the stationary scene. The calculated feature quantity vector is recorded in the feature quantity vector RAM 17.

  The feature quantity vector RAM 17 stores the feature quantity vector of the identification target scene together with the feature quantity vector of the stationary scene.

  Next, by supplying to the discriminator a feature vector for each frame image constituting the scene to be identified extracted by the above-described means, the reference vector of the discriminator and the feature vector of the scene to be discriminated are obtained. A means for outputting an unsteady scene in comparison will be described.

  As shown in FIG. 1, an unsteady scene detection unit 19 is connected to the feature vector RAM 17 together with the classifier learning unit 18 described above. The unsteady scene detection unit 19 identifies whether or not the identification target scene specified by the scene information extracted by the identification target scene extraction unit 14 is an unsteady scene. Then, the scene information of the scene to be identified that is identified as an unsteady scene is recorded in the unsteady scene information DB 21 connected to the unsteady scene detection unit 19.

  Here, the identification of whether or not the classification target scene is a non-stationary scene is performed by using the feature quantity vector of the classification target scene and the information of the classifier recorded in the classifier DB 20 connected to the non-stationary scene detection unit 19. Use to identify. That is, a target classifier is identified from the classifier information recorded in the classifier DB 20, and a feature quantity vector of the classification target scene is supplied to the identified classifier. And are identified by comparing

  FIG. 6A, FIG. 6B, and FIG. 6C show the feature quantity vectors vi (i = 1 to N) of the scenes to be identified for identifying whether the scenes are stationary or non-stationary scenes, respectively. It is the schematic diagram which showed a mode that the recognition object scene is identified by supplying to N) in time series order. As shown in FIG. 6A, FIG. 6B, and FIG. 6C, the discrimination test is performed on the feature quantity vector vi from i = 1 to N in order using the discriminator fi. If it is determined as a result of this discrimination test that the feature quantity vector vi of the scene to be discriminated exists in the discrimination plane of the discriminator fi (indicated by a circle in FIGS. 6A, 6B, and 6C), the next feature quantity vector Proceed to identification.

  As shown in FIG. 6A, if all the feature vectors vi of the identification target scene are present in the identification plane of the corresponding classifier fi, the identification target scene is identified as a stationary scene.

  On the other hand, as shown in FIG. 6B, if it is determined that the feature vector vi exists outside the discrimination plane of the discriminator fi (indicated by x in FIG. 6B), the scene to be identified is set as a non-stationary scene at that time Identify.

  If the above-described discrimination test is performed in order from i = 1 to N, and it is determined that the feature vector vi exists outside the discrimination plane of the discriminator fi for two or more times, it may be discriminated as an unsteady scene. Good. FIG. 6C is an example of identifying as an unsteady scene when it is determined that the image exists outside the identification plane for two consecutive times.

  For example, in a baseball defensive scene, when the feature quantity vector for each video is used as a parameter indicating the motion of the relay camera, if the identification result for all video units of the defensive scene to be identified is ○, the scene is a steady scene, that is, It is considered that the scene was out of the scene without any abnormal movement of the relay camera. On the other hand, if at least one or more identification results continue in the middle, it is considered that some abnormality has occurred in the motion of the relay camera at that time, and the scene is identified as an unsteady scene.

  The scene information of the unsteady scene extracted by the unsteady scene extraction unit described above is recorded in the unsteady scene information DB 21 connected to the unsteady scene detection unit 19.

  In the unsteady scene information DB 21 shown in FIG. 1, the scene information of the identification target scene identified as the unsteady scene is recorded in the form of a table.

  A result display unit 22 is connected to the unsteady scene information DB 21. The result display unit 22 is also connected to the video DB 11 described above, and corresponds to the video DB 11 connected to the result display unit 22 by referring to the scene information of the non-stationary scene recorded in the non-stationary scene information DB 21. A scene is extracted, and the extracted scene is displayed. The display form is, for example, a thumbnail display, but is not limited in this embodiment.

  As described above, since the unsteady scene is extracted from the digest video to be extracted by the unsteady scene extraction unit and displayed, only the scenes worth viewing are extracted, so that efficient viewing is realized. be able to.

  The configuration of the video processing apparatus according to the first embodiment has been described above.

  Next, the operation of the above-described video processing apparatus will be described with reference to FIGS.

  First, with reference to FIG. 7, a video processing method by the video processing apparatus according to the first embodiment will be briefly described. FIG. 7 is a flowchart illustrating a video processing method by the video processing apparatus according to the first embodiment.

  As shown in FIG. 7, in the video processing method of the present embodiment, first, learning data is created from an existing digest video and is learned by a discriminator. In this method, first, at least one digest video is specified from existing digest videos, and then a specific scene label and additional information are specified, whereby the specific scene extraction unit 13 performs the digest. The scene information of the stationary scene is extracted from the scene information of the existing digest video recorded in the information DB 12 (S101). The extracted scene information of the steady scene is recorded in the specific scene information RAM 15.

  Next, when a stationary scene is extracted and recorded in the specific scene information RAM 15, the feature amount extraction unit 16 calculates a feature amount vector for each video unit constituting the extracted stationary scene (S102). The calculated feature quantity vector is recorded in the feature quantity vector RAM 17.

  Next, when the feature vector of each video unit constituting the stationary scene is calculated and recorded in the feature vector RAM 17, the classifier learning unit 18 records the feature vector in the feature vector RAM 17 as shown in FIG. 5B. The set 182 of feature quantity vectors thus obtained is used as learning data, and a discriminating surface 181 as shown in FIG. 5A is learned as a discriminator fi (S103). Information on the discriminator fi is recorded in the discriminator DB 20.

  Through the above steps S101 to S103, learning data is created from the existing digest video, and the discriminator is trained. Subsequently, the unsteady scene is extracted from the extraction target digest video, and the extracted unsteady scene is displayed.

  In this method, first, when an extraction target digest video is specified from the digest video recorded in the digest information DB 12, the identification target scene extraction unit 14 determines a specific scene from the scene information of the specified extraction target digest video. The scene information of the scene to be identified having the same scene label as the label is extracted. The extracted scene information of the scene to be identified is recorded in the specific scene information RAM 15. Then, the feature amount vector of the identification target scene recorded in the specific scene information RAM 15 is calculated (S104). The calculated feature vector of the scene to be identified is recorded in the feature vector RAM 17.

  Next, when an identification target scene is extracted, a feature vector of this scene is calculated and recorded in the feature vector RAM 17, the unsteady scene detection unit 19 records the feature quantity of the identification target scene recorded in the feature vector RAM 17. Using the vector and the information of the classifier recorded in the classifier DB 20, it is identified whether or not the scene to be identified is an unsteady scene (S105). The scene information of the identified unsteady scene is recorded in the unsteady scene information DB 21.

  By the steps of S104 and S105, an unsteady scene is extracted from the extraction target digest video.

  Finally, when the scene information of the unsteady scene is extracted and recorded in the unsteady scene information DB 21, the unsteady scene is extracted from the video DB 11 by referring to this scene information, and the unsteady scene is obtained as a result. The information is displayed on the display unit 22 (S106).

  Next, each step described above will be described in detail with reference to FIGS.

  A method for extracting a stationary scene from an existing digest video in step S101 will be described.

  FIG. 8 is a flowchart showing a method for extracting a stationary scene in the specific scene extracting unit 13. As shown in FIG. 8, in the extraction of a stationary scene, first, i = 1 is set (S201), and it is determined whether or not i exceeds the number of existing digest video scenes recorded in the digest information DB 12 (see FIG. 8). S202). If so, the extraction operation ends. Here, the number of scenes refers to the number of scenes included in the digest video. Since each scene constituting the digest video is assigned a continuous number from the first in chronological order, the number of scenes is substantially the maximum value of the scene number.

  On the other hand, if i does not exceed the number of scenes, the specified scene label and additional information are referred to in the scene with the scene number i in the existing digest video (hereinafter referred to as the i-th scene). If the i-th scene is a stationary scene to be extracted, the scene information of the i-th scene is recorded in the specific scene information RAM 15 (S203). The scene information recorded in the specific scene information RAM 15 is given a new consecutive scene number in the order of recording.

  Next, i = i + 1 is set (S204), and the next scene, that is, the i + 1th scene is referred to.

  A stationary scene is extracted from an existing digest video by repeating the operations of S202 to S204 described above until i exceeds the number of scenes in the digest video to be extracted.

  Next, a method for calculating a feature vector from a steady scene in step S102 will be described.

  FIG. 9 is a flowchart showing a method for calculating the feature quantity vector for each video unit constituting the steady scene in the feature quantity extraction unit 16.

  As shown in FIG. 9, in calculating the feature vector, first, i = 1 is set (S301), and it is determined whether i exceeds the number of scenes of the stationary scene recorded in the specific scene information RAM 15 (see FIG. 9). S302). If so, the extraction operation ends.

  On the other hand, if i does not exceed the number of scenes, j = 1 is set (S303), and j is a video unit of the video unit constituting the i-th scene recorded in the specific scene information RAM 15. It is determined whether the number is exceeded (S304). If so, i = i + 1 is set (S305), and the process returns to S302. For example, if the number of video units sampled from one steady scene is 100, the number of video units is 100. If i is larger than 100, i = i + 1 is set and the process returns to step S302.

  On the other hand, if j does not exceed the number of video units in the i-th scene, the feature vector of the j-th video unit in the i-th scene is calculated with reference to the video DB 11. The calculated feature vector is recorded in the feature vector RAM 17 (S306).

  Next, when a feature quantity vector is calculated and recorded in the feature quantity vector RAM 17, j = j + 1 is set (S307), and the process returns to the step of S304.

  Extracting from the existing digest video by repeating the operations of S302 to S308 described above until j exceeds the number of video units of the i-th scene and i exceeds the number of scenes recorded in the specific scene information RAM 15. A feature quantity vector for each video unit constituting the steady scene is calculated.

  Next, a description will be given of a method in which learning data is created from the feature amount vector calculated from the steady scene and the discriminator learns this learning data in step S103.

  FIG. 10 is a flowchart showing a method of creating the discriminator fi by learning the discriminant surface 181 as shown in FIG. 5A in the discriminator learning unit 18. In the following description, the number of stationary scenes recorded in the feature vector RAM 17 is M, and the number of video units of the j-th stationary scene is Nj, as described above. Here, the feature quantity vector of the i-th video unit among the unit videos constituting the j-th stationary scene is assumed to be v (j, i).

  As shown in FIG. 10, the method of learning the discriminant plane 181 from the learning data recorded in the feature quantity vector RAM 17 and constructing the discriminator fi first sets i = 1 (S401). It is determined whether or not the number of video units Nj (j = 1 to M) of the stationary scene recorded in the vector RAM 17 is exceeded (S402). If so, the feature quantity vector (learning data) extraction operation is terminated.

  On the other hand, if i does not exceed the number of video units Nj (j = 1 to M) of the stationary scene recorded in the feature vector RAM 17, all of the M stationary scenes recorded in the feature vector RAM 17 are used. , M feature quantity vectors v (j, i) (j = 1 to M) calculated from the i-th video unit are collected. Then, the discriminating surface 181 is learned using the collected M feature quantity vectors as learning data, and the discriminator fi is created (S403).

  Next, i = i + 1 is set (S404), and the next unit video discriminator fi + 1 is moved to learning.

  The operations of S402 to S404 are repeated until i exceeds the number of video units Nj (j = 1 to M) of the stationary scene recorded in the feature vector RAM 17, whereby each of the Nj discriminators f is changed. learn.

  Next, a description will be given of a method of extracting an identification target scene from the extraction target digest video recorded in the digest information DB 12 in step S104 and calculating a feature vector of this scene.

  FIG. 11 is a flowchart illustrating a method for extracting an identification target scene from an extraction target digest video.

  As shown in FIG. 11, in the extraction of a scene to be identified, first, when an extraction target digest video is designated, i = 1 is set (S501), and this i is included in the extraction target digest video recorded in the digest information DB12. It is determined whether the number of scenes has been exceeded (S502). If so, the extraction of the scene to be identified is terminated.

  On the contrary, if i does not exceed the number of scenes included in the extraction target digest video, the i-th scene of the extraction target digest video is referred to, and the i-th scene is the same as the scene label specified in step S101. If it has the designated scene label, the scene is recorded in the specific scene information RAM 15 with the scene as an identification target scene (S503). The scene information recorded in the specific scene information RAM 15 is given a new consecutive scene number in the order of recording.

  Next, the feature quantity extraction unit 16 calculates the feature quantity vector of the identification target scene from the scene information of the identification target scene recorded in the specific scene information RAM 15, and records the result in the feature quantity vector RAM 17 (S504). Here, the feature amount vector of the identification target scene is calculated in the same manner as the operation shown in FIG.

  Next, i = i + 1 is set, and the next scene is referred to (S505).

  By repeating the operations of S502 to S505 until i exceeds the number of scenes included in the extraction target digest video, the identification target scene is extracted from the extraction target digest video, and the feature vector of this scene is calculated. .

  Next, a method for extracting an unsteady scene from the scene to be identified in step S105 will be described.

  FIG. 12 is a flowchart showing a method for extracting a non-stationary scene from a scene to be identified.

  As shown in FIG. 12, in extracting a non-stationary scene, first, i = 1 is set (S601), and i is the number N of feature quantity vectors of the identification target scene, that is, the number of video units constituting the identification target scene. It is determined whether or not it exceeds (S602). If so, the detection of the unsteady scene is terminated.

  On the other hand, if i does not exceed the number N of video units constituting the classification target scene, the i-th feature quantity vector vi of the classification target scene recorded in the feature quantity vector RAM 17 is recorded in the discriminator DB 20. The discriminator fi is applied (S603) to perform discrimination.

  Here, as shown in FIGS. 6A to 6C, the i-th feature quantity vector vi is plotted in the feature quantity space and compared with the learned discrimination plane 181 of the classifier fi. The identification surface 181 is learned from each feature quantity vector of a plurality of stationary scenes. As a result of the comparison, if the i-th feature vector vi of the identification target scene is out of the identification plane, the identification target scene is identified as a non-stationary scene (S604), and the scene of the identification target scene is stored in the non-stationary scene information DB 21. Information is recorded (S605).

  On the other hand, if the i-th feature vector vi is within the discrimination plane of the discriminator fi, i = i + 1 is set, and the next feature vector is identified (S606).

  By repeating the operations of S602 to S606 until i exceeds the number N of feature vectors, a non-stationary scene is extracted from the scene to be identified. On the other hand, as a result of identification, if the feature quantity vector vi of the identification target scene is applied to the discriminator fi and is not identified as an unsteady scene, the identification target scene is identified as a stationary scene.

  In the above-described identification method, if it is determined that the feature vector is outside the identification plane, which is learning data learned by the classifier, the scene is processed as an unsteady scene. However, this determination may be processed as an unsteady scene when it is determined that the image is outside the identification plane a plurality of times in succession. That is, if it is determined at least once that the feature vector is outside the discrimination surface, which is learning data learned by the discriminator, it is processed as an unsteady scene.

  In the case described above, the number of feature vectors and the number of classifiers are the same. However, when the number of feature vectors is larger than the number of classifiers, feature vectors corresponding to the number of classifiers are used. By determining, it may be determined whether the scene is stationary or non-stationary, and when it is recognized that the number of feature vectors is larger than the number of classifiers, this scene is determined to be an unsteady scene. Also good. This determination is particularly because there is a high possibility that some abnormality has occurred in the movement of the camera when the number of feature vectors is larger than the number of classifiers.

  As described above, when the unsteady scene is extracted from the identification target scene and the scene information of the unsteady scene is recorded in the unsteady scene information DB 21, the unsteady scene is displayed on the result display unit 22 in accordance with the step of S106. The

  According to the video processing apparatus and method thereof according to the first embodiment described above, the learning unit causes the classifier to learn the feature quantity vector of the stationary scene extracted from the existing digest video. Then, in the unsteady scene extracting means, only the unsteady scene is extracted from the classification target scene extracted from the extraction target digest video by using this classifier. Finally, the extracted unsteady scene is displayed. Therefore, it is possible to create an efficient digest video in which only unsteady scenes that are worth viewing are extracted from the digest video to be extracted.

(Second Embodiment)
Next, as another embodiment of the present invention, a video processing apparatus according to a second embodiment will be described with reference to the drawings.

FIG. 13 is a block diagram illustrating a configuration of a video processing apparatus according to the second embodiment. As shown in FIG. 13, the video processing apparatus according to the second embodiment
The point which has the digest production | generation part 23 connected to image | video DB11 and digest information DB12 differs from the image processing apparatus which concerns on 1st Embodiment.

  FIG. 14 is a flowchart showing the operation of the video processing apparatus according to the second embodiment. Regarding the operation of the video processing apparatus according to the second embodiment, for example, a digest video is created from the video recorded in the video DB 11 by the above-described conventional digest video creation technique, and the created digest video is shown in FIG. 3B. Except for storing in the digest information DB 12 in the form of a table (S700), the operation is the same as the operation of the video processing apparatus according to the first embodiment, and a detailed description thereof will be omitted.

  As described above, even the video processing apparatus according to the second embodiment can obtain the same effects as those of the first embodiment.

(Third embodiment)
Next, as another embodiment of the present invention, a video processing apparatus according to a third embodiment will be described with reference to the drawings.

  FIG. 15 is a block diagram illustrating a configuration of a video processing apparatus according to the third embodiment. As shown in FIG. 15, the video processing apparatus according to the third embodiment can obtain additional information added to the digest video created by the digest creation unit 23 from the outside through a network such as the Internet 24, for example. This is different from the second embodiment.

  In other words, the digest information DB 12 is connected with an additional information adding unit 25 for adding additional information to the digest video. This additional information is stored in the additional information DB 26 connected to the additional information adding unit 25, and can be obtained from this DB 26. The additional information DB 26 is connected to a network such as the Internet 24 via an additional information acquisition unit 27, and the additional information stored in the additional information DB 26 can be acquired from the outside via the Internet 24.

  FIG. 16 is a flowchart showing the operation of the video processing apparatus according to the third embodiment described above. Regarding the operation of the video processing apparatus according to the third embodiment, the results from the step of creating a digest video by the digest creation unit (S800) and the step of extracting the steady scene from the digest video created in S800 (S801) Since the process up to the step of displaying (S806) is the same as in the second embodiment, detailed description thereof is omitted. Here, the step of adding additional information to the scene information of the scene recorded in the digest information DB 12 will be described.

  First, when a digest video is created by the digest creation unit 23 (S800), the additional information obtaining unit 27 obtains additional information through the Internet 24 and stores it in the additional information DB 26 (S8001). For example, a score such as “2go” or “3 lost” is obtained from score information of baseball on the Internet.

  Next, the additional information adding unit 25 adds the additional information obtained through the Internet 24 and stored in the additional information DB 26 to the scene information of the scene recorded in the digest information DB 12 (S8002). For example, score information such as “2go” or “3 lost” is added as additional information to the scene information of each baseball defensive scene.

  After adding the additional information to the digest video by the steps of S8001 and S8002 described above, the process is the same as that of the first embodiment.

  Thus, even the video processing apparatus according to the third embodiment can obtain the same effects as those of the first embodiment.

  The video processing apparatus according to the embodiment of the present invention has been described above. The video processing method according to each of the first, second, and third embodiments includes a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), and a semiconductor memory as programs that can be executed by a computer. It can be stored and distributed in a recording medium.

  Further, the present invention is not limited to the above-described embodiment as it is, and can be embodied by freely modifying and deleting components and the like without departing from the spirit of the present invention.

  DESCRIPTION OF SYMBOLS 11 ... Video | video DB, 12 ... Digest information DB, 13 ... Specific scene extraction part, 14 ... Identification object scene extraction part, 15 ... Specific scene information RAM, 16 ... Feature-value extraction ,... Feature quantity vector RAM, 18... Classifier learning section, 181... Discriminating plane, 182... Set of feature quantity vectors, 19. Discriminator DB, 21 ... Unsteady scene information DB, 22 ... Result display unit, 23 ... Digest creation unit, 24 ... Internet, 25 ... Additional information adding unit, 26 ... Additional information DB, 27... Additional information acquisition unit.

Claims (5)

A video database for storing video content information;
A digest information database for storing digest information for specifying a digest video composed of a specific scene included in the video content;
A specific scene extraction unit for extracting scene information for specifying a plurality of stationary scenes composed of a plurality of frame images from the digest information stored in the digest information database;
The plurality of stationary scenes corresponding to the scene information extracted by the specific scene extracting unit are extracted from the video database, and feature amounts are obtained for each frame image of a predetermined time interval constituting each of the plurality of stationary scenes. A feature extraction unit for extracting vectors;
A discriminator learning unit that creates a discriminator based on a feature vector set for each frame image corresponding to the plurality of stationary scenes extracted by the feature amount extraction unit;
An identification target scene extraction unit for extracting scene information for identifying an identification target scene composed of a plurality of frame images from the extraction target digest information stored in the digest information database;
The feature amount extraction for extracting the identification target scene corresponding to the scene information extracted by the identification target scene extraction unit from the video database, and extracting a feature vector for each frame image constituting the identification target scene. And
A non-stationary scene detection unit for identifying whether or not the scene to be identified is a non-stationary scene by supplying the feature vector extracted by the feature amount extraction unit to the classifier for comparison;
A result display unit for displaying the unsteady scene identified by the unsteady scene detection unit;
A video processing apparatus comprising:   The video processing according to claim 1, further comprising: a digest creation unit that is connected to the video database and the digest information database and creates the digest video from the video content recorded in the video database. apparatus.   The additional information adding means for acquiring additional information via a network and adding the acquired additional information to the digest information database for storing the digest information. Video processing equipment.   The means for discriminating a non-stationary scene by comparing the classifier and the feature quantity vector of the classification target scene is characterized in that at least one of the feature quantity vectors of the classification target scene corresponds to the feature quantity vector corresponding to the classifier. The video processing apparatus according to claim 1, wherein the scene to be identified is identified as the non-stationary scene when it is outside a range of a region formed in space. In the specific scene extraction unit, scene information for specifying a plurality of stationary scenes composed of a plurality of frame images is extracted from the digest information for specifying a digest video composed of specific scenes included in the video content stored in the digest information database. And steps to
In the feature amount extraction unit, the plurality of stationary scenes corresponding to the scene information extracted in the step are extracted from the video database, and each frame image of a predetermined time interval constituting each of the plurality of stationary scenes is extracted. Extracting a feature vector into
In the discriminator learning unit, creating a discriminator from a feature vector set for each frame image corresponding to the plurality of stationary scenes extracted by the step of extracting the feature vector;
A step of extracting scene information for identifying a scene to be identified composed of a plurality of frame images from the digest information to be extracted stored in the digest information database in the scene to be identified for identification;
In the feature amount extraction unit, the identification target scene corresponding to the scene information extracted by the step of extracting scene information specifying the identification target scene is extracted from the video database, and the identification target scene is configured Extracting a feature vector for each frame image;
In the unsteady scene extraction unit, identifying whether the identification target scene is an unsteady scene by extracting the feature quantity vector of the identification target scene; and
In the result display unit, displaying the unsteady scene identified by the step of identifying whether or not the unsteady scene; and
A video processing method comprising:

Images