JP3488626B2 - Video division method, apparatus and recording medium recording video division program - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention analyzes a background sound of sound information contained in a video and records the video splitting method, device and video splitting program for splitting the video based on the similarity of their feature amounts. Regarding the medium.

[0002]

2. Description of the Related Art Some methods of dividing an image mainly use image information. For example, there is a method of detecting a cut point at which a camera is switched and dividing the image into shots.

[0003]

As an application example of a technique for dividing image information by using a method of detecting a cut point, a leading still image of a shot is a typical still image (representative image) representing the shot. There is a video expression method that allows you to view the contents of the video by displaying them side by side, but since there are frequent cut points, the number of representative images is long when a long video is targeted. There was a problem that it would increase too much. In order to reduce the number of representative images, it is necessary to divide the video more roughly.

From the viewpoint of image production, a set of shots is a scene, and it is possible to divide the image by capturing the scene, but a normal scene is a connection of the same scenes, and it is difficult to automatically divide it. Met.

An object of the present invention is to roughly divide an image by utilizing the characteristic that background sounds are likely to be similar in the same scene.

[0006]

In order to achieve the above object, in the present invention, an image is inputted, the inputted image is accumulated, and music and voice are detected from sound information of the image,
For sections of sound information that do not include music and voice
Frequency average of the sound information and time-averaged spectrum
The extracted feature quantity is extracted, and the mutual phase of the extracted feature quantity
The degree of similarity obtained by calculating the relationship is calculated, and an image including a section with a high degree of similarity and sound information sandwiched between the sections is combined into one unit.
The video is roughly divided by dividing it as a segment by regarding it as a certain section .

[0007]

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a video division device according to an embodiment of the present invention. The video division device of the present embodiment includes a video input unit 101 for inputting a video, a video storage unit 102 for storing a video, a music detection unit 103 for detecting music, a voice detection unit 104 for detecting voice, and a music detection unit 104. And for the section that does not include voice,
A feature extraction unit 105 that extracts a feature amount, and a video division unit that calculates a similarity between the extracted feature amounts and divides a video including a section with high similarity and sound information sandwiched between the sections as one segment. It is composed of 106.

FIG. 2 is a flow chart showing the flow of processing of the video division apparatus according to the embodiment of the present invention. Even when the present invention is realized by software, the same processing flow is used. The processing of one loop is performed for a video segment of about 1 second.

First, video is stored in the video storage processing 201, and music detection processing 202 is performed on the sound information of the video.
At decision 203, it is decided whether or not it is music, and if it is music, the process jumps to decision 208. If not music,
The voice detection process 204 is performed. In judgment 205, it is judged whether or not it is a voice, and if it is a voice, the process jumps to judgment 208. For music detection, the peak of the frequency spectrum of the sound information is temporally stable in the frequency direction, and for speech detection, a comb filter method is used (Minami et al. Video indexing by analysis ",
IEICE General Conference, D-12-64, 199
7) etc. are effective.

[0011]

If it is not a voice, the feature extraction processing 206 is performed as a background sound that does not include music and voice during the period, that is, as a segment corresponding to the background sound. In the feature extraction processing 206, frequency analysis is performed on the sound information,
Obtain the long-term average spectrum. The long-term average spectrum is a temporal average value of the power of the spectrum at each frequency.

Next, in the image division processing 207, the correlation between the long-time average spectrum calculated one loop before and the current long-time average spectrum is obtained, and if the correlation is high, it is considered that the scene is the same, and labeling is performed. To do. The music or voice segments existing in the two segments for which the correlation has been obtained are also labeled as the same scene. The label information, together with the segment time information, is stored in the video storage unit 102.
Stored in.

Although the video division has been described above, the mode of the division can be held in the form of a program that can be executed by the data processing apparatus, and the present invention also includes a recording medium recording the program.

[0015]

(1) The inventions of claims 1, 2 and 3 are as follows:
Input the video, store the input video, detect music and voice from the sound information of the video, and detect the frequency of the sound information in the section that does not include the music and voice in the sound information.
Extract the feature quantity for which the time-averaged spectrum is obtained by numerical analysis , calculate the similarity degree by calculating the mutual correlation of the extracted feature quantity, and include the section with high similarity degree and the sound information sandwiched between the sections. It is possible to divide an image into segments by treating the image as one united section, and to roughly divide the image.

[0016]

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a video division device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing flow of the video division device according to the embodiment of the present invention and a processing flow when the present invention is realized by software.

[Explanation of symbols]

101 Video input section 102 video storage 103 music detector 104 voice detector 105 Feature Extraction Unit 106 video division unit 201 Image storage processing 202 Music detection processing 203 Music determination processing 204 voice detection processing 205 voice determination processing 206 Feature extraction process 207 Video division processing 208 Video end determination processing

Continuation of the front page (56) References JP-A-9-214879 (JP, A) JP-A-8-95596 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 5 / 76-5/956 G10L 3/00

Claims (3)

(57) [Claims] 1. A method of dividing a given image according to a scene, the image inputting step of inputting the image, the image accumulating step of accumulating the image, and the music detecting music from the sound information in the image. A detection step, a voice detection step of detecting voice from sound information, and a section of the sound information that does not include music and voice,
Frequency average of the sound information and time-averaged spectrum
Image including a feature extraction step of extracting a feature amount obtained, video and sound information sandwiched between the interval of the extracted feature quantity calculated to the similarity degree of similarity calculated correlation mutual high section of And a video segmentation step of segmenting the video into a plurality of segments by regarding them as one coherent section and grouping the segments into segments, and a video segmentation method comprising: 2. An apparatus for dividing a given video image according to a scene, the video input unit for inputting the video image, the video storage unit for accumulating the video image, and the music for detecting music from the sound information in the video image. A detection unit, a voice detection unit that detects voice from sound information, and a section of the sound information that does not include music and voice,
Frequency average of the sound information and time-averaged spectrum
Image including a feature extracting section for extracting a feature amount obtained, video and sound information sandwiched between the interval of the extracted feature quantity calculated to the similarity degree of similarity calculated correlation mutual high section of And a video dividing unit that divides the video into a plurality of segments by regarding them as one coherent section, and dividing the video into a plurality of segments. 3. A recording medium in which a program for dividing a given video corresponding to a scene is recorded, the video input processing for inputting the video, the video storage processing for storing the video, and the sound information in the video. Music detection processing that detects music, voice detection processing that detects voice from sound information, and for the section of the sound information that does not include music and voice,
Frequency average of the sound information and time-averaged spectrum
Image including a feature extraction process of extracting a feature amount obtained, video and sound information sandwiched between the interval of the extracted feature quantity calculated to the similarity degree of similarity calculated correlation mutual high section of Is regarded as one cohesive section, and is grouped as a segment to divide the video into a plurality of segments, and a video division program for causing a computer to execute is recorded. Recording medium.