JP6857983B2 - Metadata generation system - Google Patents

Description

The present invention relates to a system that generates metadata, and more particularly to a system that generates metadata related to a television broadcast program or an Internet-distributed moving image.

Traditionally, the importance of metadata related to television broadcast programs or Internet-distributed videos has increased. Metadata is not the data itself, but the information associated with that data. The date and time when the data was created, the creator, the data format, the title, the annotation, etc. can be considered. This is important information for efficient management and retrieval of data.
For example, the patent applicant has in the past a recording means for recording a TV broadcast program broadcast by a TV broadcasting station, and metadata for storing metadata summarizing the program contents corresponding to the video recorded by the recording means. A storage means and a display means capable of displaying the metadata on the screen are provided, and the user visually recognizes the metadata displayed on the screen and appropriately selects the metadata to obtain an image corresponding to the metadata. We have applied for and obtained a patent for an invention relating to a video system configured to be displayed on a screen so that it can be visually recognized (Patent Document 1).

However, metadata about television broadcast programs is generally created by humans, which is time consuming and costly. Further, since the metadata once created is generally used only for the program, it is difficult to repeatedly use the same information, and there is a problem that the efficiency is not good.
Since the above situation exists not only for TV broadcast programs but also for Internet-distributed videos that have been rapidly put into practical use, there is a similar problem with metadata related to Internet-distributed videos.

Patent No. 4227866

The present invention is for solving the above-mentioned conventional problems, and the problem is to create metadata about a television broadcast program or an Internet-distributed video in a short time and reduce human costs. It is to provide a system that can do it.

In order to solve the above problem, in the invention according to claim 1, a recording means having a recording file for recording an image and a character for acquiring character information displayed on the image recorded in the recording file. The information acquisition means, the character information writing means for aggregating and writing the character information acquired by the character information acquisition means, and the character information written by the character information writing means in the recording file. It is provided with a metadata storage means for storing the metadata of the recorded video in a metadata storage file, and the character information acquisition means performs image analysis on the video recorded in the recording file and characters from the video. Character information extraction means for extracting information and
The person, logo, belongings of the person or the facial expression of the person included in the video are collated with the person information, logo information, physical information or facial expression information, and the person, logo, belongings of the person or the facial expression information included in the video are collated. A video recognition information extraction means that extracts the facial expression of the person as character information,
A voice information extraction means that performs voice analysis on the voice recorded together with the video recorded in the recording file and extracts character information from the voice, the character information extraction means, the video recognition information extraction means, and Each of the voice information extracting means has a compound information collating means for collating the extracted character information with each other.

Here, the character information is information on words and sentences displayed on the video and related to the video, and is a concept including, for example, a character string of the telop displayed on the video.
Therefore, when an image is recorded in the recorded file by the recording means, the character information displayed in the image recorded in the recording file is acquired by the character information acquisition means, and the character information text is obtained. The acquired character information is documented by the conversion means, and the documented character information is stored in the metadata storage file as metadata of the video by the metadata storage means.
When a video is recorded in the recorded file by the recording means, the text information extraction means extracts the text information from the video by image-analyzing the video recorded in the recording file. Then, the image recognition information extraction means collates the person, logo, belongings of the person or the facial expression of the person with the person information, logo information, object information or facial expression information included in the image, and includes the person, the logo information, the object information or the facial expression information. The person, logo, belongings of the person, or facial expression of the person are extracted as character information, and the voice recorded together with the video recorded in the recording file is voice-analyzed by the voice information extraction means. Character information is extracted from the voice, and the extracted character information is collated with each other by the compound information collating means, the character information extracting means, the video recognition information extracting means, and the voice information extracting means, respectively. ..

In the invention according to claim 2, the character information acquisition means includes a character information extraction means that performs image analysis on a video recorded in the recording file and extracts character information from the video, and the character information extraction means. It is characterized by having a dictionary collating means for collating the character information extracted by the character information extracting means with a dictionary file.
Here, the dictionary file includes dictionary data having characters and idioms related to each country's language so that they can be collated.
Therefore, the character information is extracted from the video by image-analyzing the video recorded in the recording file by the character information extracting means, and the character information extracted by the dictionary collating means is the dictionary file. Is matched with.

In the invention described in claim 3, the character information extraction means includes an image that has been image-analyzed and an image.
It is characterized by having an image analysis means for collating with an image analysis storage file having character information extracted from the image-analyzed video and performing image analysis.
Here, the image-analyzed video means the video that has been image-analyzed so far, and the character information extracted from the image-analyzed video is correctly extracted from the video as a result of the image analysis. Means character information.
Therefore, the image analysis means collates the image recorded in the recorded file with the image analysis storage file having the image-analyzed image and the character information extracted from the image-analyzed image. Is image-analyzed.

In the invention according to claim 4, the character information extracting means obtains the image analysis storage file based on the image analyzed by the image analysis means and the character information extracted from the image. It is characterized by further having an image analysis learning means for modifying.
Here, modification is a concept including addition and deletion.
Therefore, the image analysis learning means modifies the image analysis storage file based on the image analyzed by the image analysis means and the character information extracted from the image.

In the invention according to claim 5, the character information acquisition means includes a character information extraction means that performs image analysis on a video recorded in the recording file and extracts character information from the video, and the character information extraction means. It is characterized by having an Internet collation means that searches the character information extracted by the character information extraction means on the Internet and collates it with the acquired information.
Here, the information obtained by searching on the Internet is information obtained from sites such as major newspapers, local newspapers, news distribution companies, TV companies, news specialized sites, news summary sites, and other general websites. It is a concept that includes information on glossary obtained from online dictionaries and the like.
Therefore, the character information is extracted from the video by image analysis of the video recorded in the recording file by the character information extracting means, and the character information extracted by the Internet collation means is searched by the Internet. It is collated with the acquired information.

The invention according to claim 6 is characterized in that the character information acquisition means further includes a dictionary update means for modifying the dictionary file based on the character information extracted by the character information extraction means. To do.
Here, modification is a concept including addition and deletion.
Therefore, the dictionary updating means modifies the dictionary file based on the character information extracted by the character information extracting means.

In the invention according to claim 7, the dictionary file has dictionary data and a frequency parameter of the dictionary data, and the dictionary collating means gives priority to dictionary data having a large frequency parameter as a collation target. It is characterized by selecting the target.
Here, the frequency parameter is a parameter indicating how often words, idioms, etc. included in the dictionary data are displayed in the video. Specifically, the frequency parameter is updated every time the dictionary collating means collates the character information extracted from the video by the character information extracting means with the dictionary file.
Therefore, the dictionary collating means preferentially selects the dictionary data having a large frequency parameter as a collation target, and the selected dictionary data is collated with the character information extracted by the character information extracting means. To.

In the invention according to claim 8, the character information writing means refers to the metadata storage file, and the metadata related to the character information acquired by the character information acquisition means is used as the character information. It is characterized by being used for writing.
Therefore, when the character information documenting means aggregates and documents the character information acquired by the character information acquisition means, the character information documenting means has been created in relation to the character information by referring to the metadata storage file. The metadata of the above can be used for writing the character information.

The invention according to claim 9 is characterized in that the character information writing means acquires electronic program guide data of a video recorded in the recording file and uses it for writing the character information. To do.
Here, the electronic program guide data is data including information such as the broadcast date / time, distribution date / time, genre, title, performer, etc. of the broadcast program video broadcast by the television broadcasting station or the video video distributed via the Internet. ..
Therefore, when the character information documenting means aggregates and documents the character information acquired by the character information acquisition means, the character information documenting means acquires the electronic program guide data of the video and documents the character information. Can be used for.

In the invention according to claim 10, the character information acquisition means includes a person, a logo, a property of the person or a facial expression of the person, and personal information, logo information, physical information or facial expression information. It is characterized by having a video recognition information extraction means for extracting a person, a logo, a property of the person, or a facial expression of the person included in the video as character information.
Therefore, the image recognition information extracting means collates the person, logo, belongings of the person, or the facial expression of the person with the person information, logo information, object information, or facial expression information, and includes the person, logo, object information, or facial expression information. The person, the logo, the belongings of the person, or the facial expression of the person are extracted as character information.

In the invention according to claim 11, the person information, the logo information, the object information, or the facial expression information is based on an image-analyzed image and character information extracted from the image-analyzed image. It is characterized by being configured.
Here, the image-analyzed video means the video that has been image-analyzed so far, and the character information extracted from the image-analyzed video is correctly extracted from the video as a result of the image analysis. Means character information.
Therefore, the person information and the logo information in which the video recorded in the recording file by the video recognition information extracting means has an image-analyzed video and character information extracted from the image-analyzed video. By collating with the object information or the facial expression information, the person, the logo, the possession of the person, or the facial expression of the person included in the video is extracted as character information.

In the invention according to claim 12, the character information acquisition means obtains the person information based on the image analyzed by the image recognition information extraction means and the character information extracted from the image. It is characterized by further having an image recognition learning means for modifying the logo information, the object information, or the facial expression information.
Here, modification is a concept including addition and deletion.
Therefore, the image recognition learning means has image-analyzed the person information, the logo information, the object information, or the facial expression information by the image recognition information extraction means, and the character information extracted from the image. It will be corrected based on.

In the invention according to claim 13, the character information acquisition means performs voice analysis on the voice recorded together with the video recorded in the recording file, and extracts the character information from the voice. It is characterized by having an extraction means and a dictionary collation means for collating the character information extracted by the voice information extraction means with a dictionary file.
Therefore, the voice information extraction means extracts character information from the voice by voice analysis of the voice recorded together with the video recorded in the recording file, and the extracted voice is extracted by the dictionary collation means. The character information is collated with the dictionary file.

In the invention described in claim 14, the voice information extracting means collates the voice analyzed voice with a voice analysis storage file having character information extracted from the voice analyzed voice, and the voice is voiced. It is characterized by having a voice analysis means for analysis.
Here, the voice that has been voice-analyzed means the voice that has been voice-analyzed so far, and the character information extracted from the voice that has been voice-analyzed is correctly extracted from the voice as a result of voice analysis. Means textual information.
Therefore, the voice recorded by the voice analysis means together with the video recorded in the recording file has the voice analyzed voice and the character information extracted from the voice analyzed voice. Voice analysis is performed by collating with the file.

In the invention according to claim 15, the character information acquisition means obtains the voice analysis storage file based on the voice analyzed by the voice analysis means and the character information extracted from the voice. It is characterized by further having a voice analysis learning means for modifying.

Here, modification is a concept including addition and deletion.
Therefore, the voice analysis learning means modifies the voice analysis storage file based on the voice analyzed by the voice analysis means and the character information extracted from the voice.

The invention according to claim 16 is characterized in that the video is a broadcast program video broadcast by a television broadcasting station.

The invention according to claim 17 is characterized in that the video is a moving image distributed via the Internet.

In the metadata generation system according to claims 1 to 17 , when a video is recorded in the recording file by the recording means, the character information acquisition means records the video in the recording file. The character information displayed on the video is acquired, the acquired character information is documented by the character information documenting means, and the character information documented by the metadata storage means is the metadata of the image. Since it is stored in the metadata storage file, the metadata of the video can be accurately and automatically created from the character information which is displayed in the video and is information on words and sentences related to the video.
As a result, it is possible to provide a system capable of creating metadata about a television broadcast program or an Internet-distributed video in a short time and reducing human costs.

Further, in the metadata generation system according to claim 1, when a video is recorded in the recording file by the recording means, the character information extraction means records the video in the recording file. Character information is extracted from the video by image analysis of the video, and the person, logo, belongings of the person or facial expression of the person, person information, logo information included in the video are extracted by the video recognition information extraction means. , Object information or facial expression information is collated, and the person, logo, personal belongings of the person or facial expression of the person included in the video are extracted as character information, and recorded in the recording file by the audio information extraction means. Character information is extracted from the voice by analyzing the voice recorded together with the video, and the character information extracting means, the video recognition information extracting means, and the voice information extraction by the composite information collating means. By means, the extracted character information is collated with each other.
Therefore, the character information can be efficiently extracted from image analysis, voice analysis, and the person, logo, belongings of the person, or facial expression of the person included in the video.
Further, since the extracted character information is collated with each other by the compound information collating means, the character information extracting means, the video recognition information extracting means, and the audio information extracting means, for example, the character information. Characters and words that are erroneously recognized by the extraction means or cannot be completely recognized can be corrected based on the character information extracted by the voice information extraction means.
As a result, it is possible to provide a system capable of automatically generating metadata related to a television broadcast program or an Internet-distributed moving image more accurately and efficiently.

In the metadata generation system according to claim 2, the character information extraction means extracts the character information from the video by image-analyzing the video recorded in the recording file, and collates the dictionary. By the means, the extracted character information is collated with the dictionary file.
Therefore, the character information can be efficiently extracted from the video by image analysis, and the character information is collated with the dictionary file, so that, for example, it cannot be erroneously recognized or completely recognized by image analysis. It is possible to improve the accuracy of the character information by modifying the characters and words that have been lost based on the dictionary file.

In the metadata generation system according to claim 3, the video recorded in the recording file by the image analysis means is an image-analyzed video and characters extracted from the image-analyzed video. Image analysis is performed by collating with an image analysis storage file having information.
Therefore, it is possible to effectively perform image analysis using the image analysis results accumulated from the past, and as a result, it is possible to accurately create the metadata of the video in a short time.

In the metadata generation system according to claim 4, the image analysis storage file is image-analyzed by the image analysis means by the image analysis learning means, and the character information extracted from the image is used. It will be corrected based on.
Therefore, the result of the image analysis performed this time can be added to the image analysis storage file, and the erroneous information contained in the image analysis storage file can be deleted based on the result of the image analysis performed this time. , The image analysis storage file can be updated and always used in the latest state.

Further, in the metadata generation system according to claim 5, character information is extracted from the video by image analysis of the video recorded in the recording file by the character information extraction means, and the character information is extracted. The extracted character information is collated with the information searched and acquired by the Internet by the Internet collation means.
Therefore, the character information can be efficiently extracted from the video by image analysis, and the character information is collated with the information searched and acquired by the Internet, so that, for example, misrecognition or complete recognition is performed by image analysis. It is possible to improve the accuracy of the character information by correcting the characters and words that could not be performed based on the information obtained by searching on the Internet.

In the metadata generation system according to claim 6, the dictionary update means modifies the dictionary file based on the character information extracted by the character information extraction means, and thus obtains from the character information. Information such as new words and sentences can be added to the dictionary file, and erroneous information contained in the dictionary file can be deleted based on the character information, and as a result, the dictionary file is updated. And you can always use it in the latest state.
In the metadata generation system according to claim 7, the dictionary collation means is used in advance.
The dictionary data having a large writing frequency parameter is preferentially selected as a collation target, and the selected dictionary data is collated with the character information extracted by the character information extraction means. Therefore, for example, the dictionary file When a plurality of characters or words similar to each other exist in the dictionary, the dictionary data having a large frequency parameter is preferentially selected and is a collation target.
As a result, it is possible to make corrections based on the dictionary data having a large frequency parameter, and the accuracy of the character information can be improved more efficiently.

In the metadata generation system according to claim 8, when the character information documenting means aggregates and documents the character information acquired by the character information acquisition means, the metadata storage file The created metadata related to the character information can be used for writing the character information, and as a result, the metadata related to the television broadcast program or the Internet-distributed video can be accurately and more efficiently. Can be automatically generated.

In the metadata generation system according to claim 9, the character information documenting means is an electronic program of the video when the character information acquired by the character information acquisition means is aggregated and documented. It is possible to acquire the table data and use the electronic program table data including information such as broadcast date / time, distribution date / time, genre, title, performer, etc. for writing the character information. As a result, metadata related to a television broadcast program or an Internet-distributed moving image can be automatically generated with high accuracy and more efficiently.

In the metadata generation system according to claim 10, the person, logo, belongings of the person or facial expression of the person, person information, logo information, and objects included in the image are used by the image recognition information extraction means. Since the information or facial expression information is collated and the person, logo, belongings of the person or facial expression of the person included in the video are extracted as character information, the person, logo, belongings of the person or the person's belongings included in the video are extracted. The metadata of the video can be created from the facial expression of the person.

In the metadata generation system according to claim 11, the image recorded in the recording file is extracted from the image-analyzed image and the image-analyzed image by the image recognition information extraction means. By collating with the person information, the logo information, the object information, or the facial expression information having the character information, the person, the logo, the person's belongings, or the facial expression of the person included in the video can be used as the character information. Be extracted.
Therefore, it is possible to effectively perform image analysis using the image analysis results accumulated from the past, and as a result, it is possible to accurately create the metadata of the video in a short time.

In the metadata generation system according to claim 12, the person information, the logo information, the object information or the facial expression information are image-analyzed by the image recognition information extraction means by the image recognition learning means. It is corrected based on the video and the character information extracted from the video.
Therefore, the result of the image analysis performed this time may be added to the person information, the logo information, the object information or the facial expression information, or erroneous information included in the person information, the logo information, the object information or the facial expression information. Can be deleted based on the result of the image analysis performed this time, and as a result, the person information, the logo information, the object information, or the facial expression information can be updated and always used in the latest state. ..

In the metadata generation system according to claim 13, character information is obtained from the voice by analyzing the voice recorded together with the video recorded in the recording file by the voice information extraction means. The extracted character information is collated with the dictionary file by the dictionary collation means.
Therefore, the character information can be efficiently extracted from the voice recorded together with the video by voice analysis, and the character information is collated with the dictionary file. Therefore, for example, misrecognition or completeness can be achieved by voice analysis. Characters and words that could not be recognized can be corrected based on the dictionary file, and the accuracy of the character information can be improved.

In the invention described in claim 14, the voice recorded together with the video recorded in the recording file is extracted from the voice analyzed voice and the voice analyzed voice by the voice analysis means. The voice is analyzed by collating with the voice analysis storage file having the character information.
Therefore, it is possible to effectively perform voice analysis using the voice analysis results accumulated from the past, and as a result, it is possible to accurately create the metadata of the video in a short time.

In the invention according to claim 15, the voice analysis storage file is modified based on the voice analyzed by the voice analysis means and the character information extracted from the voice by the voice analysis learning means. Will be done.

Therefore, the result of the voice analysis performed this time can be added to the voice analysis storage file, and the erroneous information contained in the voice analysis storage file can be deleted based on the result of the voice analysis stored this time. , The voice analysis storage file can be updated and always used in the latest state.

FIG. 1 is a block diagram showing an embodiment of a metadata generation system according to the present invention. FIG. 2 is a flowchart showing a processing flow in the metadata generation system in one embodiment of the metadata generation system according to the present invention. 3A and 3B are schematic views showing a broadcast program image in an embodiment of a metadata generation system according to the present invention, and FIG. 3B is metadata generated from the broadcast program image of (a). ..

Hereinafter, the present invention will be described in detail based on the embodiments shown in the accompanying drawings.
(1) Configuration of Metadata Generation System 10 According to the Embodiment of the Present As shown in FIGS. 1 and 3, the metadata generation system 10 according to the embodiment of the present invention is a broadcast broadcast by the television broadcasting station 30. The recording means 12 having the recording file 11 for recording the program video V, the character information acquisition means 13 for acquiring the character information C from the video V recorded in the recording file 11, and the character information acquired by the character information acquisition means 13. The character information writing means 14 that aggregates C and makes it into a sentence, and the character information that is written by the character information writing means 14 are stored in the metadata storage file 15 as the metadata M of the video V recorded in the recording file 11. It is provided with a metadata storage means 16 for storing.

Further, as shown in FIGS. 1 and 3, the character information acquisition means 13 according to the present embodiment performs image analysis on the video V recorded in the recording file 11 and extracts the character information C from the video V. It has a character information extracting means 17 to be used, and a dictionary collating means 19 for collating the character information C extracted by the character information extracting means 17 with the dictionary file 18.
The character information extracting means 17 according to the present embodiment is an image analysis means 31 that extracts a character string by performing image analysis on the video V recorded in the recording file 11, and the extracted character string. It has a word analysis means 32 for extracting words included in the character string by performing morphological analysis.
Here, morphological analysis refers to morphological elements (roughly speaking) based on information such as the grammar of the target language and the part of speech of words called dictionaries, from text data (sentences) in natural language without notes of grammatical information. , The smallest unit that has meaning in the language), and the part of speech of each morpheme is discriminated. Specifically, it is possible to extract words such as "○ ×" (meeting name), "○ × open", "final", "advance", and "advance to the final" from the character string "○ × open final advance". it can.

As shown in FIG. 1, the image analysis means 31 according to the present embodiment collates the image-analyzed image with the image analysis storage file 35 having the character information extracted from the image-analyzed image. It is configured for image analysis.
Here, the image-analyzed video means the video that has been image-analyzed so far, and the character information extracted from the image-analyzed video is correctly extracted from the video as a result of the image analysis. Means character information.

Further, as shown in FIGS. 1 and 3, the character information extracting means 17 according to the present embodiment is based on the image V image-analyzed by the image analysis means 31 and the character information C extracted from the image V. Further, the image analysis learning means 36 for modifying the image analysis storage file 35 is provided.

Further, as shown in FIGS. 1 and 3, the character information acquisition means 13 according to the present embodiment searches the character information C extracted by the character information extraction means 17 on the Internet 20 and collates it with the acquired information. It has an Internet collation means 21.

Further, as shown in FIGS. 1 and 3, the character information acquisition means 13 according to the present embodiment is a dictionary update means for modifying the dictionary file 18 based on the character information C extracted by the character information extraction means 17. It also has 33.

Further, as shown in FIGS. 1 and 3, the dictionary file 18 according to the present embodiment includes dictionary data D and dictionary data D in which dictionary data having characters and compound words related to each country's language can be collated. The dictionary collation means 19 is configured to preferentially select dictionary data D having a large frequency parameter 34 as a collation target.

Further, as shown in FIGS. 1 and 3, the character information documenting means 14 according to the present embodiment refers to the metadata storage file 15 and is related to the character information C acquired by the character information acquisition means 13. It is configured to use the metadata M for writing the character information C.

Further, as shown in FIGS. 1 and 3, the character information writing means 14 according to the present embodiment acquires the electronic program guide data E of the video V recorded in the recording file 11, and the text of the character information C. It is configured to be used for conversion. The electronic program guide data E according to the present embodiment includes information such as the broadcast date / time, the distribution date / time, the genre, the title, and the performers of the broadcast program video V broadcast by the television broadcasting station 30.

Further, as shown in FIGS. 1 and 3, the character information acquisition means 13 according to the present embodiment includes a person P, a logo L, a person P's belongings B or a person P's facial expression F included in the image V, and a person. Video recognition information extraction means 22 that collates information, logo information, object information, or facial information, and extracts the person P, logo L, personal belongings B of person P, or facial expression F of person P included in the image V as character information C. have.

The person information, logo information, object information, or facial expression information according to the present embodiment is composed of an image-analyzed image and character information extracted from the image-analyzed image.
Here, the image-analyzed video means the video that has been image-analyzed so far, and the character information extracted from the image-analyzed video is correctly extracted from the video as a result of the image analysis. Means character information.

Further, as shown in FIGS. 1 and 3, the character information acquisition means 13 according to the present embodiment includes the image V image-analyzed by the image recognition information extraction unit 22 and the character information C extracted from the image V. It is characterized in that it further has an image recognition learning means 37 that corrects personal information, logo information, object information, or facial expression information based on the above.

Further, as shown in FIGS. 1 and 3, the character information acquisition means 13 according to the present embodiment performs voice analysis on the voice recorded together with the video V recorded in the recording file 11, and from the voice. It has a voice information extraction means 23 for extracting character information C.

As shown in FIG. 1, the voice information extracting means 23 according to the present embodiment collates the voice analyzed voice with the voice analysis storage file 38 having the character information extracted from the voice analyzed voice. It is characterized by having a voice analysis means 39 for voice analysis.
Here, the voice that has been voice-analyzed means the voice that has been voice-analyzed so far, and the character information extracted from the voice that has been voice-analyzed is correctly extracted from the voice as a result of voice analysis. Means textual information.

Further, as shown in FIGS. 1 and 3, the voice information extracting means 23 according to the present embodiment is based on the voice analyzed by the voice analysis means 39 and the character information extracted from the voice. It is characterized by further having a voice analysis learning means 40 for modifying the voice analysis storage file 38.

As shown in FIGS. 1 and 3, in the character information acquisition means 13 according to the present embodiment, the character information extraction means 17, the video recognition information extraction means 22, and the voice information extraction means 23 are used, respectively. The compound information collation means 24 for collating the extracted character information C with each other is provided.

As shown in FIGS. 1 and 3, the recording means 12 according to the present embodiment is a video of all broadcast programs broadcast from all broadcasting stations, for example, all terrestrial stations and satellite broadcasting broadcasting stations in Japan. Is a large-scale recording device having a hard disk-type storage device having a predetermined capacity so that the recording can be performed for a predetermined period, for example, one month.
In the present embodiment, the recording file 11 in the hard disk provided in the recording means 12 includes a program content 25 composed of a video V broadcast by the television broadcasting station 30, and a channel name 26 on which the program content 25 is broadcast. , Has information about the time code 27 of the program content 25.
In this case, the program content 25 is composed of a broadcast program unit, a corner unit constituting the broadcast program, or an article unit constituting the broadcast program.

Further, as shown in FIGS. 1 and 3, in the present embodiment, the metadata storage file 15 of the metadata storage means 16 contains the program content summary text data 28 and the channel name 29 on which the program content 25 is broadcast. And the time code 27 of the program content 25 are recorded, and all of them are data constituting the metadata M in the present embodiment.
The program content summary text data 28 is a textual summary of the contents of a television program broadcast by the television broadcasting station 30. Similar to the program content 25, the program content summary text data 28 is composed of a broadcast program unit, a corner unit constituting the broadcast program, or an article unit constituting the broadcast program.
In addition, the program content summary text data 28 can include nuance parameters. Here, the "nuance parameter" is a numerical value of the nuance (impression) of the site information in which a word or phrase corresponding to the search keyword appears by an automatic system such as artificial intelligence or a human judgment.
For example, if the program content is good (good), it is high (positive evaluation), if it is bad (bad), it is low (negative evaluation), and if it is neutral content (neutral) that only states the facts, it is 0. (Zero evaluation) can be set.

(2) Process flow of the metadata generation system 10 according to the present embodiment As shown in FIG. 2, the metadata generation system 10 according to the present embodiment performs processing according to the following steps. First, the recording means 12 records the broadcast program video V broadcast by the television broadcasting station 30 in the recording file 11 (S1).
At this time, the recording means 12 records images of all broadcast programs broadcast from all broadcasting stations, for example, all terrestrial stations and satellite broadcasting broadcasting stations in Japan, for a predetermined period, for example, one month. You can also do it.

Next, as shown in FIG. 2, the character information acquisition means 13 acquires the character information C displayed on the video V recorded in the recording file 11.
At this time, the character information extracting means 17 performs image analysis on the video V recorded in the recording file 11 and extracts the character information C from the video V (S2a).
In particular, as shown in FIGS. 1 and 3, in the character information extraction means 17 according to the present embodiment, the image analysis means 31 performs image analysis on the video V recorded in the recording file 11. The character string is extracted by the above, and the words included in the character string are extracted by performing morphological analysis on the character string extracted by the word analysis means 32.
Specifically, as shown in FIG. 3A, the image analysis means 31 performs image analysis on the video V of the program content 25 to perform "XX news" and "breaking news ○ △ player ○ × open". The character string "advance to the final" can be extracted.
Subsequently, by performing morphological analysis on these character strings extracted by the word analysis means 32, "XX" (program name, channel name), "news", "XX news", "breaking news", Words such as "○ △" (player name), "player", "○ △ player", "○ ×" (region name, tournament name), "open", "○ × open", "final", "advance" Can be extracted.
As shown in FIGS. 1 and 3, in the character information extracting means 17 according to the present embodiment, the image analysis means 31 has the image V recorded in the recording file 11 and the image-analyzed image. And the image analysis storage file 35 having the character information extracted from the image-analyzed video.
Image analysis is performed by collating with.

Further, as shown in FIG. 2, the image recognition information extracting means 22 includes the person P, the logo L, the possession B of the person P, or the facial expression F of the person P, and the person information, the logo information, the object information, or the person information. By collating with the facial expression information, the person P, the logo L, the possession B of the person P, or the facial expression F of the person P included in the image V is extracted as the character information C (S2b).
Specifically, as shown in FIG. 3A, the video recognition information extracting means 22 with respect to the person P, the logo L, the personal belongings B of the person P, and the facial expression F of the person P included in the video V of the program content 25. By collating person information, logo information, object information, and facial expression information, person P is "○ △ player", logo L is "○ × open", and person P's belongings B is "tennis (racket)". It is verified that the facial expression F of the person P is the "full facial expression", and each can be extracted as the character information C.
As shown in FIGS. 1 and 3, in the present embodiment, the image recognition information extraction means 22 includes the image V recorded in the recording file 11, the image-analyzed image, and the image-analyzed image. By collating the person information, logo information, object information, or facial expression information having the character information extracted from the video, the person P, the logo L, the personal belongings B of the person P, or the facial expression F of the person P included in the video V. Is extracted as character information C.

Further, as shown in FIG. 2, the voice information extracting means 23 performs voice analysis on the voice recorded together with the video V recorded in the recording file 11, and extracts the character information C from the voice (S2c). ..
As shown in FIGS. 1 and 3, in the voice information extraction means 23 according to the present embodiment, the voice analysis means 39 has the image V recorded in the recording file 11 and the image-analyzed image. And the voice analysis is performed by collating with the voice analysis storage file 38 having the character information extracted from the image-analyzed video.

Subsequently, as shown in FIG. 2, the composite information collating means 24 collates the extracted character information with each other by the character information extracting means 17, the video recognition information extracting means 22, and the voice information extracting means 23, respectively. (S3).
Specifically, as shown in FIGS. 1 and 3, the “○ △ player” and “○ × open” extracted by the character information extracting means 17 are “○ △” extracted by the video recognition information extracting means 22. It is possible to improve the accuracy of the character information C by collating with "player" (extracted from the person P) and "○ × open" (extracted from the logo L) and confirming that the character information C is correctly extracted.
If the processing speed is prioritized, the collation step S3 by the composite information collation means 24 may be omitted.

Here, as shown in FIGS. 1 and 3, the image analysis learning means 36 accumulates image analysis based on the image V image-analyzed by the image analysis means 31 and the character information C extracted from the image V. File 35 can be modified.

Further, as shown in FIGS. 1 and 3, the video recognition learning means 37 is based on the video V image-analyzed by the video recognition information extraction means 22 and the character information C extracted from the video V. , Person information, logo information, object information or facial expression information can be modified.

Further, as shown in FIG. 1, the voice analysis learning means 40 modifies the voice analysis storage file 38 based on the voice analyzed by the voice analysis means 39 and the character information extracted from the voice. can do.

Subsequently, as shown in FIG. 2, the dictionary collating means 19 collates the character information C extracted by the character information extracting means 17, the video recognition information extracting means 22, or the voice information extracting means 23 with the dictionary file 18. (S4a). If the character information C does not match the dictionary file 18 as a result of collation, the character information C is corrected based on the dictionary file 18. On the other hand, when the character information C matches the dictionary file 18, the collation process ends as it is.
Here, as shown in FIG. 1, the dictionary collation means 19 can preferentially select dictionary data D having a large frequency parameter 34 as a collation target. For example, when the character information C is mistakenly extracted as "continuous report" instead of "flash report" by the character information extracting means 17, it is collated with the dictionary file 18 and the word "continuous report" is registered. However, since the word "flash report" is frequently used and the frequency parameter of the word is large, it is possible to determine that the character information C is "flash report" and correct the character information C.

Further, as shown in FIGS. 1 and 3, the dictionary updating means 22 can modify the dictionary file 18 based on the character information C extracted by the character information extracting means 17.
For example, when "Dokudanjo" is extracted by the character information extraction means 17, in addition to the correct expression "Dokusenjo" existing in the dictionary file 18, "Dokudanjo" An originally incorrect but commonly used expression can be added to the dictionary file 18.

Further, as shown in FIG. 2, the Internet collation means 21 searches and acquires the character information C extracted by the character information extraction means 17, the video recognition information extraction means 22, or the voice information extraction means 23 by the Internet 20. It can also be collated with the information I (S4b).
If the character information C does not match the information I as a result of collation, the character information C is corrected based on the information I. On the other hand, when the character information C matches the information I, the collation process is terminated as it is.
When the processing speed is prioritized, only one of the collation step S4a by the dictionary collation means 19 and the collation step S4b by the Internet collation means 21 needs to be executed, while the accuracy (accuracy) of the character information C is prioritized. If so, both steps can be performed in any order.

Next, as shown in FIG. 2, the character information writing means 14 aggregates the acquired character information C and puts it into a sentence (S5).
Specifically, as shown in FIG. 3, the acquired character information C is "XX news", "breaking news", "○ △ player", "○ × open", "final", "advancement", "Tennis (racket)" can be aggregated and written into textual information such as "[XX News] XX Open Japanese top tennis player XX has advanced to the final." ..
At this time, the character information writing means 14 can refer to the metadata storage file 15 and use the metadata M related to the character information C acquired by the character information acquiring means 13 for writing the character information C. it can.
For example, the metadata related to the video broadcast the day before says "(02/28 12:00) [XX News] XX Japan's top tennis player XX has advanced to the semi-finals." If it is a thing, use the sentences "I participated in the ○ × open", "Japan's top tennis player ○ △ player", and "advanced to the semi-final" to quickly write the text information C. Can be processed to improve accuracy.

Further, as shown in FIGS. 1 and 3, the character information writing means 14 may acquire the electronic program guide data E of the video V recorded in the recording file 11 and use it for writing the character information C. it can. For example, if the electronic program guide data E contains the information "March 1, 12:00 XX news", the metadata M contains the information "(03/01 12:00) [XX news]". In addition, it is possible to quickly process the textualization of the character information C and improve the accuracy.

Next, as shown in FIG. 2, the metadata storage means 16 stores the character information documented by the character information documenting means 14 in the metadata storage file 15 as the metadata M of the video V recorded in the recording file 11. Store (S6).
Specifically, as shown in FIG. 3B, the metadata storage means 16 opens "(03/01 12:00) [XX news] XX news" as the metadata M of the video V of the program content 25. It is possible to store the metadata that "the top Japanese tennis player ○ △ player who participated in the game has advanced to the final" in the metadata storage file 15.
From the above, the metadata M of the video V can be created from the character information C which is displayed on the video V and is information on words and sentences related to the video V.

(3) Effect of Metadata Generation System 10 According to the Present Embodiment As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the recording file 11 is provided by the recording means 12. When the video is recorded in, the character information acquisition means 13 acquires the character information C displayed in the video V recorded in the recording file 11, and the character information documenting means 14 acquires the character information. C is documented, and the documented character information is stored in the metadata storage file 15 as the metadata M of the video V by the metadata storage means 16, so that the words displayed in the video V and related to the video V are displayed. , The metadata M of the video V can be automatically created with high accuracy from the character information C which is the text information.
As a result, it is possible to provide a system capable of creating metadata about a television broadcast program in a short time and reducing human costs.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the image V recorded in the recording file 11 is image-analyzed by the character information extraction means 17. The character information C is extracted from V, and the extracted character information C is collated with the dictionary file 18 by the dictionary collating means 19.
Therefore, the character information C can be efficiently extracted from the video V by image analysis, and the character information C is collated with the dictionary file 18, so that, for example, it cannot be erroneously recognized or completely recognized by image analysis. It is possible to improve the accuracy of the character information C by correcting the characters and words that have been lost based on the dictionary file 18.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the image V recorded in the recording file 11 by the image analysis means 31 is the image that has been image-analyzed and the image V that has been image-analyzed. The image is analyzed by collating with the image analysis storage file 35 having the character information extracted from the image-analyzed video.
Therefore, it is possible to effectively perform image analysis using the image analysis results accumulated from the past, and as a result, it is possible to accurately create the metadata M of the image V in a short time.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the image V in which the image analysis storage file 35 is image-analyzed by the image analysis means 31 by the image analysis learning means 36. And the character information C extracted from the image V, and the correction is made.
Therefore, the result of the image analysis performed this time can be added to the image analysis storage file 35, and the erroneous information contained in the image analysis storage file 35 can be deleted based on the result of the image analysis performed this time. , The image analysis storage file 35 can be updated and always used in the latest state.

Further, as shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the image V recorded in the recording file 11 is image-analyzed by the character information extraction means 17. Character information C is extracted from the video V, and the extracted character information C is collated with the information I searched and acquired by the Internet collation means 21.
Therefore, the character information C can be efficiently extracted from the video V by image analysis, and the character information C is collated with the information I searched and acquired by the Internet, so that, for example, it is erroneously recognized by image analysis or completely. Characters and words that could not be recognized can be corrected based on the information I obtained by searching on the Internet, and the accuracy of the character information C can be improved.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the dictionary file 18 is based on the character information C extracted by the character information extracting means 17 by the dictionary updating means 33. Information such as new words and sentences obtained from the character information C is added to the dictionary file 18, and erroneous information contained in the dictionary file 18 is deleted based on the character information C. As a result, the dictionary file 18 can be updated and always used in the latest state.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the dictionary data D having a large frequency parameter 34 is preferentially selected as a collation target by the dictionary collation means 19. Since the selected dictionary data D and the character information C extracted by the character information extraction means 17 are collated, for example, when a plurality of characters or words similar to each other exist in the dictionary file 18, the frequency parameter 34 The dictionary data D having a large size is preferentially selected and becomes a collation target.
As a result, the correction can be made based on the dictionary data having a large frequency parameter 34, and the accuracy of the character information C can be improved more efficiently.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the character information writing means 14 aggregates the character information C acquired by the character information acquisition means 13. When writing the text, the metadata storage file 15 can be referred to, and the created metadata M related to the character information C can be used for writing the character information C, and as a result, a television broadcast program or the Internet. It is possible to automatically generate metadata about the delivered video with high accuracy and more efficiently.

As shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the character information writing means 14 aggregates the character information C acquired by the character information acquisition means 13. When writing the text, the electronic program guide data E of the video V is acquired, and the electronic program guide data E including information such as the broadcast date and time, the delivery date and time, the genre, the title, and the performers is written as the character information C. Can be used for. As a result, metadata related to a television broadcast program or an Internet-distributed moving image can be automatically generated with high accuracy and more efficiently.

Further, as shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the image recognition information extracting means 22 causes the person P, the logo L, and the person P included in the image V to be connected. The facial expression F of the possession B or the person P is collated with the personal information, the logo information, the physical information or the facial expression information, and the personality P, the logo L, and the facial expression F of the personal belongings B or the person P included in the image V are characters. Since it is extracted as the information C, the metadata M of the image V can be created from the person P, the logo L, the belongings B of the person P, or the facial expression F of the person P included in the image V.

Further, in the metadata generation system 10 according to the present embodiment, the image V recorded in the recording file 11 by the image recognition information extraction means 22 is an image-analyzed image and an image-analyzed image. By collating with the person information, the logo information, the object information, or the facial expression information having the character information extracted from, the person P, the logo L, the person P's belongings B or the person P included in the image V. The expression F of is extracted as the character information C.
Therefore, it is possible to effectively perform image analysis using the image analysis results accumulated from the past, and as a result, it is possible to accurately create the metadata M of the image V in a short time.

Further, in the metadata generation system 10 according to the present embodiment, the image recognition learning means 37 displays the person information, the logo information, the object information, or the facial expression information by the image recognition information extracting means 22. It is corrected based on the analyzed video V and the character information C extracted from the video V.
Therefore, the result of the image analysis performed this time may be added to the person information, the logo information, the object information or the facial expression information, or erroneous information included in the person information, the logo information, the object information or the facial expression information. Can be deleted based on the result of the image analysis performed this time, and as a result, the person information, the logo information, the object information, or the facial expression information can be updated and always used in the latest state. ..

Further, as shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, the audio recorded together with the video V recorded in the recording file 11 by the audio information extraction means 23 is recorded. The character information C is extracted from the voice by the voice analysis, and the extracted character information C is collated with the dictionary file 18 by the dictionary collating means 19.
Therefore, the character information C can be efficiently extracted from the voice recorded together with the video V by voice analysis, and the character information C is collated with the dictionary file 18, so that, for example, misrecognition or complete recognition is performed by voice analysis. Characters and words that could not be recorded can be corrected based on the dictionary file 18, and the accuracy of the character information C can be improved.

In the metadata generation system 10 according to the present embodiment, the voice recorded together with the video V recorded in the recording file 11 by the voice analysis means 39 is the voice that has been voice-analyzed and the voice that has been analyzed. The voice is analyzed by collating with the voice analysis storage file 38 having the character information extracted from the voice of.
Therefore, it is possible to effectively perform voice analysis using the voice analysis results accumulated from the past, and as a result, it is possible to accurately create the metadata M of the video V in a short time.

Further, in the metadata generation system 10 according to the present embodiment, the voice in which the voice analysis storage file 38 is voice-analyzed by the voice analysis means 39 by the voice analysis learning means 40 and the characters extracted from the voice. It is corrected based on the information C.
Therefore, the result of the voice analysis performed this time can be added to the voice analysis storage file 38, and the erroneous information contained in the voice analysis storage file 38 can be deleted based on the result of the voice analysis stored this time. , The voice analysis storage file 38 can be updated and always used in the latest state.

Further, as shown in FIGS. 1 and 3, in the metadata generation system 10 according to the present embodiment, when the video V is recorded in the recording file 11 by the recording means 12, character information is extracted. Character information C is extracted from the video V by image analysis of the video V recorded in the recording file 11 by the means 17, and the person P, the logo L, and the person included in the video V are extracted by the video recognition information extraction means 22. The facial expression F of P's belongings B or person P is collated with the personal information, logo information, physical information or facial expression information, and the personality P, logo L, person P's belongings B or person P's facial expression F included in the video V is collated. Is extracted as character information C, and the voice information extraction means 23 extracts the character information C from the voice by analyzing the voice recorded together with the video V recorded in the recording file 11, and the composite information collation means. By 24, the extracted character information C is collated with each other by the character information extracting means 17, the video recognition information extracting means 22, and the voice information extracting means 23, respectively.
Therefore, the character information C can be efficiently extracted from the image analysis, the voice analysis, and the person P, the logo L, the possession B of the person P, or the facial expression F of the person P included in the video V.
Further, the compound information collating means 24 collates the extracted character information C with each other by the character information extracting means 17, the video recognition information extracting means 22, and the voice information extracting means 23, respectively. Therefore, for example, the character information. Characters and words that are erroneously recognized by the extraction means 17 or cannot be completely recognized can be corrected based on the character information C extracted by the voice information extraction means 23.
As a result, it is possible to provide a system capable of automatically generating metadata related to a television broadcast program or an Internet-distributed moving image more accurately and efficiently.

In the present embodiment, the video V has been described as an example of a broadcast program video broadcast by the television broadcasting station 30, but the video V is not limited to the above configuration, and the video V is a video distributed by the Internet. It may be a video.

The present invention is widely applicable to systems that generate metadata and has industrial applicability.

10: Metadata generation system 11: Recorded file 12: Recording means 13: Character information acquisition means 14: Character information documenting means 15: Metadata storage file 16: Metadata storage means 17: Character information extraction means 18: Dictionary file 19 : Dictionary collation means 20: Internet 21: Internet collation means 22: Video recognition information extraction means 23: Audio information extraction means 24: Composite information collation means 25: Program content 26: Channel name 27: Time code 28: Program content summary text data 29: Channel name 30: Television broadcasting station 31: Image analysis means 32: Word analysis means 33: Dictionary update means 34: Frequency parameter 35: Image analysis storage file 36: Image analysis learning means 37: Video recognition learning means 38: Voice analysis Storage file 39: Voice analysis means 40: Voice analysis Learning means B: Person's belongings C: Character information D: Dictionary data E: Electronic program guide data F: Person's facial expression L: Logo M: Metadata P: Person V: Video

Claims (17)

A recording means having a recording file for recording a video, a character information acquisition means for acquiring character information from the video recorded in the recording file, and the character information acquired by the character information acquisition means are aggregated and documented. It is provided with a character information documenting means for storing the character information documented by the character information documenting means and a metadata storage means for storing the character information documented by the character information documenting means in a metadata storage file as metadata of a video recorded in the recording file.
The character information acquisition means includes a character information extraction means that performs image analysis on the video recorded in the recording file and extracts character information from the video.
The person, logo, belongings of the person or the facial expression of the person included in the video are collated with the person information, logo information, physical information or facial expression information, and the person, logo, belongings of the person or the facial expression information included in the video are collated. A video recognition information extraction means that extracts the facial expression of the person as character information,
A voice information extraction means that performs voice analysis on the voice recorded together with the video recorded in the recording file and extracts character information from the voice.
A metadata generation system comprising the character information extracting means, the video recognition information extracting means, and a composite information collating means for collating the character information extracted by the voice information extracting means with each other. The character information acquisition means performs image analysis on the video recorded in the recorded file, extracts character information from the video, and obtains the character information extracted by the character information extraction means. The metadata generation system according to claim 1, further comprising a dictionary collation means for collating with a dictionary file. The character information extracting means is characterized by having an image analysis means for collating an image-analyzed video and an image analysis storage file having character information extracted from the image-analyzed video for image analysis. The metadata generation system according to claim 2. The character information extracting means further includes an image analysis learning means for modifying the image analysis storage file based on the image analyzed by the image analysis means and the character information extracted from the image. The metadata generation system according to claim 3. The character information acquisition means performs image analysis on the video recorded in the recorded file, extracts character information from the video, and obtains the character information extracted by the character information extraction means. The metadata generation system according to claim 1, further comprising an Internet collation means for collating with information searched and acquired by the Internet. The metadata generation system according to claim 2, wherein the character information acquisition means further includes a dictionary update means for modifying the dictionary file based on the character information extracted by the character information extraction means. 2. The dictionary file has dictionary data and a frequency parameter of the dictionary data, and the dictionary collation means preferentially selects dictionary data having a large frequency parameter as a collation target. Described metadata generation system. The character information writing means refers to the metadata storage file, and uses the metadata related to the character information acquired by the character information acquisition means for writing the character information. Item 1. The metadata generation system according to item 1. The metadata generation system according to claim 1, wherein the character information writing means acquires electronic program guide data of a video recorded in the recorded file and uses it for writing the character information. The character information acquisition means collates the person, logo, belongings of the person or the facial expression of the person with the person information, logo information, object information or facial expression information included in the video, and the person included in the video. The metadata generation system according to claim 1, further comprising a video recognition information extracting means for extracting a logo, the belongings of the person, or the facial expression of the person as character information. The person information, the logo information, the object information, or the facial expression information is characterized in that it is composed of an image-analyzed image and character information extracted from the image-analyzed image.
10. The metadata generation system according to claim 10. The character information acquisition means obtains the person information, the logo information, the object information, or the facial expression information based on the image analyzed by the image recognition information extracting means and the character information extracted from the image. The metadata generation system according to claim 11, further comprising a video recognition learning means for modifying. The character information acquisition means has a voice information extraction means for extracting character information from the voice by performing voice analysis on the voice recorded together with the video recorded in the recording file. The metadata generation system described in. The voice information extracting means is characterized by having a voice analysis means for collating a voice analyzed voice and a voice analysis storage file having character information extracted from the voice analyzed voice to perform voice analysis. The metadata generation system according to claim 13. The character information acquisition means further includes a voice analysis learning means for modifying the voice analysis storage file based on the voice analyzed by the voice analysis means and the character information extracted from the voice. 14. The metadata generation system according to claim 14. The metadata generation system according to any one of claims 1 to 15, wherein the video is a broadcast program video broadcast by a television broadcasting station. The metadata generation system according to any one of claims 1 to 16, wherein the video is a moving image distributed via the Internet.

Images