JP2002245066A - Device and method for retrieving information and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To retrieve contents by utilizing the featured values of the both video and voice in AV(audio visual) contents. SOLUTION: A database where voice featured values are connected to a voice identifying code, a database where video featured values are connected to a video identifying code and an AV contents table where the voice identifying code is connected to the video identifying code are prepared so as to retrieve the video and the voice in parallel. When the video is retrieved, the AV contents table is referred to concerning the video identifying code so as to pickup the corresponding voice identifying code. The code is given to the voice retrieval and, then, the retrieval is completed when collation is obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval technique, and more particularly to an information retrieval technique for retrieving AV contents composed of video and music data.

More specifically, the present invention relates to an information retrieval technique for AV contents based on characteristic amounts of original data, and more particularly, to an AV retrieval method using both image and music characteristic amounts.
The present invention relates to an information search technology for searching for content.

[0003]

2. Description of the Related Art Recently, information processing technology based on arithmetic processing on a computer has been dramatically improved. Along with this, in addition to mere computer data, various data contents such as video and audio have also been digitized,
It is being treated as a computer file. Therefore, also from the viewpoint of effective use of information resources, search techniques for various data / contents have become increasingly important.

[0004] Alternatively, in the field of content distribution and broadcasting, when a pay-as-you-go system is introduced for the content usage amount, or when the use of the content is restricted from the viewpoint of copyright protection, the content is distributed or broadcast. Must be counted and monitored. For example, an information search technique is applied in order to specify what content has been distributed.

For example, in the case of data retrieval of a relatively small size in the text format, it is possible to perform an exhaustive retrieval based on the matching of texts, depending on the computing power of the computer. On the other hand, in the case of AV contents including video and music data, since the data amount is generally enormous, it is technically impossible to perform an exhaustive search from the beginning to the end of the contents.

[0006] As a music information search, there is a method in which music data is transcribed and collated (total search) with music data to be searched. In this method, pattern matching is performed from the beginning to the end of the music. However, the exhaustive search using no index has a disadvantage that the search speed is slow.

[0007] For example, in music broadcasting, monitoring based on signal feature extraction has been already performed instead of comparison and collation on a musical note basis. That is, ISRC (International
The feature amount of the music corresponding to the unique code of the music such as Standard Recording Number is registered in the database in advance. For ISRC and music collection, C
Use existing recording media such as D (Compact Disc). Then, it receives the broadcast, compares the feature amount with the feature amount registered in the database, and specifies the ISRC of the received music (see FIG. 6).

[0008] The same search performed in music broadcasting is called A
To extend the scope of application to V content,
There is a problem that it is not possible to extract a silent part if it is performed by voice. For example, since there is much silence in a movie or the like as compared with music-only content, a search based on feature extraction of music data is limited.

[0009] On the other hand, when searching for AV contents only by video, there are many portions where the content of the video changes little, such as scenes in which characters are talking, so that search based on feature extraction of video data is limited. There is.

In the movie industry, even for the same movie, language correspondence according to the official language of each country or region to which the movie is sold, that is, voice dubbing processing is performed. As a result of the dubbing process, if a voice-based content search is performed, there is no match even for the same movie.

[0011]

SUMMARY OF THE INVENTION An object of the present invention is to provide an excellent information retrieval apparatus and method capable of quickly and reliably retrieving AV contents composed of video and music data. .

[0012]

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and a first aspect of the present invention is an AV system comprising a combination of video data and audio data.
An information search device or method for searching for content,
Using an audio database in which audio features and audio identification codes are linked, a video database in which video features and video identification codes are linked, an AV content table in which audio identification codes and video identification codes are linked, and an audio database Audio data search means or step for searching for audio data, video data search means or step for searching for video using a video database, the audio data search means or step, and the video data search means or step. An information retrieval apparatus or method comprising: an AV content determining unit or step of determining a corresponding AV content from the AV content table based on the extracted audio identification code and video identification code. .

[0013] An information retrieval apparatus or method according to the present invention includes a database in which audio features and audio identification codes are linked, a database in which video features and video identification codes are linked, and a voice identification code and a video identification code. A connected
Prepare a V content table. Then, the video search and the audio search are performed in parallel.

[0014] For example, when the video identification code is found in the video search, the video identification code is referred to in the AV content table to extract the corresponding audio identification code. This code is given to the voice search, and if the collation is obtained, the search is completed.
Similarly, when the audio identification code is found in the audio search, the audio identification code is referred to in the AV content table to extract the corresponding video identification code. This code is given to the video search, and if the collation is obtained, the search is completed.

When such a search is performed in a broadcasting system or the like, it may be necessary to allow a certain degree of distortion as a system, such as the influence of multipath, distortion of a transmission / reception circuit, or distortion due to compression. In this case, it is impossible to perform a search with the exact same search, so that it is necessary to give a certain degree of tolerance for the judgment at the time of matching the feature amounts. Conversely, by giving the allowance, there is a possibility that a plurality of search results will appear for one certain search target.

When the speed of the search is increased, the possibility that a plurality of search results will appear is inevitably increased. In order to eliminate such a situation, both video and audio may be searched for to improve reliability.

[0017] If the video is searched first and found,
The video identification code is referred to the AV content table, and the audio identification code is given to the audio search for collation. Conversely, if the search is performed by voice and found earlier, this voice identification code is referred to the AV content table, and a video identification code is given to the video search for verification. Then, the determination is made by seeing that the results of both the video and the audio match. When a plurality of video and audio search results appear, all combinations are determined, and a result with a small error is selected.

The information retrieval apparatus or method according to the first aspect of the present invention further comprises a content identification code acquisition means or step for an AV content to be retrieved, and the audio data retrieval means or step and / or The video data search means or step may preferentially collate the AV content corresponding to the content identification code.

If a mechanism for obtaining information such as an identification code for identifying the content is provided, and if the information can be obtained without fear of tampering, there is no need for signal collation. However, the copyright holder of the content fears that a fake identification code will be generated by the user falsifying the playback device or the recording device. Therefore, it is necessary to perform signal collation for confirming tampering.

In many cases, the content has not been tampered with, and the information retrieval speed of the entire system can be increased by preferentially performing collation processing on the content corresponding to the content identification code.

The audio data search means or step and / or the video data search means or step may include:
You may make it omit the collation process with respect to the obviously different AV contents.

It is not necessary to perform audio data retrieval and video data retrieval for AV contents which can be easily recognized as being clearly different. By omitting such steps, the computer load can be reduced and more similarity can be obtained. It is possible to perform a high-precision matching process by reducing an error in the content. In addition, since extra collation processing for content that is clearly different is omitted, the information retrieval speed of the entire system can be increased.

According to a second aspect of the present invention, computer software written to execute, on a computer system, an information retrieval process for retrieving AV contents composed of a combination of video data and audio data in a computer readable format. A physically stored storage medium, wherein the computer software performs an audio data search step of searching for audio using an audio database in which audio features and audio identification codes are linked; A video data search step for searching for a video using a video database to which an identification code is connected; and an audio identification code based on the audio identification code and the video identification code extracted by the audio data search step and the video data search step. AV with video identification code
An AV content determining step of determining the corresponding AV content from a content table.

The storage medium according to the second aspect of the present invention is a medium that provides computer software in a computer-readable format to a general-purpose computer system that can execute various program codes.
Such a medium is, for example, a CD (Compact Disc) or F
It is a removable and portable storage medium such as D (Floppy Disk) and MO (Magneto-Optical disc). Alternatively, it is technically possible to provide computer software to a specific computer system via a transmission medium such as a network (a network may be either wireless or wired).

Such a storage medium defines a structural or functional cooperative relationship between the computer software and the storage medium in order to realize predetermined computer software functions on a computer system. It is. In other words, by installing predetermined computer software into the computer system via the storage medium according to the second aspect of the present invention, a cooperative action is exerted on the computer system, and the second aspect of the present invention is realized. The same operation and effect as those of the information search device and method according to one aspect can be obtained.

Still other objects, features and advantages of the present invention are:
It will become apparent from the following more detailed description based on the embodiments of the present invention and the accompanying drawings.

[0027]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 schematically shows the configuration of an information retrieval apparatus 1 according to one embodiment of the present invention. Hereinafter, each component of the information search device 1 will be described with reference to FIG.

CPU (Central Processing Unit) 11
Is a central controller that comprehensively controls the operation of the entire information search device 1, and has an operating system (O
Under the program execution environment provided by S), the control program stored (or loaded) in the memory 12 and the match determination program stored (or loaded) in the other memory 13 are executed. I have.

The match determination program referred to here includes a video database collation process based on video feature values, an audio database collation process based on audio feature values, and AV content / video content based on video and audio database collation results.
This includes table collation processing. Further, the control program includes an information search processing program related to AV contents such as a process of specifying AV contents based on a result of the match determination.

The CPU 11 is interconnected with each module in the apparatus 1 via a bus 22. The bus 22 is a common signal transmission path including an address bus, a data bus, and a control bus.

The video / audio feature extraction unit 14 performs a demultiplexing process from AV contents to video and audio data, a feature extraction process for video data, and a feature extraction process for audio data. The feature values of the video and audio extracted by the video / audio feature extraction unit 14 are input to the match determination program.

The feature database 15 is a database used for a collation / information search process relating to AV contents. More specifically, the feature database 15
Is composed of an audio database in which audio features and audio identification codes are linked, a video database in which video features and video identification codes are linked, and an AV content table in which audio identification codes and video identification codes are linked. You.
An example of the configuration of the AV content table related to a movie is shown below.

[0034]

[Table 1]

In the AV content table described above, each record is composed of a combination of a movie name, a video identification code, an audio identification code, and an AV content identification code. Therefore, when the video identification code is found, the voice identification code that hits the video identification code can be narrowed down. Conversely, when the voice identification code is found, the video identification code that hits this can be narrowed down. Also,
If you specify both the video identification code and the audio identification code,
The AV content identification code, that is, the AV content is uniquely determined by referring to the AV content table.

The priority collation list 16 and the video / audio collation list 17 are used to temporarily store a part of work data when performing information retrieval, that is, collation processing of AV contents.

The priority collation list 16 is a storage module for listing AV contents to be preferentially collated. For example, an AV content to which a content identification code has been passed from the outside is registered in the priority collation list 16 until the collation / search range can be limited by this identification code to speed up the processing time.

The video / audio collation list 17 is used for temporarily storing a database collation result for each video data and audio data. For example, if the video data matches but the corresponding audio data does not match,
Since there is no need to perform video data collation processing, this fact is entered in the video collation list. Conversely, if the audio data matches but the corresponding video data does not match, there is no need to perform the matching process on the audio data, so that fact is entered in the audio matching list. If the audio is clearly different, enter that fact in the audio collation list so that the high-precision collation processing is skipped. Fill in and skip the verification process.

The information input section 18 is a functional module for inputting information from an external device. The information input here includes A in which video data and audio data are multiplexed.
V content and its content identification code are included. As a medium for inputting and outputting information, a LAN (Lo
cal Area Network) and computer networks such as the Internet, CD (Compact Disc) and DVD (Di
gital Versatile Disc) and the like. Information supplied from an external device via the information input unit 18 is temporarily stored in, for example, a buffer memory 19.

The display unit 20 outputs the result of the calculation by the CPU 11 on the screen to provide visual feedback of information to the user. The user input unit 21 is configured by an input device such as a keyboard and a mouse (both not shown), and receives commands and data input from the user.

The information search device 1 can be configured using, for example, a general computer system. An example of such a computer system is PC / AT (Pe
rsonal Computer / Advanced Technology) compatible machine or its successor.

FIG. 2 shows a schematic procedure for the AV content search processing executed on the information search device 1.
It is shown in the form of a flowchart. Hereinafter, the flow of the AV content search process according to the present embodiment will be schematically described with reference to this flowchart.

First, AV contents to be searched are demultiplexed and separated into video signals and audio signals (step S1). Although not shown, the video signal and the audio signal are respectively decompressed and decompressed as necessary.

Next, a feature amount is extracted from each of the video signal and the audio signal, and is compared with each data registered in advance in the feature database 15 to extract a content identification code. Step S2A Voice Verification and Step S2
The video collation of B is performed in parallel. For the content identification code, the identification code of the video combined with the audio is extracted with reference to the AV content table (described above) in the feature database 15 (step S3A). An identification code is extracted (step S3B).

The extracted voice identification code is used for voice verification 2
(Step S4A), the feature amount corresponding to the content identification code is checked against the voice database, and the content identification code of the corresponding voice is confirmed or selected. Similarly, the extracted video identification code is passed to video collation 2 (step S4B), and the feature amount corresponding to the content identification code is collated with the video database to confirm or select the content identification code of the corresponding video. I do.

Speeding up the identification of AV contents is achieved by setting the decision logic (step S5) so as to adopt the earlier obtained content identification code from the audio collation 2 or video collation 2. be able to.

When the AV content table shown in [Table 1] is used, for example, if the video identification code (4) is extracted by video collation and the AV content table is referred to, the corresponding audio identification code is (6). , 7, 8)
Can be taken out. These audio identification codes are input to the audio collation 2 where the collation matches and the AV content identification code (for example, 4-7) is determined.

In order to cope with distortions, such as distortions in the transmission system, multi-paths during transmission, distortions during compression and recording / reproduction, and to deal with distortions. Two contents may correspond to one identification code at the time of collation. In such a case, the decision logic can be configured so as to employ an identification code in which the audio verification 2 and the video verification 2 match, thereby improving reliability.

In the flowchart shown in FIG. 2, a path indicated by a dotted line is provided, and if the identification code cannot be obtained from the audio collation 2 or the video collation 2, the video identification or audio success Use and provide user feedback such as "The sound is not normal but the identification code can be read as XXXXXX from the video" or "The video does not seem normal but the identification code can be read as XXXXXX from the audio" May be performed via the display unit 20 or the like.

FIG. 3 is a flowchart showing an example of a specific procedure of an AV content search process executed on the information search device 1 according to the present embodiment. The processing procedure shown in the figure is an example in a case where an error is not allowed in the collation.

First, the AV contents to be searched are demultiplexed and separated into video signals and audio signals (step S11). Although not shown, the video signal and the audio signal are respectively decompressed and decompressed as necessary.

Next, feature values are extracted from the video signal and the audio signal, respectively (step S12). Then, the characteristic value of each of the video and audio is compared with each data registered in advance in the characteristic database 15 (step S1).
3) It is determined whether they match (step S14).
The matching speed of video and audio varies depending on the content. In the present embodiment, the video and audio matching processes are performed simultaneously and in parallel on a best-effort basis.

If no match is found for either video or audio (step S14), the process proceeds to the next content in the feature database (step S2).
0) Similarly, the matching of the video / audio feature amount is repeatedly executed. If no matching content is found in the feature database, a display indicating that there is no corresponding content is output from the display unit 20 to notify the user (step S21), and the entire processing routine ends.

If matching data is found for the video (step S15), the voice collation list is referred to
It is determined whether or not there is a match with the audio corresponding to the video data (step S18). If there is no match for the corresponding audio, the fact that the video does not match is recorded in the video collation list (step S19), and the process returns to step S13 to jump to the content of the corresponding video, and Compare with the feature database.

When matching data is found for the audio (step S15), it is determined whether or not there is a matching video for the audio data by referring to the video collation list (step S16). .
If there is no matching video, the fact that the voice does not match is recorded in the voice collation list (step S17), and the process returns to step S13 to jump to the content of the corresponding voice, and Is compared with the feature database.

As a result of the decision block S16 or S18, if a matching AV content for video and audio is found, the content identification code is output as a search result, and the entire processing routine ends.

FIG. 4 is a flowchart showing another example of the specific procedure of the AV content search process executed on the information search device 1 according to the present embodiment. The processing procedure shown in the figure is an example of a case where a search is performed using a content identification code given by some means and an error is not allowed in the collation.

First, the content identification code of the AV content to be searched is obtained by some means (step S31) and registered in the priority collation list 16 (step S32). The information search device 1 according to the present embodiment preferentially performs a matching process on the AV contents registered in the priority matching list 16 when, for example, continuously inputting AV content as a search target.

Next, the AV content corresponding to the re-priority content identification code registered in the priority collation list 16 is demultiplexed and separated into a video signal and an audio signal (step S33). Although not shown, the video signal and the audio signal are respectively decompressed and decompressed as necessary.

Next, feature values are extracted from the video signal and the audio signal, respectively (step S34). Then, the characteristic value of each of the video and the audio is collated with each data registered in advance in the characteristic database 15 (step S3).
5) It is determined whether they match (step S36).
The matching speed of video and audio varies depending on the content. In the present embodiment, the video and audio matching processes are performed simultaneously and in parallel on a best-effort basis.

If no match is found for either video or audio (step S36), the process proceeds to the next content in the feature database (step S4).
2) Similarly, the matching of the video / audio feature amount is repeatedly executed. If no matching content is found in the feature database, a display indicating that there is no corresponding content is output from the display unit 20 to notify the user (step S43), and the entire processing routine ends.

If matching data is found for the video (step S37), the voice collation list is referred to
It is determined whether or not there is a match with the audio corresponding to the video data (step S40). If there is no match for the corresponding audio, the fact that the video does not match is recorded in the video collation list (step S41), and the process returns to step S35 to jump to the content of the corresponding video, and Is compared with the feature database.

If matching data is found for the audio (step S37), it is determined whether or not there is a matching video for the audio data by referring to the video collation list (step S38). .
If there is no matching video, the fact that the voice does not match is recorded in the voice collation list (step S39), and the process returns to step S35 to jump to the content of the corresponding voice, and Is compared with the feature database.

As a result of the decision block S38 or S40, if a matching AV content is found for video and audio, the content identification code is output as a search result, and the entire processing routine ends.

If the content ID of the AV content to be searched can be obtained without fear of tampering, signal collation is not necessary. However, the copyright holder of the content,
It is feared that a fake ID is created by modifying the reproducing device or the recording device. Therefore, it is necessary to perform signal collation to confirm the tampering. In many cases, the content has not been tampered with, and as described above, the content in the priority collation list 16 is preferentially collated, and if there is a match, the content search can be terminated. Speed can be increased.

FIG. 5 is a flowchart showing another example of a specific procedure of the AV content search processing executed on the information search apparatus 1 according to the present embodiment. The processing procedure shown in the figure is an example in which the speed is increased by excluding the AV contents registered in the feature database that are clearly different from the AV contents to be searched for and performing the matching process.

First, the AV content to be searched is demultiplexed and separated into a video signal and an audio signal (step S51). Although not shown, the video signal and the audio signal are respectively decompressed and decompressed as necessary.

Next, feature values are extracted from the video signal and the audio signal, respectively (step S52). Then, the feature value of each of the video and audio is compared with each data registered in advance in the feature database 15 (step S5).
3) It is determined whether or not the AV content is clearly different from the AV content to be searched (step S54). The matching speed of video and audio varies depending on the content. In the present embodiment, the video and audio matching processes are performed simultaneously and in parallel on a best-effort basis. Here, since the purpose is only to verify the difference between the contents, the collation processing in which a relatively large error is recognized may be performed in step S53.

If it is determined in step S54 that the difference is clear, it is further determined which of the video and the sound is clearly different (step S55).

If there is a clear difference in the video,
After recording that the video does not match in the video collation list (step S56), the process returns to step S53, and the collation process of the corresponding video is skipped.

If there is a clear difference between the voices, the fact that the voices do not match is recorded in the voice verification list (step S57), and the process returns to step S53 to skip the corresponding voice verification process. .

If the AV content is not clearly different from the AV content to be searched, the process proceeds to the next content in the feature database (step S58), and whether the AV content is also clearly different from the AV content to be searched. The process of determining whether or not to perform the determination is repeatedly performed.

After verifying the difference between all the data in the feature database and the AV content to be searched, the feature values of the video and audio are collated with the data registered in advance in the feature database 15 (step S59), it is determined whether or not the AV content is different from the search target AV content (step S60). The matching speed of video and audio varies depending on the content. In the present embodiment, the video and audio matching processes are performed simultaneously and in parallel on a best-effort basis. Here, since the purpose is to narrow down the data, step S59 is performed.
Then, a collation process in which the error is made smaller than in the preceding step S53 is performed.

If it is determined in step S60 that there is a difference, it is further determined which of video and audio is different (step S61).

When there is a difference between the videos, the fact that the videos do not match is recorded in the video collation list (step S62), and the process returns to step S59 to skip the collation process of the corresponding video.

If there is a difference between the voices, the fact that the voices do not match is recorded in the voice collation list (step S63), and the process returns to step S59 to skip the matching process of the corresponding voice.

After verifying the difference between all the data in the feature database and the AV content to be searched, the entire processing routine ends.

By gradually reducing the collation error in this way, it is possible to quickly search for a content that is finally close to features. In the steps up to step S58, if the contents that are not clearly different are narrowed down to only one, it is not necessary to execute the steps after step S59 together.

If it is assumed that the signal under test has a large amount of distortion, in steps S53 to S58,
By gradually reducing the degree of difference and repeating it many times, it is possible to search in detail for the content with the smallest error.

The improvement of the retrieval performance has been described above. Finally, an application example of the information retrieval according to the present invention will be described.

(1) A first application example is a broadcast monitoring. Generally, a business model is constructed between a sponsor of a broadcast program and a broadcaster, in which the sponsor pays a corresponding amount (a sponsor fee) to the broadcaster in response to insertion of an advertisement.

However, sponsors need to monitor the actual broadcast, as advertisements may not be inserted as promised. In such a case, the feature amount of the advertisement is registered in the database in advance, the broadcast is received, the feature is extracted from the broadcast content, and collated with the database.

(2) Further, there can be mentioned an application in which the copyright holder who produces a program monitors whether or not the broadcast content is illegally copied and used for broadcasting.

When a copy or a part of the copy is stored on a recording medium such as a tape, the information search method according to the present invention can be used for the purpose of searching for the main part from an archive.

It is also a wise method to perform feature extraction at the same time as shooting for program production, and create a database of feature amounts. In this case, it is more convenient to register data such as identification codes of cameras and photographers, date and time, latitude and longitude using GPS (Global Positioning System), and the like.

(3) Even at home, a VTR (Video
A service that extracts features from content recorded by a pe recorder or a disc recorder, notifies the user as a recommended version of a related program from the viewing history, or automatically records the service is also conceivable. Feature extraction uses multiple VTs
The rationalization can be achieved by performing the processing between receivers such as R, DVD (Digital Versatile Disc) players, other recording / reproducing devices, tuners, etc. where all contents are collected.

A service that teaches a user a recommended program from a user's viewing history as a recommended version is a service for each viewer, and therefore requires an identification code of a receiving device to identify a viewer.

Further, in order to specify an individual in the home, a remote commander is occupied for each individual and its identification code is used. These identification codes also need to be sent to the service provider. At the time of reception, the search becomes easier if the time and channel are known. Also, the signal quality (for receiving, playing,
When the reproduction mode (for example, three times as large as the standard) is known, this is sent to the database side, and the search speed or the search certainty can be improved by adaptively controlling the tolerance at the time of collation.

[Supplement] The present invention has been described in detail with reference to the specific embodiments. However, it is obvious that those skilled in the art can modify or substitute the embodiment without departing from the spirit of the present invention. That is, the present invention has been disclosed by way of example, and should not be construed as limiting. In order to determine the gist of the present invention, the claims described at the beginning should be considered.

[0090]

As described above in detail, according to the present invention,
It is possible to provide an excellent information retrieval apparatus and method capable of quickly and surely retrieving AV contents composed of video and music data.

According to the information retrieval apparatus and method according to the present invention, it is possible to retrieve AV content more quickly than video content or audio content alone. In addition, even when a search is performed while allowing distortion in the system, A
V content can be searched.

[Brief description of the drawings]

FIG. 1 is a diagram schematically showing a configuration of an information search device 1 according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a schematic procedure of an AV content search process performed on the information search device 1.

FIG. 3 is a flowchart showing an example of a specific procedure of an AV content search process executed on the information search device 1.

FIG. 4 is a flowchart illustrating another example of the specific procedure of the AV content search processing executed on the information search device 1.

FIG. 5 is a flowchart showing another example of the specific procedure of the AV content search processing executed on the information search device 1.

FIG. 6 is a diagram (conventional example) showing a music search mechanism using a unique code of a music such as ISRC.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Information retrieval device 11 ... CPU 12 ... Memory (control program) 13 ... Memory (coincidence discrimination program) 14 ... Feature extraction part 15 ... Feature database 16 ... Priority collation list 17 ... Video / audio collation list 18 ... Information input part 19 ... Buffer 20 ... Display unit 21 ... User input unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 15/28 G10L 3/00 551P H04N 5/76

Claims (9)

[Claims] An information retrieval apparatus for retrieving AV contents comprising a combination of video data and audio data, comprising: an audio database in which audio features and audio identification codes are linked; A linked video database, an AV content table in which a voice identification code and a video identification code are linked, a voice data search means for searching for a voice using the voice database, and a video data for searching for a video using the video database Search means; and a corresponding A from the AV content table based on the audio identification code and the video identification code extracted by the audio data search means and the video data search means.
An information retrieval device comprising: AV content determination means for determining V content. 2. The audio data search means and the video data search means execute a search in parallel. If the video data search means finds the audio data first, the AV content determination means outputs the video identification code. Is referred to in the AV content table to derive a corresponding voice identification code, and the voice identification code is given to voice data search means for verification. The content determining means refers to the audio identification code in the AV content table, derives a corresponding video identification code, and provides the video identification code to the video data search means for collation. 1
An information retrieval device according to item 1. 3. An audiovisual data retrieval means and / or a video data retrieval means, wherein said audio data retrieval means and / or the video data retrieval means preferentially give priority to the AV contents corresponding to the contents identification code. The information search device according to claim 1, wherein the information search is performed. 4. The information search apparatus according to claim 1, wherein the audio data search means and / or the video data search means omit a collation process for an apparently different AV content. 5. An information retrieval method for retrieving AV content composed of a combination of video data and audio data, wherein the audio data retrieval is performed using an audio database in which audio features and audio identification codes are linked. A video data search step of searching for a video using a video database in which a video feature amount and a video identification code are linked; a voice data search step; a voice identification code and a video extracted by the video data search step; An AV content determining step of determining a corresponding AV content from an AV content table in which an audio identification code and a video identification code are linked based on the identification code. 6. The audio data search step and the video data search step perform search in parallel. If the audio data search step and the video data search step are performed first, the AV content determination step includes determining the video identification code. Is referred to in the AV content table, a corresponding voice identification code is extracted, and the voice identification code is input to a voice data search step for verification. If the voice identification code is found earlier by the voice data search step, In the AV content determination step, the audio identification code is referred to in the AV content table to derive a corresponding video identification code, and the video identification code is input to a video data search step for collation. The information search method according to claim 5. 7. The audio data search means and / or the video data search step further comprises a step of obtaining a content identification code of the AV content to be searched. 6. The information search method according to claim 5, wherein a collation is performed. 8. The information search method according to claim 5, wherein the audio data search step and / or the video data search step omit a matching process for an AV content that is clearly different. 9. A storage medium physically storing computer software described in a computer readable format so as to execute, on a computer system, information retrieval processing for retrieving AV contents composed of a combination of video data and audio data. Wherein the computer software comprises: an audio data search step of searching for audio using an audio database in which audio features and audio identification codes are linked; and a video database in which video features and video identification codes are linked. A video data search step of searching for a video by using a voice identification code and a video identification code based on the audio identification code and the video identification code extracted by the audio data search step and the video data search step. The corresponding AV from the AV content table Storage medium characterized by comprising the AV content determining step of determining the content, the.