KR101396413B1 - Information providing system and method using digital fingerprinting - Google Patents

Abstract

A fingerprint indexing unit for extracting minutiae points of broadcast data and constructing a database so that the minutiae can be used for searching. The system receives a program recognition request from a client device and transmits the program recognition request to a fingerprint matching unit. Receiving a search result related to the recognition request from the fingerprint matching unit and transmitting the search result to the client device, and performing a search using the fingerprint indexing unit based on the program recognition request, extracting the search result And transmits the generated fingerprint to the fingerprint web server.

Description

TECHNICAL FIELD [0001] The present invention relates to an information providing system and method using digital fingerprinting,

The present invention relates to an information providing system and method, and more particularly, to an information providing system and method using digital fingerprinting.

Digital fingerprinting technology extracts unique characteristic values (DNA) from multimedia contents such as audio, image and video and builds database in advance in the server. Then, users can identify (smartphone, smart pad, etc.) And extracts the corresponding feature value from the client terminal. At this time, if the extracted feature value is compared with the feature values registered in the database constructed in the server to find the matching content feature value, it is possible to recognize what the content acquired by the current user is and to provide the related information to the user terminal It is widely used for service.

This feature value based fingerprinting technology is also used as a copyright protection technology to monitor and limit illegal contents distribution of special type of OSP (Online Service Provider) such as P2P and WebHard.

In constructing a content identification service platform using digital fingerprinting technology, in order to support quick search in service, a feature value (DNA), which is unique identification information of a content, is extracted and indexed in a database, There is a part. This is because the contents identification service system using the digital fingerprinting technology takes quick search as a priority performance index.

This characteristic is not a serious problem because a sufficient amount of time can be allocated to register the identification information in a general file-based application. However, when applied to an on-air broadcasting environment such as terrestrial broadcasting, . As a simple example, considering the case of live broadcasting such as a music program or a sports broadcast, the time and delay required for a viewer to watch a TV broadcast receiving apparatus after broadcasting from a broadcasting station is estimated to be about 0.5 to 1 second in the earliest case . There is much more delay in relaying and retransmitting the program of the station through IPTV / CableTV, DMB, or the Web, but the time it takes for the broadcast transmitted from the station to reach the viewer, as in terrestrial broadcasting, It can be called inside and outside.

In such a broadcast environment, in order to register identification information at the time of broadcast transmission of the broadcast or program and to provide the identification service to the viewer, the service preparation state is set within a maximum allowable delay time (broadcast delay time + identification technology buffering time) .

In practice, this time constraint is a stumbling block to applying the identification service system construction and service application to broadcast environments and broadcast contents. If the service can not be put in a waiting state within a certain delay time, There is a problem in that it is necessary to delay the broadcast transmission so as to secure a sufficient time to be ready in the standby state.

Since the system delay factor occurs before the on-air broadcasting by changing the broadcasting system currently in operation, a delay of only one second may cause a lot of inconvenience to the consumers when live broadcasting such as a football game is broadcasted. It is difficult for other broadcasting companies to accept the goal of broadcasting a goal after 1 second.

In order to solve the above-described problems, an object of the present invention is to provide a broadcast program identification service in an on-air broadcast environment, in which identification information is transmitted in real time without delaying broadcasting, And an information providing system using digital fingerprinting and a method for enabling a viewer to perform identification service.

It is another object of the present invention to provide an information providing system and method using digital fingerprinting, which can improve search efficiency and system stability of an identification service system and achieve cost reduction.

According to another aspect of the present invention, there is provided a system for providing information through a fingerprint, the system comprising: a fingerprint indexing unit for building a database for extracting minutiae points of broadcast data for use in searching; And transmits the search result to the matching unit, receives the search result related to the program recognition request from the fingerprint matching unit and transmits the search result to the client device, and performs search using the fingerprint indexing unit based on the program recognition request And a fingerprint matching unit for extracting the search result and transmitting the extracted search result to the fingerprint web server.

Wherein the fingerprint indexing unit includes a register for extracting a feature point from the broadcast data and storing data related to the extracted feature point in a database, and a database for storing the feature point related data, wherein the database stores the feature point related data in a memory A memory buffer database for continuously storing minutiae point related data for a specific program, and a fingerprint distribution database for systematically distributing minutiae point related data to a plurality of databases.

The memory buffer database may be configured in a random access memory (RAM).

The registration unit may simultaneously store the minutia related data in the memory buffer database and the fingerprint distribution database.

The fingerprint matching unit performs a search using the memory buffer database in response to a program recognition request of a client device that receives real-time broadcast data, and the fingerprint matching unit searches the fingerprint distribution database The search can be performed using the < RTI ID = 0.0 >

The fingerprint web server may extract additional information related to the search result from the fingerprint service database based on the search result and provide the extracted additional information to the client device.

In order to provide the additional information, the information providing system generates a service web page according to a specific program using the broadcast programming information and the content metadata, and generates service information for determining the web page to be linked based on the broadcast information And storing the fingerprint service in the fingerprint service database.

The fingerprint indexing unit may further include a query analyzing unit for analyzing a query pattern generated for a specific broadcast time point in a specific program based on the program recognition request.

Wherein the fingerprint indexing unit includes a determination unit for determining whether a query generated for a specific broadcast time of the same program is greater than a threshold value, And a controller for dynamically loading feature point related information into the memory buffer.

According to another aspect of the present invention, there is provided a method for providing information through a fingerprint, the method comprising: receiving a program recognition request from a client device and transmitting the received program identification request to the fingerprint matching unit; The method comprising the steps of: building a database so that the fingerprint matching unit can extract and search the fingerprint matching unit, the fingerprint matching unit performing a search using the fingerprint indexing unit based on the program recognition request, Transmitting the search result to the fingerprint web server, and receiving the search result related to the program recognition request from the fingerprint matching unit and transmitting the search result to the client device.

The database building step may include extracting a feature point from the broadcast data and storing data related to the extracted feature point in a database, wherein the fingerprint indexing unit resides the feature point related data in a memory, A memory buffer database for continuously storing minutiae point related data for the program, and a fingerprint distribution database for systematically distributing minutiae point related data to a plurality of databases.

The memory buffer database may be configured in a random access memory (RAM).

The storing in the database may include storing the feature point related data in the memory buffer database and the fingerprint distribution database at the same time.

Wherein the step of performing the search includes performing a search using the memory buffer database in response to a program recognition request of a client device that receives real-time broadcast data, and a step of performing a search corresponding to a program recognition request related to a service after real- And performing a search using the fingerprint distribution database.

The information providing method may further include extracting additional information related to the search result from the fingerprint service database based on the search result, and providing the extracted additional information to the client device.

In order to provide the additional information, the fingerprint service manager generates a service web page according to a specific program using the broadcast programming information and the content metadata, and determines a web page to be linked based on the broadcast information And generating and storing the service information in the fingerprint service database.

The information providing method may further include analyzing a query pattern generated by the fingerprint indexing unit for a specific broadcasting time point in a specific program based on the program recognition request.

Wherein the information providing method includes: determining whether a query generated for a specific broadcast time point for the same program is greater than a threshold value; And controlling the dynamic loading of the minutiae point related information in the fingerprint distribution database to the memory buffer corresponding to the specific point-in-time query generated more than the threshold based on the determination result.

According to another aspect of the present invention, there is provided a system for providing information through a fingerprint, the system comprising: a database for extracting minutiae points of broadcast data to be used for searching, receiving a program recognition request from a client device, A fingerprinting service system for performing a search using the database, extracting a search result related to the program recognition request and transmitting the search result to the client device, a client device for transmitting the program recognition request to the fingerprinting service system, And a broadcast station system for transmitting broadcast data to the fingerprinting service system.

The fingerprinting service system includes a registration unit for extracting a feature point from the broadcast data and storing data related to the extracted feature point in a database, and a database for storing the feature point related data, wherein the database stores the feature point related data in a memory And a fingerprint distribution database for distributing the minutiae point related data systematically into a plurality of databases to store the minutia point related data.

According to the information providing system and method using the digital fingerprinting of the present invention, when a fingerprinting system is applied to real broadcasting, it is possible to implement the broadcasting system immediately without a time delay element, .

1 is a block diagram schematically illustrating a configuration of an actual broadcast information providing system according to an embodiment of the present invention.
FIG. 2 is a detailed block diagram illustrating a fingerprint indexing unit of an actual broadcast information providing system according to an exemplary embodiment of the present invention.
FIG. 3 is a detailed block diagram illustrating a fingerprint matching unit of an actual broadcast information providing system according to an exemplary embodiment of the present invention.
4 is a flowchart schematically illustrating a method of providing real broadcast information according to an embodiment of the present invention.
FIG. 5 is a conceptual diagram for explaining conditions for taking a broadcast service time delay D for implementing a fingerprinting service to zero; FIG.
6 is a diagram illustrating a case where a large-scale query occurs in a fingerprinting service system,
FIG. 7 illustrates an operation for solving a large-scale query of FIG. 6 in an actual broadcast information providing system according to an embodiment of the present invention;
FIG. 8 is a detailed block diagram specifically illustrating a configuration of a fingerprint indexing unit for performing the operation of FIG. 7. FIG.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

In order to know the broadcast program ID and the broadcast time point information by using the fingerprinting technique, the minutiae points of the corresponding broadcast program should be extracted in advance and constructed as a database (DB) before the minutiae (fingerprint) search. For this reason, it has been difficult to use the fingerprinting technique in an actual broadcast. In order to use the fingerprinting technique in an actual broadcast, it is a method used in a normal broadcast system to add a time delay element in consideration of the time taken to extract a fingerprint of an actual broadcast content and store the same in a database (DB).

According to an embodiment of the present invention, a time delay element of a broadcasting system can be eliminated by quickly responding to real-time broadcasting using a memory buffer database. Also, by introducing the dynamic loading function of the fingerprint database, it is possible to improve the ability of the system to deal with the system. This will be described in detail below.

1 is a block diagram schematically illustrating a configuration of an actual broadcast information providing system according to an embodiment of the present invention. 1, an actual broadcasting information providing system according to an embodiment of the present invention may include a client device 10, a broadcasting station legacy system 100, and a fingerprinting service system 200.

1, a broadcasting station legacy system 100 includes a broadcasting unit 110, a programming / transmission operating system 120, a contents management system 130, a content management system 130, an on- / Sound source output system 140. The broadcast transmitting unit 110 transmits the broadcast program contents as broadcast signals through a wired or wireless network. The broadcast transmitting unit 110 may include a wireless antenna. The broadcast transmission unit 110 transmits a broadcast signal in a frequency band suitable for TV broadcast. The programming information / transmission operation unit 120 generates TV programming information and determines how to broadcast the program. For example, it is possible to determine the schedule of the TV program for one day, and to generate TV program information in accordance with the determined schedule, and to jet the program to a local cable broadcasting station, a newspaper company or the like. It is not necessary to determine a day's schedule, but a weekly or more unit TV program schedule can be determined. Further, in connection with the determination of the broadcasting transmission method, it is possible to decide whether to transmit through the wireless broadcasting network or through the communication network in the form of IPTV or cable broadcasting. The CMS 130 manages broadcast-related contents. The CMS 130 generates content metadata. For example, metadata indicating a specific content may be generated to indicate which data is related to which content. The on-air video / audio source output unit 140 outputs data related to the broadcast content. The on-air video / sound source output unit 140 can be controlled by the CMS 130. [ The broadcast data may include video data and audio data.

The client device 10 is a device that receives a broadcast signal from a broadcast station and consumes the content. The client device 10 can receive a broadcast signal and display content related to the broadcast signal. The client device 10 may include a smart device 10-1 (which may include a smart phone, a smart pad, etc.), a smart TV 10-2, a PC, and a tablet device. But is not necessarily limited thereto. The user of the client device 10 may want to know about the program to watch. At this time, the program recognition request for the program to be watched may be transmitted to the fingerprinting service system 200. A user captures a specific screen of a program to be watched, generates a program recognition request through a capture screen, and transmits the program recognition request to the fingerprinting service system 200. At this time, the user of the smart device may capture a screen displayed directly on the smart device 10-1 to generate a program recognition request, capture a screen displayed on the TV 10-2, and generate a program recognition request It is possible. Here, the screen capture may capture one screen frame, or may capture image data for a predetermined period (e.g., several seconds). Alternatively, the client device 10 analyzes a broadcast signal viewed by the client device 10 and generates and transmits a program recognition request based on information that the broadcast signal is transmitted from a specific broadcasting station at a specific time. For example, when a user is watching a broadcast of a channel 12 broadcasting station at around 10:00 PM and transmits a program recognition request, the program recognition request may include information indicating 10:00 PM and viewing of the channel 12 broadcast. In addition, the client device 10 may view real-time broadcasts (broadcasts currently being broadcast from the broadcast station) or broadcast data provided by broadcast-content providing services such as VOD service . The program recognition request may include real-time broadcast indicator information indicating whether the current client device 10 is related to real-time broadcast reception or broadcast data provided by a content providing service such as a VOD service after real-time broadcast . In addition, the program recognition request may include information related to the target search speed.

The program recognition request may include information for searching for program related information (e.g., program identification information) related to the captured image as described above, and returning the search result. In addition, the program recognition request may include information to return the program-related additional information based on the search result.

The fingerprinting service system 200 includes a fingerprint web server 210, a fingerprint matching unit 220, a fingerprint indexing unit 230, a fingerprint service manager 240 and a fingerprint service database 250 . The fingerprinting service system 200 and the fingerprinting service database 250 are configured to provide additional information so that the fingerprinting service system 200 in some cases (e.g., providing only program identification information) (210), a fingerprint matching unit (220), and a fingerprint indexing unit (230).

First, the fingerprint indexing unit 230 extracts fingerprints (feature points) from broadcast data that are on-aired (broadcast in real time) and uses the data for future search And stores them in the databases 234 and 236. The fingerprint indexing unit 230 may perform a search according to a search request from the fingerprint matching unit 220. The fingerprint indexing unit 230 identifies the program by comparing the image data included in the program recognition request based on the fingerprint. At this time, the unit for recognizing the program is preferably 3 seconds. It is preferable to accumulate the search results in units of 1 second in detail. For example, it is judged that the program is identified if the search results are all the same by judging three times in units of one second. This may determine that the program has been recognized if the two consecutive recognition results are the same in some cases.

FIG. 2 is a detailed block diagram illustrating a fingerprint indexing unit 230 of an actual broadcast information providing system 200 according to an exemplary embodiment of the present invention. 2, the fingerprint indexing unit 230 according to an embodiment of the present invention may include a registration unit 232, a memory buffer database 234, and a fingerprint distribution database 236. [

2, the registering unit 232 receives the on-air broadcast data from the on-air video / sound source output unit 140, and extracts and registers finger prints (feature points). The registration unit 232 may include a real-time extractor 231 and a fingerprint registration unit 233. [ The real-time extractor 231 performs a function of extracting a fingerprint in real time from on-air data broadcast in real time. It may be desirable to extract the fingerprint as a minimum unit to recognize the program at a minimum. The real-time extractor 231 can use a plurality of fingerprint extraction algorithms, which are controllable via settings. Preferably, it may be desirable to use a fingerprint algorithm that can minimize the fingerprint extraction time. The fingerprint register 233 registers / stores the extracted fingerprints in the databases 234 and 236.

Registration in the databases 234 and 236 is simultaneously registered in the memory buffer database 234 and the main fingerprint distribution database 236. The reason why the two databases 234 and 236 are registered at the same time is that the memory buffer database 234 corresponds to a program recognition request of a real time broadcast user and the fingerprint distribution database 236 corresponds to a VOD service after real- . The memory buffer database 234 resides in memory and stores the fingerprints for a particular program in a serial manner and enables them to be searched at a high speed. Accordingly, the time delay problem is important in response to the program recognition request of the real time broadcast user, and the fingerprint registration unit 233 stores the on-air data in the memory buffer database 234 because quick search is required. The memory buffer database 234 may be preferably implemented in a RAM (Random Access Memory) area of the main memory in order to reduce the time for registration. In particular, when there are a lot of people accessing a specific program at the same time, queries having a certain sequence pattern are concentrated. Therefore, the memory buffer database 234 is configured in the RAM using the above-described characteristics to function as a database. The memory buffer database 234 may be a memory database implemented in the RAM and may have basic functions (registration, deletion, retrieval, updating, and the like) of the database. Since the capacity of the memory buffer database 234 is also limited, existing stored data not related to a real-time broadcast program can be deleted. In some cases, when a real-time broadcast program-related program recognition request rushes, a plurality of memory buffer databases 234 may be used to process a search request.

The fingerprint registration unit 233 stores the fingerprint data in the main fingerprint distribution database 236 at the same time as registering in the memory buffer database 234. [ The fingerprint distribution database 236 distributes and stores the database data systematically into a plurality of databases in order to improve the retrieval speed. By doing so, if the target search speed is specified as the requirement, the search speed can be increased by increasing the number of databases. Further, according to another embodiment of the present invention, the fingerprint distribution database 236 may be implemented as a memory database. For example, it may be implemented as a memory database implemented in RAM.

In addition, considering that the fingerprint distribution database 236 and the memory buffer database 234 are separately considered, since the memory buffer database 234 has a very small amount of data to be searched compared to the fingerprint distribution database 236, The search range can be narrowed, so that it is possible to cope efficiently. In addition, it is possible to cope with a large-scale query that occurs during actual broadcasting.

1, the fingerprint web server 210 displays a search request based on the program recognition request when the user makes a program recognition request while watching a desired program through the client device 10 To the fingerprint matching system 220. The fingerprint web server 210 receives the search result from the fingerprint matching unit 220 and acquires service information (additional information) from the fingerprint server database 250 based on the search result, To the device (10). The fingerprint web server 210 can play a role of exchanging information of the client device 10 and the fingerprint printing service system 200 with each other.

The fingerprint matching unit 220 receives a search request based on the program recognition request from the fingerprint web server 210 and transmits the search request to the fingerprint indexer 230. In the case of real-time broadcasting, the fingerprint matching unit 220 may first search the memory buffer database 234 and then search the fingerprint distribution database to increase the search speed.

3 is a detailed block diagram illustrating a fingerprint matching unit 220 of the real broadcast information providing system 200 according to an exemplary embodiment of the present invention. 3, a program recognition request classifying unit 222, a search requesting unit 224, and a search result receiving unit 226 may be included.

Referring to FIG. 3, the program recognition request classifying unit 222 receives a program recognition request from the fingerprint web server 210. Then, whether the received program recognition request is a program recognition request from a real-time broadcast receiver or a program recognition request from a receiver of a content providing service after a broadcast, such as a VOD, is performed. The classification may be based on real-time broadcast indicator information. Through the program recognition request classifying unit 222, the program recognition request can be divided into a real-time broadcast-related program recognition request and a real-time broadcast program recognition request, and different processes can be performed.

The search request unit 224 may perform a search by transmitting a request for a program related to a real time broadcast to the memory buffer database 234 and transmitting the request for a program after the real time to the fingerprint distribution database 236 to perform a search. The reason for performing such a process is to enable quick search in response to a real-time broadcast-related program recognition request, as described above. In response to the real-time broadcast-related program recognition request, the searching of the memory buffer database 234 is performed, and then the fingerprint distribution database is searched. The memory buffer database 234 and the fingerprint distribution database 236 that have received the search request receive the program information included in the program recognition request based on the stored fingerprint (e.g., captured image information, program time information, ) And returns the search result. The search result may include program identification information.

The search result receiving unit 226 receives the search results from the memory buffer database 234 and the fingerprint distribution database 236.

1, the fingerprint service manager 240 receives broadcast programming information and content metadata from the programming information / transmission operation unit 120 and the CMS 130 of the broadcasting station legacy system 100, Can be implemented. Also, it is possible to implement service organization information that determines which web page is to be linked according to a program based on broadcast information. The service related information generated in the fingerprint service manager 240 is stored in the fingerprint service database 250. [ Therefore, it can be used later in the fingerprint web server 210.

The fingerprint service database 250 stores service related information (additional information) generated by the fingerprint service manager 240. The fingerprint service database 250 returns additional information upon request from the fingerprint web server 210. [ The fingerprint web server 210 can request service information based on search result information (e.g., program identification information) received from the fingerprint matching unit 220. [ The fingerprint web server 210 provides the client device 10 with the additional information received from the fingerprint service database 250 and the search result information. At this time, the additional information may be provided in the form of a web page.

4 is a flowchart schematically showing a method of providing real broadcast information according to an embodiment of the present invention.

Referring to FIG. 4, first, a user requests a fingerprint web server 210 to recognize a program in order to know identification information or additional information of a broadcast program viewed by the user. The fingerprint indexing unit 230 receives the on-air data from the on-air image / sound source output unit 140 and extracts the fingerprint in real time and outputs the extracted fingerprint to the memory buffer database 234 and the fingerprint distribution database 236). At this time, the memory buffer database 234 can be configured in the RAM area of the main memory, and data can be retrieved quickly because it resides in the memory. The fingerprint service manager 240 receives the broadcast program information and the content metadata from the programming information / transmission operation unit 120 of the broadcasting station legacy system 100 and the CMS 130 to generate service related information, And stores it in the fingerprint service database 250.

First, the fingerprint web server 210 receives the program recognition request information from the client device 10 (S410). Upon receiving the request, the fingerprint web server 210 transmits the request information to the fingerprint matching unit 220 (S420). The fingerprint matching unit 220 determines whether the request is a request related to real-time broadcast reception (S430), and classifies the program request. If it is a real-time broadcast related program recognition request, the fingerprint search is performed by referring to the memory buffer database 234 (S440). As described above, in the case of a real-time broadcast related program recognition request, the time delay must be minimized, and therefore, the memory buffer database 234 is used for fast searching. On the other hand, the real-time program recognition request refers to the fingerprint distribution database 236 and performs a search (S435). Then, the fingerprint matching unit 220 transmits the search result to the fingerprint web server 210. The fingerprint web server 210 transmits a request for obtaining additional information to the fingerprint service database 250 using the search result, and retrieves and extracts additional information from the fingerprint service database 250 (S460) . Then, the search result and the obtained additional information are transmitted to the client device 10 (S470).

FIG. 5 is a conceptual diagram for explaining conditions for taking a broadcast service time delay D for implementing a fingerprinting service to zero.

Referring to FIG. 5, the broadcast time delay time D _t experienced by the user is calculated by summing the time (d ₁ ) required for the broadcast signal to reach the TV receiver and the sum of the program recognition time (d ₂ ) do. This can be expressed as follows.

[Equation 1]

D _t = d ₁ + d ₂

Here, d ₁ is a broadcast signal delay time, which is generally known to be 1 second or less in the case of terrestrial waves. d ₂ is the fingerprint minimum program recognition unit, which usually takes about 3 seconds. This number may vary depending on the fingerprinting algorithm.

There is an indexing function as a function that is indispensable for performing fingerprinting service. In the fingerprint service system 200 of the present invention, the indexing operation is performed in the fingerprint indexing unit 230. The time required for indexing must be guaranteed. The total sum (T _t ) of the indexing times for the on-air data is the sum of the fingerprint extraction time (t ₁ ) and the fingerprint registration time (t ₂ ). This can be expressed as follows.

&Quot; (2) "

T _t = t ₁ + t ₂

Here, t ₁ is the fingerprint extraction time, which is the time taken to extract the minimum unit fingerprint data for recognizing the minimum program. According to experiments, this takes about 0.8 seconds. This number may vary depending on the fingerprinting algorithm. The minimum program recognition unit of the fingerprinting algorithm of the fingerprint service system 200 of the present invention is 3 seconds, and the search results are cumulatively determined in units of 1 second in detail. That is, if it is determined that the program has been detected three times in units of one second, it is determined that the program is recognized. t ₂ is the time for registering the fingerprint extracted at t ₁ to the indexing system. In the present invention, since the memory buffer database 234 is implemented in the RAM area of the main memory, the registration time is very short. It is usually less than 1 second. Thus, the total indexing time (T _t ) is possible to be less than 1.8 seconds. When there are many people who access a specific program at the same time, queries having a constant sequence pattern are concentrated. The memory buffer database 234 is configured in the RAM using this feature and configured to have a database function.

If the broadcast time delay (D _t ) is greater than the sum of the total indexing time and the first recognizable time, the broadcast station time delay (D) can be taken to zero. This can be expressed as follows.

&Quot; (3) "

_Dt ? _Tt + T? _I

Here, T _i represents the minimum recognizable time. According to the experimental results, it is possible to realize a system with no delay (D = 0) if D _t is 4 seconds, T _t is 1.8 seconds, and T _i is 2.2 seconds or less. If the fingerprint recognition unit is set to 1 second and the two consecutive recognition results are the same, the program can be recognized so that it is possible to provide the service without delay in real time broadcasting.

FIG. 6 is a diagram illustrating a case where a large-scale query is generated in the fingerprinting service system 200. FIG.

Referring to FIG. 6, when a large-scale query is generated in the fingerprinting service system 200, a lot of people watch the specific program. As shown in FIG. 6, this corresponds to on-air broadcasting in a broadcasting station, re-broadcasting in air, and transmission in a cable TV. In this case, the user should implement the system in the best condition to get smooth service. The system implemented in this way has many inefficiencies. Most VOD services are not related to real broadcasting, and the number of contacts is randomly generated. In FIG. 6, the portion indicated by the VOD service represents the average number of contacts that are generated during the VOD service. Since the average figure differs from the peak value in the broadcast several tens of times, the system construction cost is also proportional to this.

FIG. 7 is a diagram illustrating an operation for solving a large-scale query of FIG. 6 in an actual broadcast information providing system according to an embodiment of the present invention.

Referring to FIG. 7, the fingerprint indexing system has a query monitor function, and can analyze a query pattern. If many queries occur at the same broadcast time for the same program, this can be judged as one of the three cases mentioned in FIG. If it is determined that the query is a program contact large-volume query such as a query analysis result, the corresponding fingerprint distribution database 236 can dynamically load the database information into the memory buffer database 234. At this time, a memory buffer database 234-2 other than the existing memory buffer database 234-1 may be used, and the fingerprint distribution database 236 may transmit the database information to the newly provided memory buffer database 234-2 It can be loaded dynamically. This can improve fingerprint search speed.

FIG. 8 is a detailed block diagram specifically illustrating a configuration of the fingerprint indexing unit 230 for performing the operation of FIG. 8, the fingerprint indexing unit 230 may further include a query analysis unit 237, a determination unit 238, and a control unit 239. [

Referring to FIG. 8, the query analyzing unit 237 analyzes a search request received from the fingerprint matching unit 220. For example, if there are a lot of queries at 7:28 pm for a program broadcast from 7:00 pm to 8:00 pm, it is likely to be during broadcast, rebroadcast, or cable retransmission. Accordingly, a query pattern generated for a specific broadcast time point in a specific program is analyzed based on a search request from the fingerprint matching unit 220.

The determination unit 238 compares the amount of the analyzed query with a threshold value. The threshold value can be changed by setting. If the amount of the query is larger than the threshold value, it is determined as a large-capacity query, and control such as dynamic loading is required. The determination unit 238 compares the amount of the query and transmits the comparison result information to the control unit 239.

The control unit 239 receives the comparison result information and stores the minutiae point related data in the fingerprint distribution database 236 in a new memory buffer 234-1 in correspondence with the point- And controls the dynamic loading to the database 234-2. The control unit 239 can control the dynamic movement to the memory buffer database 234-2 by switching a part of the hard disk (HDD) of the main memory into the RAM and moving the data in the hard disk to the RAM. The control unit 239 can control to dynamically load data of a specific portion of the distributed database 236 into the newly added memory buffer database 234-2.

The fingerprint matching unit 234 may send a search request not only to the existing memory buffer database 234-1 but also to the newly added memory buffer database 234-2 to improve the search speed.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the inventions as defined by the following claims It will be understood that various modifications and changes may be made thereto without departing from the spirit and scope of the invention.

Claims (20)

A fingerprint indexing unit for extracting minutiae points of broadcast data and constructing a database for use in retrieval;
Receiving a program recognition request from a client device and transmitting the received program recognition request to the fingerprint matching unit, receiving a search result related to the program recognition request from the fingerprint matching unit, and transmitting the received search result to the client device; And
And a fingerprint matching unit for performing a search using the fingerprint indexing unit based on the program recognition request, extracting the search result, and transmitting the extracted search result to the fingerprint web server,
A registration unit for extracting a feature point from the broadcast data and storing data related to the extracted feature point in a database; And
And a database for storing the minutiae point related data,
And a memory buffer database for storing the minutiae point related data in memory and continuously storing minutia point related data for a specific program,
Wherein the memory buffer database is for real-time responding to a real-time broadcast recognition request from the client device, and stores the minutia related data extracted from the real-time broadcast program.
2. The method of claim 1, wherein the database
And a fingerprint distribution database for distributing and storing the minutiae point related data systematically in a plurality of databases.
3. The method of claim 2,
Wherein the memory buffer database is configured in a RAM (Random Access Memory).
3. The apparatus of claim 2, wherein the register
And stores the feature point related data in the memory buffer database and the fingerprint distribution database at the same time.
3. The apparatus of claim 2, wherein the fingerprint matching unit
Performing search using the memory buffer database in response to a program recognition request of a client device that receives real-time broadcast data,
Wherein the search is performed using the fingerprint distribution database in response to a program recognition request related to a service after real-time broadcasting.
The method of claim 1, wherein the fingerprint web server
Extracting additional information related to the search result from the fingerprint service database based on the search result, and providing the additional information to the client device.
The method according to claim 6,
A service web page according to a specific program is generated using the broadcast programming information and the content meta data to generate the service information for determining the web page to be linked based on the broadcast information, Further comprising a fingerprint service manager for storing the fingerprint information in a database.
3. The apparatus of claim 2, wherein the fingerprint indexing unit
Further comprising a query analyzer for analyzing a query pattern generated for a specific broadcast time in a specific program based on the program recognition request.
9. The apparatus of claim 8, wherein the fingerprint indexing unit
A determination unit for determining whether a query generated for a specific broadcast time point for the same program is greater than a threshold value;
And a controller for dynamically loading minutiae point related information in the fingerprint distribution database into the memory buffer database in response to a specific point in time query generated more than a threshold based on a determination result of the determination unit. Information providing system via fingerprint.
Receiving a program recognition request from a client device and transmitting the received program recognition request to a fingerprint matching unit;
Extracting a minutiae of the broadcast data and constructing a database so that it can be used for the search;
The fingerprint matching unit performing a search using the fingerprint indexing unit based on the program recognition request;
Extracting a search result from the search performing step and transmitting the extracted search result to the fingerprint web server; And
Receiving the search result from the fingerprint matching unit and transmitting the search result to the client device,
Wherein the fingerprint indexing unit extracts feature points from the broadcast data and stores data related to the extracted feature points in a database,
Wherein the fingerprint indexing unit includes a memory buffer database for storing the minutiae point related data in a memory to continuously store minutia point related data for a specific program,
Wherein the memory buffer database is for real-time responding to a real-time broadcast recognition request from the client device, and stores the minutia related data extracted from the real-time broadcast program.
11. The apparatus of claim 10, wherein the fingerprint indexing unit
And a fingerprint distribution database for distributing and storing the minutiae point related data systematically in a plurality of databases.
12. The method of claim 11,
Wherein the memory buffer database is configured in a RAM (Random Access Memory).
12. The method of claim 11, wherein storing
And storing the feature point related data in the memory buffer database and the fingerprint distribution database at the same time.
12. The method of claim 11,
Performing a search using the memory buffer database in response to a program recognition request of a client device receiving the real-time broadcast data; And
And performing a search using the fingerprint distribution database in response to a program recognition request related to a service after real-time broadcasting.
11. The method of claim 10,
Further comprising extracting additional information related to the search result from the fingerprint service database based on the search result and providing the extracted additional information to the client device.
16. The method of claim 15,
In order to provide the additional information, the fingerprint service manager generates a service web page according to a specific program using the broadcast programming information and the content metadata, generates service information for determining a web page to be linked based on the broadcast information And storing the fingerprint information in the fingerprint service database.
12. The method of claim 11,
Further comprising the step of the fingerprint indexing unit analyzing a query pattern generated for a specific broadcast time point in a specific program based on the program recognition request.
18. The method of claim 17,
Determining whether a query generated for a particular broadcast time for the same program is greater than a threshold; And
And controlling the dynamic loading of the minutiae point related information in the fingerprint distribution database to the memory buffer database in response to the specific time point query generated more than the threshold based on the determination result in the step of determining whether the minutia point related information is larger than the threshold value Wherein the information is provided by a fingerprint.
The method includes the steps of: extracting a minutiae point of broadcast data and constructing a database for use in a search; receiving a program recognition request from a client device, performing a search using the database based on the program recognition request, A fingerprinting service system for extracting a result and transmitting the extracted result to the client device;
A client device for transmitting the program recognition request to the fingerprinting service system; And
And a broadcast station system for transmitting the broadcast data to the fingerprinting service system, wherein the fingerprinting service system comprises:
A registration unit for extracting a feature point from the broadcast data and storing data related to the extracted feature point in a database; And
And a database for storing the minutiae point related data, wherein the database includes a memory buffer database for storing the minutia point related data in a memory to continuously store minutia point related data for a specific program,
Wherein the memory buffer database is for real-time responding to a real-time broadcast recognition request from the client device, and stores the minutia related data extracted from the real-time broadcast program.
20. The system of claim 19, wherein the database
And a fingerprint distribution database for distributing and storing the minutiae point related data systematically in a plurality of databases.

Images