KR102016674B1

KR102016674B1 - Receiving device for providing hybryd service and method thereof

Info

Publication number: KR102016674B1
Application number: KR1020120133017A
Authority: KR
Inventors: 박홍석; 윤국진; 이재준; 이진영; 임현정; 장용석; 정원식; 주유성; 허남호; 황승오
Original assignee: 한국전자통신연구원
Priority date: 2012-04-13
Filing date: 2012-11-22
Publication date: 2019-08-30
Also published as: KR20130116157A

Abstract

The receiving device is disclosed. The apparatus includes a first receiver for receiving a first signal including signaling data and signaling information through a radio frequency broadcast network, a controller for detecting a plurality of additional data source information from the signaling information, and an IP communication network ( A second receiver for receiving a second signal including additional data from a source apparatus corresponding to each of the plurality of additional data source information, the reference data included in the first signal, and the second signal. And a storage unit for storing the additional data included therein and an output unit for processing and synchronizing and outputting the reference data and the additional data, respectively. Accordingly, different signals can be synchronized and output.

Description

Receiving device that provides hybrid service and hybrid service providing method {RECEIVING DEVICE FOR PROVIDING HYBRYD SERVICE AND METHOD THEREOF}

The present invention relates to a receiving apparatus and a hybrid service providing method, and more particularly, to a receiving apparatus and a hybrid service providing method for providing a hybrid service for processing and synchronizing and outputting a plurality of signals received through different paths. will be.

With the development of electronic technology, various types of electronic devices have been developed and spread. Representative examples of these electronic devices include a receiving device such as a TV.

Recently, as the performance of TV is improved, even multimedia contents such as 3D content or full HD content are being serviced. This type of content has a larger data size than existing content.

However, the transmission bandwidth used in the broadcasting network is limited. Therefore, there is a limitation in the size of content that can be transmitted in the current broadcasting network. In order to meet this constraint, it is necessary to reduce the resolution inevitably, there is a problem that the image quality deteriorated accordingly.

In order to solve this problem, there have been attempts to provide various types of media data through various transmission environments. However, since these data are transmitted through different paths, it is not known whether the data is related to each other on the receiving device side, and thus there is a problem in that they cannot be properly synchronized.

Therefore, there is a need for a method that can properly synchronize these contents.

SUMMARY OF THE INVENTION The present invention is directed to the above-described necessity, and an object of the present invention is to provide a receiving apparatus for providing a hybrid service for receiving, synchronizing and outputting a plurality of signals transmitted through different networks and a hybrid service providing method thereof. .

According to an aspect of the present invention, there is provided a receiving apparatus comprising: a first receiving unit configured to receive a first signal and signaling information including reference data through a radio frequency broadcast network, the signaling; The controller detects a plurality of additional data source information from the information, accesses an IP protocol network, and receives a second signal including additional data from a source device corresponding to each of the plurality of additional data source information. And a second receiver configured to store the reference data included in the first signal and the additional data included in the second signal, and an output unit configured to process and synchronize and output the reference data and the additional data, respectively. do.

Here, the signaling information may include first additional data source information for designating a source of at least one first additional data constituting first content and at least one second constituting second content to be output following the first content. 2 includes second additional data source information for designating a source of additional data, and signaling information corresponding to each content may be repeatedly received in the same form while the corresponding content is output.

The controller receives a second signal including the second additional data by accessing the at least one second additional data source using the second additional data source information while the first content is output. The second receiver may be controlled to store the second additional data, and the second additional data may be stored in the storage unit.

Here, the signaling information includes a designation value for designating a hybrid service synchronization type, and the control unit controls the output unit to synchronize and output the reference data and the additional data by using the synchronization information corresponding to the designation value. Can be.

The specified value is included in a first value that specifies synchronization using PTS (Presentation Time Stamp) information of each of the reference data and the additional data, and an elementary stream (ES) of each of the reference data and the additional data. And a third value for designating to use the synchronized time code provided through a private data stream included in each of the first signal and the second signal. have.

The output unit may select unit data to be synchronized by comparing the PTS of the reference data and the PTS of the additional data when the specified value is the first value, output the selected unit data, and the specified value is the If the second value, the first time code included in the ES of the reference data and the second time code included in the ES of the additional data is selected to select the unit data to be synchronized, and the time stamp of the selected unit data is the same value To correct the time stamp, perform a synchronization output on the corrected time stamp, and if the designated value is the third value, detect synchronization information included in the private data stream, and perform synchronization output according to the detected synchronization information. have.

The signaling information may include a number of hybrid services provided using the signaling information, a hybrid service identifier, a hybrid service provision method, an indicator indicating a hybrid service provision time, a designated value for designating a hybrid service synchronization type, and a hybrid service. It may include the identifier of the, the temporal reproduction order of the additional data constituting the hybrid service, URL information for obtaining the additional data.

The output unit may include: a first detector configured to detect the reference data from the first signal; a first video processor configured to decode the reference data to generate a first video frame; and to detect the additional data from the second signal. A second detector may include a second video processor to decode the additional data to generate a second video frame, and a renderer to perform rendering by combining the decoded first video frame and the second video frame. Herein, the controller selectively controls at least one of the first detector, the second detector, the first video processor, the second video processor, and the renderer according to the specified value, so that the reference data and the Synchronization processing of additional data can be performed.

The controller may include at least one of a reserved area, a PMT descriptor area, a program and system information protocol virtual channel table (PSIP VCT), an EIT reserved, an EIT descriptor area, and a private stream in the PMT of the first signal. The signaling information can be detected.

On the other hand, according to an embodiment of the present invention, in the hybrid service providing method of the receiving device, receiving a first signal and signaling information including reference data through a radio frequency broadcast network (Radio Frequency broadcast network), from the signaling information Detecting a plurality of additional data source information, accessing an IP protocol network, and receiving a second signal including additional data from a source device corresponding to each of the plurality of additional data source information; And detecting, processing, synchronizing and outputting the reference data and the additional data from the first signal and the second signal, respectively.

Here, the signaling information may include first additional data source information for designating a source of at least one first additional data constituting first content and at least one second constituting second content to be output following the first content. The second additional data source information for designating the source of the additional data may be included. The signaling information corresponding to each content may be repeatedly received in the same form while the corresponding content is output.

In addition, when the second signal including the second additional data is received based on the second additional data source information while the first content is output, storing the second additional data. When the event to be output occurs, the method may further include outputting the second content by using the stored second additional data.

The signaling information includes a designation value for designating a hybrid service synchronization type, and the synchronization may be performed according to the synchronization information corresponding to the designation value.

The predetermined value may be included in a first value that specifies synchronization using PTS (Presentation Time Stamp) information of each of the reference data and the additional data, and an elementary stream (ES) of each of the reference data and the additional data. And a third value for designating to use the synchronized time code provided through the private data stream included in each of the first signal and the second signal. .

The outputting may include selecting unit data to be synchronized by comparing the PTS of the reference data and the PTS of the additional data when the predetermined value is the first value, and outputting the selected unit data. If the value is the second value, the unit data to be synchronized is selected by comparing the first time code included in the ES of the reference data with the second time code included in the ES of the additional data, and the time stamp of the selected unit data. Correcting to the same value, and performing a synchronization output on the corrected time stamp. If the predetermined value is the third value, the synchronization information included in the private data stream is detected and synchronized according to the detected synchronization information. It may include the step of performing.

Here, the signaling information may include the number of hybrid services provided using the signaling information, a hybrid service identifier, a hybrid service provision method, an indicator indicating a hybrid service provision time, a designated value for designating a hybrid service synchronization type, and a hybrid service. It may include the identifier of the, the temporal reproduction order of the additional data constituting the hybrid service, URL information for obtaining the additional data.

The signaling information may include at least one of a reserved area in the PMT of the first signal, a PMT descriptor area, a PSIP VCT (Program and System Information Protocol Virtual Channel Table), an EIT reserved, an EIT descriptor area, and a private stream. Can be detected from.

As described above, according to various embodiments of the present disclosure, a hybrid service for separately receiving and outputting additional data for each of a plurality of contents using signaling information may be provided.

1 is a block diagram showing the configuration of a transmission and reception system according to an embodiment of the present invention;
2 is a block diagram illustrating an example of a configuration of an output unit used in a receiving apparatus according to an embodiment of the present disclosure;
3 is a diagram illustrating a configuration example of signaling information;
4 is a view for explaining a signaling information providing method according to an embodiment of the present invention;
5 is a table illustrating types of hybrid service synchronization type designation values included in signaling information;
6 is a diagram illustrating an example of syntax of signaling information;
7 is a table indicating types of some variables included in signaling information;
8 is a view for explaining a synchronization information providing method according to an embodiment of the present invention;
9 is a diagram illustrating an example of a synchronization information structure when additional data is provided in a TS form;
10 is a diagram illustrating an example of a synchronization information structure when additional data is provided in the form of an MP4 file;
11 is a view for explaining another method of transmitting signaling information;
12 is a diagram illustrating an example of a structure of an EIT in which linkage_info_descriptor () is defined;
13 is a flowchart illustrating a hybrid service providing method according to an embodiment of the present invention;
14 is a flowchart for explaining a process of processing additional data;
15 to 17 illustrate a configuration and operation of a transmission system according to various embodiments of the present disclosure.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of a transmission and reception system according to an embodiment of the present invention. According to FIG. 1, a transmission / reception system includes a plurality of transmission devices 1 and 2 (100-1 and 100-2) and a reception device 200.

The transmitting apparatuses 1 and 2 100-1 and 100-2 transmit different signals through different paths. For example, the transmitting apparatus 1 100-1 transmits a first signal through an RF frequency network, and the transmitting apparatus 2 100-2 transmits a second signal through an IP protocol network. You can send a signal.

The first signal and the second signal each include different data constituting one multimedia content. That is, one content may include data included in the first signal and data included in the second signal. For example, in the case of 3D content, the left eye image and the right eye image may be included in the first signal and the second signal, respectively. Alternatively, the video signal may be divided into video data and audio data, or may be divided into video data, subtitle data, and other data, and may be included in the first signal or the second signal, respectively. In the present specification, data included in the first signal is referred to as reference data, and data included in the second signal is referred to as additional data. The receiving device 200 outputs one piece of content by combining the reference data included in the first signal and the additional data included in the second signal. In this specification, a service that processes data using data transmitted through different paths together is called a hybrid service. That is, the processing capacity of the existing broadcast transmission / reception system may be limited in order to service 3D content or high definition content having a large size. Accordingly, a hybrid service may be provided to be compatible with existing broadcasting systems and reception apparatuses, and to serve contents of a large size. In the case of a hybrid service, since a single content is provided using a plurality of different networks, a large sized content can be processed. In the present specification, a case in which a broadcast network and an IP network are used together will be described as an example. However, the type and number of networks may be variously implemented.

Referring to FIG. 1, the receiving device 200 according to an embodiment of the present invention may include a first receiving unit 210, a second receiving unit 220, an output unit 230, a control unit 240, and a storage unit 250. Include. The reception device 200 may be various types of devices such as a TV, a mobile phone, a tablet PC, a laptop computer, a PC, a kiosk, a set top box, a billboard, and the like.

The first receiver 210 receives a first signal and signaling information including reference data through a radio frequency broadcast network. The signaling information includes information on additional data to be processed together with the reference data included in the first signal, and various kinds of information related to the hybrid service using the reference data and the additional data. The content, structure, and transmission method of the signaling information will be described in detail later.

The controller 240 controls the overall operation of the receiving device 200. Specifically, when the signaling information is received through the first receiver 210, the controller 240 detects a plurality of additional data source information from the signaling information.

The additional data source information refers to address information of a source providing additional data. That is, one content may be divided into a plurality of units. If the data transmitted through the broadcasting network among the divided data is referred to as reference data (or basic data or main data), the data transmitted through the IP network may be referred to as additional data. The additional data may be transmitted in the form of a real time transport stream or an MP4 file.

When implemented in the form of a real-time transport stream, the second signal including the additional data may be transmitted and received using a protocol such as RTP or HTTP. In the case of using HTTP, a metadata file must be provided to obtain a second signal. The metadata is information for indicating where the multimedia content can be received. The metadata file may include information that the client needs to know in advance, such as a location on a content time corresponding to each of the plurality of separate files, a URL of a source for providing the file, and a size. Metadata files can be classified in various ways depending on the type of HTTP-based streaming. That is, in the case of a smooth streaming method, an ism (Internet Information Service (IIS) Smooth streaming Media) file is used as a metadata file. In case of Internet Engineering Task Force (IETF) HTTP live streaming, m3v8 file is used as metadata file, and adaptive HTTP streaming Rel adopted by 3GPP. 9 or adaptive HTTP streaming Rel employed in OIPF. 2, in the case of the dynamic adaptive streaming over HTTP scheme employed in MPEG, MPD (Media Presentation Description) may be used as the metadata file. The additional data source information may be address information about a source from which such metadata file can be obtained.

On the other hand, the additional data constituting one content may be composed of a plurality of streams or files. Since such additional data must be processed and synchronized with the reference data and output in synchronization, the additional data must be able to be downloaded in real time or in advance. Accordingly, in the present embodiment, the transmission apparatus 1 (100-1) that transmits the reference data may provide signaling information including a plurality of additional data source information together.

Although the signaling information has been described as being provided together, it is not necessarily implemented as described above, and the signaling information may be provided from a separate device.

The second receiver 220 accesses an IP protocol network and receives a second signal including additional data from a source device corresponding to each of the plurality of additional data source information. In the embodiment of FIG. 1, although a transmission device 2 (100-2) as one source device is illustrated, additional data may be provided from a plurality of different source devices. The second signal may be transmitted by a real time streaming method or a file downloading method.

The controller 240 may control the second receiver 220 to directly access the source device of the additional data by using the additional data source information included in the signaling information. Alternatively, when the additional data source information is source information for the metadata file, the controller 240 detects source information from a signal received through the second receiver 220 and accesses the source to obtain a metadata file. After that, the second receiver 220 may be controlled to receive the second signal using the metadata file.

The first signal may include first synchronization information along with first data, that is, reference data, and the second signal may include second synchronization information along with second data, that is, additional data. Various information may be used as the first and second synchronization information. Specifically, the synchronization information may be an SMPTE time code, a frame number, and various other information according to its form, and may be classified into a video level, an ES level, a PES level, and the like according to a provided method.

The transmitting apparatuses 1 and 2 100-1 and 100-2 may provide synchronization information in various forms and methods according to the type of content, a transmission environment, an encoding method, and a size of the content. The transmitting device 1 (100-1) may include various pieces of information necessary for hybrid service, including synchronization information, in signaling information and transmit the same.

The storage unit 250 stores the reference data included in the first signal and the additional data included in the second signal, respectively. The storage unit 250 includes various storage means such as a flash memory, an HDD, a RAM, a ROM, a buffer, and the like. When the second signal is transmitted in a real time streaming method, an operation of buffering the corresponding stream using a buffer in the storage unit 250 is performed. The controller 240 may process the buffered stream to extract additional data.

The controller 240 provides a hybrid service by using signaling information. Specifically, the signaling information may include not only information about additional data of content currently being output, but also information about additional data of content to be output next. In this case, the control unit 240 receives the additional data of the next content in advance by using the additional data source information of the next content and stores it in the storage unit 250.

As an example, the signaling information may include first additional data source information for designating a source of at least one first additional data constituting the first content and at least one first component constituting second content to be output following the first content. 2, if the second additional data source information for specifying the source of the additional data is included, the controller 240 uses the second additional data source information while the first content is being output, to determine at least one of the second additional data. The second receiver 220 is controlled to access a source and receive a second signal including the second additional data. Accordingly, when the second signal is received, the signal or the second additional data detected from the signal is stored in the storage 250.

Meanwhile, the signaling information may be repeatedly transmitted in the same form while one content is being output. Accordingly, even if the receiving device 200 is turned on after receiving the content and starts receiving the content, the receiving device 200 may secure and process additional data using the received signaling information.

The output unit 230 processes and synchronizes the reference data and the additional data stored in the storage unit 250 and outputs the synchronized data. The output unit 230 may include a plurality of data processing units for synchronization output. That is, the output unit 230 may not only provide various components for processing and displaying video data such as a video decoder, a scaler, a frame rate converter, a display panel, a backlight unit, but also audio such as an audio decoder, an amplifier, a noise filter, a speaker, and the like. It may also include components for processing and outputting data. These components may be provided in plural numbers according to the number of received signals. For convenience of description, hereinafter, only components for processing and displaying video data will be described and described, and the description and description thereof will be omitted with respect to audio data.

2 is a block diagram illustrating an example of a configuration of the output unit 230. According to FIG. 2, the output unit 230 includes a first detector 231, a second detector 232, a first video processor 233, a second video processor 234, and a renderer 235.

The first detector 231 detects reference data from the first signal, and the first video processor 233 is configured to process the detected reference data. When the first signal received by the first receiver 210 is in the MPEG2-TS format, the first detector 231 demuxes the received transport stream and outputs a video ES. In this case, the first video processor 233 may be implemented as an MPEG2 decoder. Accordingly, the first video processor 233 decodes the received video ES, and the decoded raw data is provided to the renderer 235.

The second detector 232 detects additional data from the second signal, and the second video processor 234 is configured to process the detected additional data. As described above, the second signal received by the second receiver 220 may be an MPEG2-TS format or a file format. MPEG2-TS refers to a transport stream that is encoded in MPEG2 encoding and then modulated and transmitted in ATSC 8VSB. In the case of a transport stream such as MPEG2-TS, the second detector 232 detects the video ES by demuxing the received transport stream and provides the same to the second video processor 234. On the other hand, in the embodiment received in the file format, the second detector 232 may be implemented as a file parser. Accordingly, the second detector 232 parses the received file and provides the parsing result to the second video processor 234. The second video processor 234 decodes the video data provided from the second detector 232 in an AVC manner and provides the decoded video data to the renderer 233. According to an embodiment, the second signal may be selectively provided in a TS or file format. In this case, the second detector 232 includes both a demux and a file parser, and may selectively use these according to a signal format to detect additional data.

The renderer 235 performs the rendering by combining the decoded first and second video frames provided by the first and second video processors 233 and 234. The controller 240 may control the video frames corresponding to each other to be rendered in combination to perform a synchronization output.

In detail, the decoded data provided by the first and second video processors 233 and 234, that is, the raw data, may include synchronization information such as a time code or a frame number in the form of a watermark. The rendering unit 233 detects the watermark type synchronization information included in each row data, and then directly synchronizes each frame based on the synchronization information, or corrects the time stamp of each frame, and then according to the time stamp. Rendering can be performed.

Synchronization may be accomplished in various ways depending on the type of synchronization. The controller 240 may determine the synchronization scheme according to the signaling information.

For example, the signaling information may include a designation value that designates a hybrid service synchronization type. The controller controls the output unit 230 to synchronize and output the reference data and the additional data by using synchronization information corresponding to a specified value.

Specifically, the designated value is a time code included in an ES (Elementary Stream) of each of the first value, the reference data, and the additional data, which specifies to synchronize using the PTS (Presentation Time Stamp) information of each of the reference data and the additional data. It may be one of a second value for specifying to use as the synchronization information, a third value for specifying to use the synchronization information provided through the first signal and the private data stream included in each of the second signal.

If the specified value is the first value, the output unit 230 compares the PTS of the reference data with the PTS of the additional data and performs a synchronous output. On the other hand, if the specified value is the second value, the output unit 230 compares the first time code included in the ES of the reference data with the second time code included in the ES of the additional data, and performs a synchronization output. The third value detects synchronization information included in the private data stream and performs synchronization output according to the detected synchronization information.

The controller 240 selectively selects at least one of the first detector 231, the second detector 232, the first video processor 233, the second video processor 234, and the renderer 235 according to a specified value. Control to perform synchronization processing of the reference data and the additional data.

In detail, the control unit 240 checks the PTS of the data processed by the first video processing unit 233 and the second video processing unit 234 and stored in a buffer (not shown) when the designated value is the first value. The rendering unit 235 is controlled to select. The renderer 235 selects unit data having the same PTS, and performs rendering.

On the other hand, if the specified value is the second value, the controller 240 controls the first video processor 233 and the second video processor 234 to detect the time code from the reference data and the additional data, respectively. The first and second video processors 233 and 234 extract the SMPTE time codes provided through the ES levels, and provide them to the renderer 235. The rendering unit 235 compares the time code of the first signal and the time code of the second signal, and performs a synchronization process so that frames having the same time code are synchronized and output.

On the other hand, if the specified value is the third value, the controller 240 controls the first and second detectors 231 and 232 to detect the synchronization information from the first signal and the second signal, respectively. The synchronization information detected by the first and second detectors 231 and 232 is provided to the renderer 235. The renderer 235 performs a synchronization output based on the synchronization information.

The renderer 235 may perform synchronization output by selecting unit data to be synchronized and correcting the time stamp to the same value. Alternatively, the rendering unit 235 may select unit data to be synchronized based on the time code or the frame number itself, and output the unit data together. In the case of video data, the unit data may be a video frame, and in the case of audio data, the unit data may be audio data of a time unit corresponding to a predetermined number of video frames.

3 illustrates an example of a structure of signaling information used in a transmission / reception system according to an embodiment of the present invention. According to FIG. 3, a plurality of reference data 31, 32, and 33 corresponding to different contents are sequentially transmitted in chronological order. Each of the reference data 31, 32, and 33 may include video data corresponding to different broadcast programs.

The reception device 200 may receive signaling information separately from the reference data. The signaling information may be received from the transmitting device 1 100-1 in the same manner as the reference data, but may be received from another new external device.

While the reference data 31 for the first content is being received, the reception device 200 receives the same type of signaling information a plurality of times 31-1 through 31-n. The structure and content of the signaling information received each time are the same. Accordingly, even if a content is turned on while one content is already being broadcast, the receiving device 200 may receive additional data using signaling information provided thereafter.

According to FIG. 3, the first signaling information 31-1 to 310-n received while the first reference data 31 is received includes first additional data source information 31-# 1 corresponding to the first reference data. ) And second additional data source information 31-# 2 corresponding to the second reference data. The first additional data source information 31-# 1 includes information about sources that provide a plurality of additional data (additional images # 1 and # 2) corresponding to the first reference data 31, respectively. The 2nd additional data source information 31-# 2 includes information about a source providing one additional data (additional image # 1).

Meanwhile, second signaling information 32-1 to 32-m received while the second reference data 32 is received includes second additional data source information 32-# 1 corresponding to the second reference data; Third additional data source information 32-# 2 corresponding to the third reference data is included. The second additional data source information 32-# 1 includes information of a source for providing one additional data corresponding to the second reference data 32 (additional image # 1), and the third additional data source information ( 32-# 2) includes information of sources providing two additional data (additional images # 1 and # 2) corresponding to the third reference data 33.

As such, each signaling information may also include additional data source information corresponding to the received reference data subsequent to the currently received reference data. The controller 240 may secure additional data source information for the next content in advance, receive the additional data first, and store the additional data in the storage 250. Accordingly, when the next content is played back, the synchronized play can be performed as soon as the reference data is received without receiving additional data.

Such signaling information may be provided to the receiving device 200 in various ways.

4 is a diagram for describing a method of providing signaling information in a transmission / reception system according to an exemplary embodiment. According to FIG. 4, when the first signal transmitted through the RF broadcasting network complies with the MPEG2 standard, in addition to the reference video V1 and audio A data, PSI / PSIP (Program Specific Information / Program and System Information Protocol), Private Section may be included. The signaling information may be recorded in the private section. The private section is defined in MPEG and is used to transmit SI (System Information), data specific to a specific application, and broadcast file system. The signaling information is transmitted through the private section, and the reception apparatus 200 may detect the signaling information recorded in the private section through section filtering. In FIG. 4, signaling information is referred to as referenced_media_information, but the name thereof may be variously called as necessary. The receiving device 200 may access the additional data source and receive the additional data V2 based on the signaling information recorded in the private section.

As shown in FIG. 4, the reference data is transmitted at different times for each broadcast program. That is, the program XX is received and output for 09:00 ~ 10:00, and the program YY is received and output for 10:00 ~ 10:50.

The signaling information may include various information necessary to implement a hybrid service. Specifically, the number of hybrid services provided using signaling information, a hybrid service identifier, a hybrid service provision method, an indicator indicating when to provide a hybrid service, a designation value for designating a hybrid service synchronization type, an identifier of the hybrid service, and a hybrid service. The signaling information may include a temporal reproduction order of the additional data configuring the additional data, additional data source information for obtaining the additional data configuring the hybrid service, and the like.

Among these, the designated value for designating the hybrid service synchronization type may be determined according to the type of the synchronization information, the insertion position of the synchronization information, and the like.

5 is a diagram illustrating an example of a designated value specifying a synchronization type. According to FIG. 5, the designated values include 0x00, 0x01, and 0x02.

The first value, ie, 0x00, is a value designated to synchronize using time stamps of the reference data and the additional data without using separate synchronization information. According to the MPEG standard, a transport stream for transmitting broadcast data includes a PCR (Program Clock Reference) and a PTS (Presentation Time Stamp). PCR refers to reference time information for adjusting the time reference to the transmitters 100-1 and 100-2 in a receiving apparatus (set top box or TV) that conforms to the MPEG standard. When the specified value is set to the first value, the control unit 240 of the receiving apparatus 200 adjusts the value of the system time clock (STC) according to the PCR. PTS refers to a time stamp indicating a reproduction time for synchronizing video and audio in a broadcast system according to the MPEG standard. In this specification, it is called a time stamp. The controller 240 checks the time stamp according to the PCR, and controls the output unit 230 to combine and output frames having the same time stamp among the video frames in the reference data and the video frames in the additional data. For example, in the case of 3D content, the reference data may include a left eye image, and the additional data may include a right eye image. If the reception apparatus 200 is an active shutter glass type, the left eye image and the right eye image having the same time stamp are alternately displayed. On the other hand, in the passive type, a left eye image and a right eye image having the same time stamp are divided into a plurality of lines, respectively, and a frame is formed by combining odd lines and even lines.

On the other hand, 0x01 is a value for designating to use the time code included in each of the elementary stream (ES) of the reference data and the additional data as synchronization information. A time code is a series of pulse signals produced by a time code generator and is a signal standard developed for easy editing management. When creating and editing content, the same time code is used for synchronized management of left and right eye images. Therefore, the time code can maintain the same pair regardless of the time of stream generation or delivery. Specifically, SMPTE (Society of Motion Picture and Television Engineers) time code may be used. That is, in SMPTE 12M, the time code is represented in the form of "hour: minute: second: frame". The SMPTE time code may be classified into a longitude time code (LTC) or a vertical interval time code (VITC) according to a recording method. In the case of the LTC, a total of 80 bits of data including time information (25 bits), user information (32 bits), synchronization information (16 bits), storage area (4 bits), and frame mode display (2 bits) may be configured. VITC is recorded on two horizontal lines within the vertical blanking period of the video signal. SMPTE RP-188 defines an interface specification that allows LTC or VITC type timecode to be transmitted as ancillary data. When the specified value is recorded as a second value, that is, 0x01, the controller 240 compares the time code recorded in each video frame of the reference data and the time code in which each video frame of the additional data is recorded. Accordingly, it is determined that video frames having time codes that match each other are video frames that match each other, and the output unit 230 is controlled to synchronize and output the video frames.

On the other hand, 0x02 is a value for designating to use the synchronization information provided through the private data stream included in the first signal and the second signal, respectively. Synchronization information such as the time code and the frame number described above may be provided through various areas such as a Program Map Table (PMT), a Program and System Information Protocol Virtual Channel Table (PSIP VCT), an Event Information Table (EIT), and the like. In addition, it may be provided as a private data stream. Here, the frame number means identification information such as a number assigned to each frame. When the specified value is recorded as the third value, that is, 0x02, the controller 240 detects synchronization information from the private data stream and uses the same to perform synchronization output.

When the specified value is the second value or the third value, the output unit 230 may directly compare the time code or the frame number and select video frames matching each other. The time stamp difference between the selected video frames may be checked, and the time stamp may be corrected according to the identified value to achieve synchronization. Alternatively, the output unit 230 may immediately perform synchronization, decoding, scaling, and rendering of a video frame based on the time code or the frame number itself, without correcting the time stamp.

Although only three designated values are shown in FIG. 5, this is only an example, and various designated values may be used according to the type of synchronization information, the recording position, and the like. For example, frame index information other than frame number, Coordinated Universal Time (UTC) information, etc. may also be used as synchronization information. Such synchronization information may be transmitted at various levels, such as a video level, an ES level, a PES level, and the like.

On the other hand, as described above, the signaling information may include a variety of information. In addition, the structure of the signaling information may also be variously implemented.

6 illustrates syntax of signaling information according to an embodiment of the present invention. According to FIG. 6, the signaling information includes num_hybrid_services, which indicates the number of hybrid services to be provided, hybrid_service_id, which indicates a delimiter for identifying hybrid services, and whether the hybrid service is provided by streaming or downloading. Hybrid_service_type indicating a hybrid service, hybrid_service_current_next_indicator, which is an indicator indicating when to provide a hybrid service, hybrid_service_sync_type indicating a specified value for specifying a synchronization type of a hybrid service, and processing in a hybrid service. Referenced_media_id, which is an identifier indicating reference media to be used, referenced_media_playback_order, which indicates the temporal playing order of additional data constituting the hybrid service, and hybrid It includes referenced_media_URI_byte information such as the URL for acquiring the addition data included in the service.

In FIG. 6, hybrid_service_sync_type may be recorded as a plurality of designated values as described with reference to FIG. 5.

FIG. 7 shows an example of a hybrid_service_sync_type and a hybrid_service_current_next_indicator among the variables described in FIG. 6. According to FIG. 7, hybrid_service_sync_type is 0 indicating a streaming type and 1 indicates a downloading type. In addition, if hybrid_service_current_next_indicator is 0, this indicates that the service is currently output, and if 1, it indicates that it is the next service to be output.

As described above, various pieces of information may be included in the signaling information and provided to the receiving device 200. The hybrid_service_sync_type value among the signaling information may be set differently according to the synchronization information to be used and its transmission location.

8 illustrates a case in which synchronization information is provided to the receiving device 200 through a separate private data stream. As described above, when hybrid_service_sync_type is 0x02, synchronization information may be transmitted through a private data stream. In this case, not only the first signal but also the second signal provide synchronization information such as a time code or a frame number through a private data stream. The receiving device 200 detects synchronization information from the private data streams of the first signal and the second signal, and synchronizes and outputs the reference data and the additional data matched with each other.

On the other hand, when the specified value is 0x00, since the reference data and the additional data are synchronized based on the PTS information, no separate synchronization information is provided. In addition, in the case of 0x01, since the SMPTE time code provided through the ES of the reference data and the additional data is used as synchronization information, synchronization information is not separately provided to the private data stream.

On the other hand, the synchronization information may be provided in different ways depending on the case where the additional data is a transport stream (TS) type and the file type.

9 shows a form of providing a second signal in the case of a transport stream type. Specifically, when hybrid_service_sync_type is 0x02, this indicates a structure of synchronization information when applied to the TS. According to FIG. 9, the media synchronization information (media_pairing_information) includes reference media, that is, referenced_media_id, which is an ID for designating reference data, and includes a media_pairing_info_type value indicating whether to use a time code or a frame number among the synchronization information. . If the media_pairing_info_type value is 0x01, synchronization is performed using the SMPTE timecode in the private data stream. If the media_pairing_info_type value is 0x02, synchronization is performed using the frame number in the private data stream.

FIG. 10 is a diagram illustrating a form of provision of a second MP4-based signal transmitted through an IP communication network. According to FIG. 10, a value for calculating a frame number using stts (Time-to-Sample Atom), ctts (Composition Time To Sample Atom), and stss (Sync Sample Atom) that provides existing frame reproduction time information in a moov header Can be provided implicitly. That is, the MP4 file provides a relative time value with respect to the playback time from the file start position, such as PTS and DTS of TS, through stts, ctts, and stss. However, the frame number does not have a specific time unit because only the relative order between frames is provided, such as # 1 and # 2. Therefore, referring to relative time values provided by stts, ctts, etc., it is possible to infer the order of frames, that is, the frame number. Alternatively, the SMPTE time code or frame number may be explicitly provided through a separate box extension. More specifically, an additional box may be defined in the ISO media base file format 14496-12, or a time code may be provided by extending a field in a predefined box. For example, a time code may be provided by extending a box of "stss (sync sample table)" that provides random access.

11 is a diagram for explaining another method of transmitting signaling information. According to FIG. 11, linkage_info_descriptor () may be defined in an event information table (EIT), and signaling information may be provided using the linkage_info_descriptor (). That is, similar signaling information may be selectively provided through the PSIP or through the EIT, or may be provided by using both of them in duplicate.

12 is a diagram illustrating an example of a structure of an EIT in which linkage_info_descriptor () is defined. According to FIG. 12, the linkage_info_descriptor () includes a referenced_media_id, which is an identifier for the hybrid service, and a referenced_media_URI_byte, which indicates URL information for acquiring additional data of the hybrid service. The receiving device 200 may obtain additional data using this information.

As described above, according to various embodiments of the present disclosure, the receiving device may be provided with a plurality of additional data source information by using signaling information. Accordingly, additional data may be provided in parallel from a source of each additional data, or additional data of content to be serviced later may be secured in advance. As a result, delays due to securing additional data can be reduced, and hybrid services using additional data of a larger data size can be supported.

13 is a flowchart illustrating a hybrid service providing method in a receiving apparatus according to an embodiment of the present invention. According to FIG. 13, the receiving device receives the first signal and the signaling information (S1310). The first signal and signaling information may be received through an RF broadcast network. The first signal includes reference data constituting one content, and the signaling information may store additional data constituting the content together with the reference data and various information about a hybrid service provided using these data. Since the type and the transmission method of the signaling information have been described in detail in the above description, redundant description is omitted.

The reception device 200 detects additional data source information from the signaling information in operation S1320. There may be a plurality of additional data source information. That is, information about at least one source of at least one additional data corresponding to the content to be output or currently output, as well as information about at least one source of additional data corresponding to the next output content may be included.

The reception apparatus 200 accesses the source apparatus using the additional data source information and receives a second signal in operation S1330. The second signal includes additional data. The reception device 200 may access a plurality of source devices to receive a plurality of additional data.

Thereafter, the reception apparatus 200 finds data to be synchronized with each other among the reference data and the additional data, and outputs the synchronized data. For example, in the case of 3D content, the left eye image and the right eye image that are to be output at the same time point are searched for, and the combination is output. On the other hand, when the reference data is video data and the additional data is audio data, display and speaker output are performed at the same timing. As the synchronization information, a time code and a frame number may be used, and in some cases, various information may be used in addition to these information. The synchronization type may also vary depending on the designated value included in the signaling information. Since it has been described in detail in the above-described portion, redundant description is omitted.

14 is a flowchart for explaining a processing method when a plurality of additional data is received. According to FIG. 14, the reception device receives signaling information together with reference data of the first content while the first content is being output. The receiving device analyzes the received signaling information (S1410) and determines whether there is additional data of the second content (S1420). As a result of the analysis, when it is determined that the second additional data source information exists in the signaling information, the receiving device accesses the source device according to the source information and receives the second signal (S1430). Accordingly, when the second signal is received, the second additional data included in the second signal is detected and stored (S1440).

After that, when the output of the first content is terminated and the event to output the second content is started (S1450). While receiving the second reference data for configuring the second content through the RF broadcast network, and outputs the second content using the stored second additional data and the second reference data together (S1460).

On the other hand, when the second additional data source information does not exist in the signaling information received during the first signal output, the reception device may not secure the second additional data in advance. In this case, when an event to output the second content occurs (S1470), the reception device receives signaling information corresponding to the second content and checks the source of the second additional data. Accordingly, the second additional data is received separately from the second reference data and synchronized with the second reference data to be output (S1480).

As described above, the reception apparatus may appropriately secure the reference data and the additional data by using the signaling information and provide a hybrid service.

Meanwhile, in various exemplary embodiments as described above, the description has been made based on the case where two contents are processed, such as the first content and the second content. However, the additional data may be provided in advance for a plurality of subsequent contents. For example, information about additional data about the third content or the fourth content may also be included in the signaling information of the first content and provided together.

15 is a diagram illustrating an example of a configuration of a transmission system for providing the hybrid service described above. According to FIG. 15, the transmission system includes a plurality of source devices 300-1 and 300-2, a transmission device 1 100-1, and a transmission device 2 100-2.

The source device 1 300-1 refers to a content server that provides pre-recorded content, and the source device 2 300-2 refers to a live source that provides content in real time. The raw video provided by the source devices 1 and 2 300-1 and 300-2 includes synchronization information in a watermark format. In FIG. 15, a time code or frame numbers such as frames # 1, # 2, and # 3, which are sequentially added to each frame, such as 01.00.00.00, 01.00.00.01, and 01.00.00.02 added to each frame, are included in the form of a watermark. Status is displayed.

In FIG. 15, the left eye low video data is provided to the transmitting device 1 100-1 from each of the source devices 1 and 2 300-1 and 300-2, and the right eye low video data is transmitted to the transmitting device 2 (100-2). The case provided by) is shown. However, this is only an example, and the data of the source apparatus 1 300-1 is provided to the transmitting apparatus 1 100-1, and the data of the source apparatus 2 300-2 is transmitted to the transmitting apparatus 2 100-2. It may be provided as.

The transmission device 1 100-1 includes an encoder 110-1, a mixer 120-1, and a modulator 130-1. The encoder 110-1 encodes the raw data using the MPEG 2 encoding scheme to generate a video elementary stream (ES), and then provides it to the video processor 120-1. The muxer 120-1 muxes the additional data for the video ES to generate MPEG2-TS (Transport Stream), and then provides it to the modulator 130-1. The modulator 130-1 performs modulation by the ATSC 8-VSB modulation method and outputs the modulated signal.

The transmission apparatus 2 100-2 includes an encoder 110-2, a processor 120-2, a file generator 130-2, and a server 140-2. The encoder 110-2 encodes the raw data in the Advanced Video Coding (AVC) scheme to generate a video ES. The encoder 110-2 may provide the video ES to the 120-2 when the content is transmitted in the TS format. Museo 120-2 muxes the additional data to the video ES and provides it to the server 140-2.

On the other hand, when transmitting in the MP4 file format, the encoder 110-2 may provide the video ES to the file generator 130-2. The file generation unit 130-2 converts the video ES into a file format and provides it to the server 140-2.

The server 140-2 stores video data provided by the processor 120-2 or the file generator 130-2. When the server 140-2 receives a video request for video data from the receiving apparatus 200, the server 140-2 streams the stored TS through the IP communication network or receives the stored file through the IP communication network according to the request. It may be provided to the device 200.

Although FIG. 15 illustrates a configuration in which a 3D video data is requested and provided from a receiving device, the same processing may be performed on 2D video data or other audio data instead of 3D video data.

In FIG. 15, signaling information including additional data source information is inserted into PMT, PSIP VCT, EIT, etc. in a TS or a file by encoder units 110-1 and 110-2 and 120-1 and 120-2. Or may be generated as a separate private stream. Such signaling information may be provided from the source apparatuses 1 and 2 300-1 and 300-2, and may be generated by the encoder units 110-1 and 110-2, and 120-1 and 120-2. Alternatively, a value stored in a separate storage unit (not shown) may be used as it is. Alternatively, the signaling information may be generated by a separately provided control unit (not shown) and provided to the encoder units 110-1 and 110-2 and the muck 120-1 and 120-2. The controller may generate signaling information suitable for the characteristics of the system and provide the signaling information to the encoder units 110-1 and 110-2 or the encoder 120-1 and 120-2.

16 is a view for explaining the configuration and operation of a transmission system according to another embodiment of the present invention. According to FIG. 16, the transmission system includes a plurality of source devices 300-1 and 300-2, a transmission device 1 100-1, and a transmission device 2 100-2. Since the basic configuration is the same as that of the embodiment of FIG. 15, description of overlapping portions will be omitted.

According to FIG. 16, an SMPTE time code is included in a VBI section of raw data transmitted from source device 1 300-1 implemented as a content server and source device 2 300-2 implemented as a live source. Accordingly, the MPEG2 encoder 110-1 and the AVC encoder 110-2 included in the transmission apparatuses 1 and 2 100-1 and 100-2 and the AVC encoder 110-2 respectively capture the SMPTE time codes and include them in the ES level stream. . Accordingly, the file generators 130-1 and 120-2 generate the TS or the file including the time code at the ES level, respectively, so that the later components, that is, the modulator 130-2, are generated. 1) or to the server 140-2.

17 is a view for explaining the configuration and operation of the transmission system according to another embodiment of the present invention. Since the transmission system of FIG. 17 is also basically the same as that of the transmission system of FIGS. 15 and 16, description of overlapping portions is omitted.

Referring to FIG. 17, the source device 1 300-1 implemented as a content server and the source device 2 300-2 implemented as a live source include an SMPTE timecode in a VBI section of raw video data or in a program unit. Include a separate start marker to indicate the starting point of the. For example, when the synchronization information is an SMPTE time code, the MPEG2 encoder 110-1 and the AVC encoder 110-2 respectively capture the SMPTE time code included in the low video data VBI section and include it in the ES level stream. On the other hand, if the synchronization information is a frame number, the file generator 130-2 recognizes the start marker and transfers the generated frame number to the master 120-1 in the transmission device 1 100-1. Accordingly, the frame number value may be included in the first signal.

In FIG. 17, a subject for generating and inserting PES level synchronization information may be determined in various ways depending on whether it is a transport stream or an MP4 file. For example, in the case of a transport stream, it can be either Encoder or Muxer, or a separate PES can be generated directly in the Encoder. Alternatively, the PES level synchronization information may be generated by extracting a time code, a frame number, and the like delivered to the ES level. On the other hand, in the case of MP 4, time code and frame number delivered to the ES level through the encoder can be extracted, and the corresponding information can be inserted by extending a part of the file format Moov in the file generator. As described above, the intermediate processing for generating the stream may be variously combined or configured.

The receiving apparatus analyzes the signaling information provided in the transmission system having various configurations as shown in FIGS. 15 to 17, detects additional data source information, accesses a source apparatus corresponding to the additional data source information, and receives the additional data. Since the configuration and operation of the reception apparatus have been described in detail with reference to FIGS. 1 and 2, redundant description thereof will be omitted.

The above-described transmission / reception system may be applied to various environments for transmitting and receiving different data. That is, in addition to the 3D content consisting of the left eye image and the right eye image, it can be used for various types of hybrid services for separately transmitting and transmitting content based on a broadcasting network and a network network.

For example, the present invention may be applied to a data broadcasting service system that transmits 2D broadcasting through a broadcasting network and transmits data such as multilingual audio and multilingual subtitles through a network. Alternatively, the present invention may be applied to a UHD broadcasting service system that transmits 2D broadcasting through a broadcasting network and transmits UHD broadcasting data through a network. In addition, a multi-view broadcasting service system that transmits 2D broadcasting through a broadcasting network and data such as a depth map or another viewpoint view through a network, or a 2D broadcasting through a broadcasting network, May be applied to a multi-angle service system that provides image data of another photographing angle.

In addition, in the above examples, 2D broadcasting is illustrated as being transmitted only through a broadcasting network, but this is only an example for utilizing an existing broadcasting system, and is not necessarily limited thereto. That is, multilingual audio data, multilingual subtitle data, UHD broadcast data, depth map data, and other view view data corresponding to 2D content data may also be transmitted through the broadcast network.

In addition, in the above various examples, a hybrid system using both an RF broadcasting network and an IP communication network is illustrated. However, the type of communication network may be variously set.

The signal processing method of the transmitting apparatus or the signal processing method of the receiving apparatus according to the various embodiments described above may be coded in software and mounted on various devices.

Specifically, according to an embodiment of the present invention, receiving a first signal and signaling information including reference data through a radio frequency broadcast network, a plurality of additional data source information from the signaling information Detecting, accessing an IP protocol network and receiving a second signal including additional data from a source device corresponding to each of the plurality of additional data source information, the first signal and the A non-transitory readable medium having stored therein a program for detecting, processing, synchronizing and outputting the reference data and the additional data from the second signal may be installed.

The non-transitory readable medium refers to a medium that stores data semi-permanently and is readable by a device, not a medium storing data for a short time such as a register, a cache, a memory, and the like. Specifically, the various applications or programs described above may be stored and provided in a non-transitory readable medium such as a CD, a DVD, a hard disk, a Blu-ray disk, a USB, a memory card, a ROM, or the like.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

210: first receiver 220: second receiver
230: output unit 240: control unit
250: Storage unit 100-1, 100-2: Transmitter 1, 2
300-1, 300-2: source device 1, 2

Claims

A first receiver configured to receive a first signal and signaling information including reference data through an RF frequency network;
If it is determined that the hybrid service is performed based on the identifier indicating that the hybrid service is performed from the signaling information, the address information provides the additional data to receive the additional data from the Internet Protocol Communication Network (IP). A controller for detecting additional data source information;
A second receiver configured to access an IP communication network based on the address information and receive a second signal including the additional data from a source device corresponding to each of the additional data source information;
A storage unit which stores the reference data included in the first signal and the additional data included in the second signal; And
And an output unit configured to synchronize and output the reference data and the additional data.

The method of claim 1,
The signaling information includes first additional data source information for designating a source of at least one first additional data constituting first content and at least one second additional constituting second content to be output following the first content. A second additional data source information for specifying a source of data,
And the signaling information corresponding to each content is repeatedly received in the same form while the corresponding content is output.

The method of claim 2,
The control unit controls the second receiving unit to receive the second signal by accessing the source of the at least one second additional data using the second additional data source information while the first content is output, And the second additional data is stored in the storage unit.

The method of claim 2,
The signaling information includes a designation value for designating a hybrid service synchronization type.
And the control unit controls the output unit to synchronize and output the reference data and the additional data using synchronization information corresponding to the designated value.

The method of claim 4, wherein
The specified value is
A first value that specifies synchronization using PTS information of each of the reference data and the additional data, and a time code included in the elementary stream (ES) of each of the reference data and the additional data as synchronization information. And a second value which specifies to use, and a third value which specifies to use the synchronization information provided through the first signal and the private data stream included in the second signal, respectively.

The method of claim 5,
The output unit,
If the specified value is the first value, the unit data to be synchronized is selected by comparing the PTS of the reference data with the PTS of the additional data, and outputs the selected unit data,
If the specified value is the second value, the first time code included in the ES of the reference data and the second time code included in the ES of the additional data are compared to select unit data to be synchronized, and Correct the time stamp to the same value, and output in synchronization with the corrected time stamp.
And if the specified value is the third value, the synchronization information included in the private data stream is detected and synchronized according to the detected synchronization information.

The method of claim 3,
The signaling information,
The number of hybrid services provided using the signaling information, a hybrid service identifier, a hybrid service provision method, an indicator indicating a hybrid service provision time, a designation value for designating a hybrid service synchronization type, and temporal data of additional data constituting the hybrid service. And a playback order.

The method of claim 4, wherein
The output unit,
A first detector detecting the reference data from the first signal;
A first video processor configured to generate the first video frame by decoding the reference data;
A second detector detecting the additional data from the second signal;
A second video processor to decode the additional data to generate a second video frame;
And a rendering unit which combines the decoded first video frame and the second video frame to perform rendering.
The control unit,
The at least one of the first detector, the second detector, the first video processor, the second video processor, and the renderer may be selectively controlled according to the specified value to perform synchronization processing of the reference data and the additional data. A receiving device, characterized in that performing.

The method according to any one of claims 1 to 8,
The control unit may include a reserved area in a PMT (Program Map Table) of the first signal, a PMT descriptor area, a program and system information protocol virtual channel table (PSIP VCT), an event information table reserved (EIT), and an EIT and the signaling information is detected from at least one of a descriptor region and a private stream.

In the hybrid service providing method of the receiving device,
Receiving a first signal and signaling information including reference data through an RF frequency network;
If it is determined from the signaling information that the hybrid service is performed based on an identifier indicating that the hybrid service is performed, address information for providing the additional data to receive additional data from an Internet Protocol Communication Network is obtained. Detecting additional data source information comprising;
Accessing an IP communication network based on the address information, and receiving a second signal including additional data from a source device corresponding to each of the additional data source information;
Detecting the reference data and the additional data from the first signal and the second signal, respectively, and synchronizing and outputting the reference data and the additional data.

The method of claim 10,
The signaling information includes first additional data source information for designating a source of at least one first additional data constituting first content and at least one second additional constituting second content to be output following the first content. A second additional data source information for specifying a source of data,
The signaling information corresponding to each content is repeatedly received in the same form while the corresponding content is output hybrid service providing method.

The method of claim 11,
Storing the second additional data when the second signal is received based on the second additional data source information while the first content is output;
And outputting the second content by using the stored second additional data when an event to output the second content occurs.

delete