JP2022003834A - Transmission method, reception method, transmitter and receiver - Google Patents

Abstract

To reduce an amount of processing related to generation of data to be decoded.SOLUTION: A transmission method includes: a division step of dividing a picture into multiple regions (S101); an encoding step of encoding each of the plurality of regions so as to be decoded independently to generate encoded data corresponding to each of the multiple regions (S102); a packetizing step of storing the generated multiple encoded data in multiple packets (S103); and a transmission step of transmitting the multiple packets (S104). The header information of the packet includes identification information indicating that the packet stores a part corresponding to the header of the basic data unit, or that the packet stores a part other than the part corresponding to the header of the basic data unit.SELECTED DRAWING: Figure 8

Description

The present invention relates to a transmitting method, a receiving method, a transmitting device and a receiving device.

With the sophistication of broadcasting and communication services, the introduction of ultra-high-definition video content such as 8K (7680 x 4320 pixels: also referred to as 8K4K below) and 4K (3840 x 2160 pixels: also referred to as 4K2K below) is being considered. Has been done. The receiving device needs to decode and display the coded data of the received ultra-high-definition moving image in real time, but the processing load at the time of decoding is particularly large for the moving image having a resolution such as 8K. It is difficult to decode a moving image with one decoder in real time. Therefore, a method of reducing the processing load per decoder and achieving real-time processing by parallelizing the decoding processes using a plurality of decoders is being studied.

Further, the coded data is multiplexed and transmitted based on a multiplexing method such as MPEG-2 TS (Transport Stream) or MMT (MPEG Media Transport). For example, Non-Patent Document 1 discloses a technique for transmitting encoded media data for each packet according to MMT.

Information technology-High efficiency coding and media delivery in heterogeneous environment-Part1: MPEG media transfer (MMT), ISO / IEC-1 DIS

However, since the coded data is multiplexed and transmitted based on a multiplexing method such as MPEG-2 TS or MMT, the receiving device receives the encoded data of the moving image from the multiplexed data prior to decoding. Need to be separated. Hereinafter, the process of separating the coded data from the multiplexed data is referred to as demultiplexing.

When parallelizing the decoding process, the receiving device needs to distribute the coded data to be decoded to each of the decoders. At this time, the receiving device needs to analyze the coded data itself. In particular, since the bit rate is very high for contents such as 8K, the processing load related to analysis is large. As a result, there is a problem that the demultiplexing process becomes a bottleneck and reproduction in real time may not be possible.

Therefore, an object of the present invention is to provide a transmission method or a reception method that can reduce the amount of processing related to the generation of data to be decoded.

In order to achieve the above object, the transmission method according to one aspect of the present invention encodes a division step for dividing a picture into a plurality of regions and encoding each of the plurality of regions so that the data can be independently decoded. By this means, the coding step for generating the coded data corresponding to each of the plurality of regions, the packetizing step for storing the generated plurality of the coded data in a plurality of packets, and the plurality of packets. Each of the plurality of coded data including the transmission step to be transmitted is associated one-to-one with a basic data unit which is a unit of data stored in one or more packets, and the plurality of coded data is associated with the plurality of coded data. Each of the above is stored in one or more packets, and the header information of each said packet includes (1) only the packet in the basic data unit, and (2) a plurality of packets in the basic data unit. , And the packet is the first packet of the basic data unit. (3) The basic data unit includes a plurality of packets, and the packet is a packet other than the beginning and the end of the basic data unit. , And (4) the packetization step, which includes identification information indicating whether the basic data unit contains a plurality of packets and the packet is the last packet of the basic data unit. Then, the control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets in which the plurality of coded data are stored, and the header information of the packet is stored in the basic data unit. The header information of the packet includes identification information indicating that only the packet is included, and the header information of the packet is offset information indicating a bit length from the beginning of the coded data of the picture to the beginning of the coded data included in the packet. including.

Further, the receiving method according to one aspect of the present invention is a receiving method in a receiving device including a plurality of decoding units, and a plurality of regions obtained by dividing a picture can be independently decoded. The receiving step of receiving the plurality of packets obtained by packaging the plurality of encoded data obtained by being encoded in the above manner, and the plurality of decoding units of the plurality of encoded data. Each of the plurality of coded data includes a decoding step of decoding in parallel, and each of the plurality of coded data is associated one-to-one with a basic data unit which is a unit of data stored in one or more packets, and the plurality of codes are associated with each other. Each of the converted data is stored in one or more packets, and the header information of each packet includes (1) only the packet in the basic data unit, and (2) a plurality of packets in the basic data unit. The packet is included and the packet is the first packet of the basic data unit. (3) A plurality of packets are included in the basic data unit and the packet is a packet other than the beginning and the end of the basic data unit. And (4) the picture, which includes identification information indicating whether the basic data unit contains a plurality of packets and the packet is the last packet of the basic data unit. The control information used for the decoding unit is stored in one packet different from the plurality of packets in which the plurality of coded data are stored, and the header information of the packet is the packet in the basic data unit. The header information of the packet includes the identification information indicating that only is included, and the header information of the packet includes offset information indicating the bit length from the beginning of the coded data of the picture to the beginning of the coded data contained in the packet. ..

It should be noted that these general or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer readable CD-ROM, and the system, method, integrated circuit, computer program. And may be realized by any combination of recording media.

The present invention can provide a transmission method or a reception method that can reduce the amount of processing related to the generation of data to be decoded.

It is a figure which shows the example which divides a picture into slice segments. It is a figure which shows an example of the PES packet string which stored the data of a picture. It is a figure which shows the division example of the picture which concerns on embodiment. It is a figure which shows the division example of the picture which concerns on the comparative example of embodiment. It is a figure which shows an example of the data of the access unit which concerns on embodiment. It is a block diagram of the transmission device which concerns on embodiment. It is a block diagram of the receiving device which concerns on embodiment. It is a figure which shows an example of the MMT packet which concerns on embodiment. It is a figure which shows another example of the MMT packet which concerns on embodiment. It is a figure which shows an example of the data input to each decoding unit which concerns on embodiment. It is a figure which shows an example of the MMT packet and the header information which concerns on embodiment. It is a figure which shows another example of the data input to each decoding unit which concerns on embodiment. It is a figure which shows the division example of the picture which concerns on embodiment. It is a flowchart of the transmission method which concerns on embodiment. It is a block diagram of the receiving device which concerns on embodiment. It is a flowchart of the receiving method which concerns on embodiment. It is a figure which shows an example of the MMT packet and the header information which concerns on embodiment. It is a figure which shows an example of the MMT packet and the header information which concerns on embodiment.

(Knowledge that became the basis of the present invention)
In recent years, the resolution of displays such as TVs, smartphones, and tablet terminals has been increasing. Especially for broadcasting in Japan, 8K4K (resolution is 8K x 4K) service is scheduled for 2020. Since it is difficult to decode an ultra-high resolution moving image such as 8K4K in real time with a single decoder, a method of performing decoding processing in parallel using a plurality of decoders is being studied.

H. standardized by MPEG and ITU. 264 and H. In a moving image coding method such as 265, the transmission device can divide a picture into a plurality of regions called slices or slice segments, and encode each of the divided regions so that they can be independently decoded. Therefore, for example, H. In the case of 265, the receiving device that receives the broadcast can realize parallelization of the decoding process by separating the data for each slice segment from the received data and outputting the data of each slice segment to a separate decoder. ..

FIG. 1 is a diagram showing an example of dividing one picture into four slice segments in HEVC. For example, the receiving device comprises four decoders, each of which decodes any one of the four slice segments.

In conventional broadcasting, the transmitting device stores one picture (access unit in the MPEG system standard) in one PES packet, and multiplexes the PES packet into a TS packet string. Therefore, the receiving device separates the payload of the PES packet, analyzes the data of the access unit stored in the payload, separates each slice segment, and decodes the data of each separated slice segment. Needed to be output to.

However, the present inventor has found that there is a problem that it is difficult to perform this processing in real time because the amount of processing when analyzing the data of the access unit and separating the slice segments is large.

FIG. 2 is a diagram showing an example in which the data of a picture divided into slice segments is stored in the payload of a PES packet.

As shown in FIG. 2, for example, the data of a plurality of slice segments (slice segments 1 to 4) are stored in the payload of one PES packet. Further, the PES packet is multiplexed into the TS packet string.

The transmission method according to one aspect of the present invention comprises a division step of dividing a picture into a plurality of regions and encoding each of the plurality of regions so that the plurality of regions can be independently decoded. A coding step for generating the coded data corresponding to each, a packetizing step for storing the generated plurality of the coded data in a plurality of packets, and a transmission step for transmitting the plurality of packets are included. In the packetization step, the plurality of coded data are stored in the plurality of packets so that the coded data corresponding to different regions are not stored in one packet.

According to this, since the coded data of each area is stored in different packets, the receiving device does not analyze the coded data stored in the payload of the packet, and the data stored in the packet can be stored in the packet. It is possible to determine which region the coded data is. As a result, the receiving device can perform the decoding target data generation processing of each decoding unit with a small amount of processing. In this way, the amount of processing related to the generation of the data to be decoded in the receiving device is reduced.

For example, in the packetization step, the control information commonly used for all the decoding units in the picture may be stored in a packet different from the plurality of packets in which the plurality of coded data are stored. ..

According to this, the receiving device can determine the packet in which the control information is stored without analyzing the coded data stored in the payload of the packet. As a result, the amount of processing related to the generation of the data to be decoded in the receiving device can be reduced.

Further, the receiving method according to one aspect of the present invention is a receiving method in a receiving device including a plurality of decoding units, and a plurality of regions obtained by dividing a picture can be independently decoded. The plurality of encoded data obtained by being encoded in the above manner receive the plurality of packets obtained by being packetized so that the encoded data in different regions are not stored in one packet. The reception step, the control information commonly used for all the decoding units in the picture contained in any of the plurality of packets, and each of the plurality of coded data in the plurality of regions are combined. This includes a binding step of generating a plurality of combined data and a decoding step in which the plurality of decoding units decode the plurality of combined data in parallel.

According to this, since the coded data of each area is stored in different packets, the receiving device does not analyze the coded data stored in the payload of the packet, and the data stored in the packet can be stored in the packet. It is possible to determine which region the coded data is. As a result, the receiving device can perform the decoding target data generation processing of each decoding unit with a small amount of processing. In this way, the amount of processing related to the generation of the data to be decoded in the receiving device is reduced.

For example, the control information may be stored in a packet different from the plurality of packets in which the plurality of coded data are stored.

According to this, the receiving device can determine the packet in which the control information is stored without analyzing the coded data stored in the payload of the packet. As a result, the amount of processing related to the generation of the data to be decoded in the receiving device can be reduced.

For example, in the binding step, the header information of the packet may be used to determine which region of the plurality of regions the data stored in the packet is encoded data.

According to this, the receiving device can determine in which region the data stored in the packet is the coded data by using the header information of the packet.

For example, each of the plurality of coded data is associated with a basic data unit, which is a unit of data stored in one or more packets, on a one-to-one basis, and each of the plurality of coded data is associated with the above 1. The header information of each of the above packets is stored in the above packets, and the header information of each packet includes (1) only the packet in the basic data unit, (2) a plurality of packets in the basic data unit, and the packet. Is the first packet of the basic data unit, (3) the basic data unit includes a plurality of packets, and the packet is a packet other than the beginning and the end of the basic data unit, and (4). ) The basic data unit includes a plurality of packets, and includes identification information indicating whether the packet is the last packet of the basic data unit. The identification information indicating that only the packet is included in the basic data unit, or (2) a plurality of packets are included in the basic data unit, and the packet is the first packet of the basic data unit. The head of the payload data included in the packet having the header information including the above may be determined to be the head of the coded data in each region.

According to this, the receiving device can determine in which region the data stored in the packet is the coded data by using the header information of the packet.

For example, the header information of the packet further includes offset information indicating a bit length from the beginning of the encoded data of the picture including the plurality of encoded data to the beginning of the encoded data included in the packet. In the join step, (1) the basic data unit contains only the packet, or (2) the basic data unit contains a plurality of packets, and the packet is the head of the basic data unit. The head of the payload data contained in the packet having the header information including the identification information indicating that the packet is a packet and the offset information indicating the non-zero bit length is encoded in the respective regions. It may be determined that it is the beginning of the data.

According to this, the receiving device can determine in which region the data stored in the packet is the coded data by using the header information of the packet.

For example, the receiving method further relies on each of the plurality of combined data based on at least one of the resolution of the picture, the method of dividing the picture into the plurality of regions, and the processing capacity of the plurality of decoding units. A determination step may be included to determine the decoding unit.

According to this, the receiving device can appropriately allocate the coded data of each region to a plurality of decoding units.

Further, in the transmission device according to one aspect of the present invention, the divided portion for dividing the picture into a plurality of regions and the plurality of regions are encoded so that each of the plurality of regions can be independently decoded. A coding unit that generates coded data corresponding to each of the regions, a packetization unit that stores the generated plurality of the coded data in a plurality of packets, and a transmission unit that transmits the plurality of packets. The packetizing unit stores the plurality of coded data in the plurality of packets so that the coded data corresponding to different regions is not stored in one packet.

According to this, since the coded data of each area is stored in different packets, the receiving device does not analyze the coded data stored in the payload of the packet, and the data stored in the packet can be stored in the packet. It is possible to determine which region the coded data is. As a result, the receiving device can perform the decoding target data generation processing of each decoding unit with a small amount of processing. In this way, the amount of processing related to the generation of the data to be decoded in the receiving device is reduced.

Further, the receiving device according to one aspect of the present invention has a plurality of codes obtained by encoding a plurality of regions obtained by dividing the picture so that they can be independently decoded. The data is included in one of the receiver and the plurality of packets obtained by packetizing the coded data in different regions so that the coded data is not stored in one packet. A coupling unit that generates a plurality of combined data by combining the control information commonly used for all the decoding units in the picture and each of the plurality of coded data in the plurality of regions. It includes a plurality of decoding units that decode the plurality of combined data in parallel.

According to this, since the coded data of each area is stored in different packets, the receiving device does not analyze the coded data stored in the payload of the packet, and the data stored in the packet can be stored in the packet. It is possible to determine which region the coded data is. As a result, the receiving device can perform the decoding target data generation processing of each decoding unit with a small amount of processing. In this way, the amount of processing related to the generation of the data to be decoded in the receiving device is reduced.

It should be noted that these comprehensive or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer readable CD-ROM, and the system, method, integrated circuit, computer program. And may be realized by any combination of recording media.

Hereinafter, embodiments will be specifically described with reference to the drawings.

In addition, all of the embodiments described below show a specific example of the present invention. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, the order of steps, etc. shown in the following embodiments are examples, and are not intended to limit the present invention. Further, among the components in the following embodiments, the components not described in the independent claim indicating the highest level concept are described as arbitrary components.

(Embodiment)
In the following, as a moving image coding method, H.I. The case where 265 is used will be described as an example, but H. This embodiment can also be applied when another coding method such as 264 is used.

FIG. 3 is a diagram showing an example in which the access unit (picture) in the present embodiment is divided into division units. The access unit is H. A feature called tiles introduced by 265 divides the tiles horizontally and vertically into two equal parts, for a total of four tiles. Also, slice segments and tiles are associated one-to-one.

The reason for dividing into two equal parts in the horizontal and vertical directions will be described. First, at the time of decoding, a line memory for storing data of one horizontal line is generally required, but when the resolution becomes ultra-high resolution such as 8K4K, the size in the horizontal direction becomes large, so that the size of the line memory increases. In mounting the receiving device, it is desirable that the size of the line memory can be reduced. Vertical division is required to reduce the size of the line memory. Vertical division requires a data structure called tiles. For these reasons, tiles are used.

On the other hand, since images generally have a high horizontal correlation, the coding efficiency is improved if a wide range can be referred to in the horizontal direction. Therefore, from the viewpoint of coding efficiency, it is desirable that the access unit is divided in the horizontal direction.

By dividing the access unit into two equal parts in the horizontal and vertical directions, these two characteristics can be compatible with each other, and both the mounting surface and the coding efficiency can be considered. When a single decoder can decode a 4K2K moving image in real time, the 8K4K image is divided into four equal parts, and each slice segment is divided into 4K2K to obtain a receiver. Can decode 8K4K images in real time.

Next, the reason why the tiles obtained by dividing the access unit in the horizontal and vertical directions and the slice segments have a one-to-one correspondence will be described. H. In 265, the access unit is composed of a plurality of units called NAL (Network Adjustment Layer) units.

The payload of the NAL unit is an access unit delimiter that indicates the start position of the access unit, SPS (Sequence Parameter Set) that is initialization information at the time of decoding that is commonly used for each sequence, and the initial at the time of decoding that is commonly used in the picture. Either PPS (Picture Parameter Set), which is the information to be converted, SEI (Supplemental Enchantment Information), which is not necessary for the decoding process itself but is necessary for processing and displaying the decoding result, and coded data of the slice segment. To store. The header of the NAL unit contains type information for identifying the data stored in the payload.

Here, the transmission device multiplexes the encoded data by MPEG-2 TS, MMT (MPEG Media Stream), MPEG DASH (Dynamic Adaptive Streaming over HTTP), or RTP (Real-time Transport Protocol). At the time of conversion, the basic unit can be set to the NAL unit. In order to store one slice segment in one NAL unit, it is desirable to divide the access unit into slice segment units when dividing the access unit into areas. For this reason, the transmitter associates tiles with slice segments in a one-to-one relationship.

As shown in FIG. 4, the transmission device can collectively set tiles 1 to 4 into one slice segment. However, in this case, all the tiles are stored in one NAL unit, and it is difficult for the receiving device to separate the tiles in the multiplexing layer.

The slice segment includes an independent slice segment that can be independently decoded and a reference slice segment that refers to the independent slice segment. Here, a case where the independent slice segment is used will be described.

FIG. 5 is a diagram showing an example of data of an access unit divided so that the boundaries between the tile and the slice segment match as shown in FIG. The data of the access unit includes the NAL unit in which the access unit delimiter placed at the beginning is stored, the NAL units of SPS, PPS, and SEI placed after that, and tiles 1 to 4 placed after that. Includes data from the slice segment in which the data is stored. The data of the access unit may not include a part or all of the NAL units of SPS, PPS and SEI.

Next, the configuration of the transmission device 100 according to the present embodiment will be described. FIG. 6 is a block diagram showing a configuration example of the transmission device 100 according to the present embodiment. The transmission device 100 includes a coding unit 101, a multiplexing unit 102, a modulation unit 103, and a transmission unit 104.

The coding unit 101 converts the input image into, for example, H.I. Encoded data is generated by encoding according to 265. Further, the coding unit 101 divides the access unit into four slice segments (tiles) and encodes each slice segment, for example, as shown in FIG.

The multiplexing unit 102 multiplexes the coded data generated by the coding unit 101. The modulation unit 103 modulates the data obtained by the multiplexing. The transmission unit 104 transmits the modulated data as a broadcast signal.

Next, the configuration of the receiving device 200 according to the present embodiment will be described. FIG. 7 is a block diagram showing a configuration example of the receiving device 200 according to the present embodiment. The receiving device 200 includes a tuner 201, a demodulation unit 202, a demultiplexing unit 203, a plurality of decoding units 204A to 204D, and a display unit 205.

The tuner 201 receives the broadcast signal. The demodulation unit 202 demodulates the received broadcast signal. The demodulated data is input to the demultiplexing unit 203.

The demultiplexing unit 203 separates the demodulated data into division units, and outputs the data for each division unit to the decoding units 204A to 204D. Here, the division unit is a division area obtained by dividing the access unit, and is, for example, H.A. The slice segment at 265. Further, here, the 8K4K image is divided into four 4K2K images. Therefore, there are four decoding units 204A to 204D.

The plurality of decoding units 204A to 204D operate in synchronization with each other based on a predetermined reference clock. Each decoding unit decodes the coded data of the division unit according to the DTS (Decoding Time Stamp) of the access unit, and outputs the decoding result to the display unit 205.

The display unit 205 generates an 8K4K output image by integrating a plurality of decoding results output from the plurality of decoding units 204A to 204D. The display unit 205 displays the generated output image according to the PTS (Presentation Time Stamp) of the access unit acquired separately. When integrating the decoding results, the display unit 205 performs filter processing such as a deblock filter so that the boundaries are not visually conspicuous in the boundary areas of the division units adjacent to each other, such as the boundaries of tiles. May be good.

In the above description, the transmitting device 100 and the receiving device 200 that transmit or receive the broadcast have been described as an example, but the content may be transmitted and received via the communication network. When the receiving device 200 receives the content via the communication network, the receiving device 200 separates the multiplexed data from the IP packet received by the network such as Ethernet.

In broadcasting, the transmission line delay from the transmission of the broadcast signal to the arrival at the receiving device 200 is constant. On the other hand, in a communication network such as the Internet, the transmission path delay until the data transmitted from the server reaches the receiving device 200 is not constant due to the influence of congestion. Therefore, the receiving device 200 often does not perform strict synchronous reproduction based on the reference clock as in PCR in the MPEG-2 TS of broadcasting. Therefore, the receiving device 200 may display the output image of 8K4K according to the PTS on the display unit without strictly synchronizing the decoding units.

Further, due to congestion of the communication network or the like, the decoding process of all the division units may not be completed at the time indicated by the PTS of the access unit. In this case, the receiving device 200 skips the display of the access unit, or delays the display until the decoding of at least four division units is completed and the generation of the 8K4K image is completed.

The content may be transmitted and received in combination with broadcasting and communication. In addition, this method can also be applied when reproducing multiplexed data stored in a recording medium such as a hard disk or a memory.

Next, when MMT is used as the multiplexing method, a method for multiplexing access units divided into slice segments will be described.

FIG. 8 is a diagram showing an example of packetizing the data of the access unit of HEVC into MMT. SPS, PPS, SEI, etc. do not necessarily have to be included in the access unit, but the case where they exist is illustrated here.

NAL units arranged before the first slice segment in the access unit such as the access unit delimiter, SPS, PPS, and SEI are collectively stored in the MMT packet # 1. Subsequent slice segments are stored in separate MMT packets for each slice segment.

As shown in FIG. 9, the NAL unit arranged before the first slice segment in the access unit may be stored in the same MMT packet as the first slice segment.

Also, if NAL units such as End-of-Sequence or End-of-Bitstream indicating the end of the sequence or stream are added after the final slice segment, they will be in the same MMT packet as the final slice segment. Stored. However, since NAL units such as End-of-Sequence or End-of-Bitstream are inserted at the end point of the decoding process or the connection point of two streams, the receiving device 200 inserts these NAL units. , It may be desirable to be easily available in the multiplexing layer. In this case, these NAL units may be stored in an MMT packet separate from the slice segment. Thereby, the receiving device 200 can easily separate these NAL units in the multiplexing layer.

As the multiplexing method, TS, DASH, RTP, or the like may be used. Also in these methods, the transmission device 100 stores different slice segments in different packets. This can ensure that the receiving device 200 can separate slice segments in the multiplexing layer.

For example, when TS is used, the coded data is PES packetized in slice segment units. When RTP is used, the coded data is RTP packetized in slice segment units. Even in these cases, the NAL unit and the slice segment arranged before the slice segment may be packetized separately as in MMT packet # 1 shown in FIG.

When TS is used, the transmission device 100 indicates a unit of data stored in a PES packet, such as by using a data alignment descriptor. Further, since DASH is a method of downloading MP4 format data units called segments by HTTP or the like, the transmission device 100 does not packetize the coded data at the time of transmission. Therefore, the transmitting device 100 creates a subsample for each slice segment so that the receiving device 200 can detect the slice segment in the multiplexing layer in the MP4, and stores the information indicating the storage position of the subsample in the header of the MP4. You may.

Hereinafter, MMT packetization of slice segments will be described in detail.

As shown in FIG. 8, by packetizing the coded data, the data commonly referred to at the time of decoding all the slice segments in the access unit such as SPS and PPS is stored in the MMT packet # 1. In this case, the receiving device 200 concatenates the payload data of the MMT packet # 1 with the data of each slice segment, and outputs the obtained data to the decoding unit. In this way, the receiving device 200 can easily generate input data to the decoding unit by concatenating the payloads of a plurality of MMT packets.

FIG. 10 is a diagram showing an example in which input data to the decoding units 204A to 204D is generated from the MMT packet shown in FIG. The demultiplexing unit 203 concatenates the payload data of the MMT packet # 1 and the MMT packet # 2, so that the decoding unit 204A generates the data necessary for decoding the slice segment 1. The demultiplexing unit 203 similarly generates input data from the decoding unit 204B to the decoding unit 204D. That is, the demultiplexing unit 203 generates the input data of the decoding unit 204B by concatenating the payload data of the MMT packet # 1 and the MMT packet # 3. The demultiplexing unit 203 generates the input data of the decoding unit 204C by concatenating the payload data of the MMT packet # 1 and the MMT packet # 4. The demultiplexing unit 203 generates the input data of the decoding unit 204D by concatenating the payload data of the MMT packet # 1 and the MMT packet # 5.

The demultiplexing unit 203 removes NAL units such as the access unit delimiter and SEI, which are not necessary for the decoding process, from the payload data of the MMT packet # 1, and only the SPS and PPS NAL units required for the decoding process. May be separated and added to the data of the slice segment.

When the coded data is packetized as shown in FIG. 9, the demultiplexing unit 203 outputs the MMT packet # 1 including the head data of the access unit in the multiplexing layer to the first decoding unit 204A. .. Further, the demultiplexing unit 203 analyzes the MMT packet including the head data of the access unit in the multiplexing layer, separates the SPS and PPS NAL units, and slices the separated SPS and PPS NAL units from the second onward. By adding to each of the segment data, input data for each of the second and subsequent decoding units is generated.

Further, the receiving device 200 uses the information contained in the header of the MMT packet to store the type of data stored in the MMT payload, and when the slice segment is stored in the payload, the slice segment in the access unit. It is desirable to be able to identify the index number. Here, the data types are the data before the slice segment (the NAL units arranged before the first slice segment in the access unit are collectively referred to as this), and the data of the slice segment. Either. When storing a unit obtained by fragmenting an MPU such as a slice segment in an MMT packet, a mode for storing an MFU (Media Fragment Unit) is used. When this mode is used, the transmission device 100 uses, for example, a Data Unit, which is a basic unit of data in the MFU, as a sample (a data unit in the MMT, which corresponds to an access unit) or a subsample (a sample). Can be set to a divided unit).

At this time, the header of the MMT packet includes a field called a Fragmentation indicator and a field called a Fragmentation counter.

The Fragmentation indicator indicates whether the data stored in the payload of the MMT packet is a fragment of the Data unit, and if it is a fragment, the fragment is the first or last fragment in the Data unit, or , Indicates whether the fragment is neither the beginning nor the end. In other words, the Fragmentation indicator included in the header of a certain packet is (1) the Data unit, which is the basic data unit, contains only the packet, and (2) the Data unit is divided into a plurality of packets and stored. The packet is the first packet of the Data unit, (3) the Data unit is divided and stored in a plurality of packets, and the packet is a packet other than the beginning and the end of the Data unit, and (4). This is identification information indicating whether the Data unit is divided into a plurality of packets and stored, and the packet is the last packet of the Data unit.

The Fragment counter is an index number indicating which fragment the data stored in the MMT packet corresponds to in the Data unit.

Therefore, the transmitting device 100 sets the sample in the MMT to the Data unit, and sets the pre-slice segment data and each slice segment to each fragment unit of the Data unit, so that the receiving device 200 sets the header of the MMT packet. The information contained in can be used to identify the type of data stored in the payload. That is, the demultiplexing unit 203 can generate input data to the decoding units 204A to 204D by referring to the header of the MMT packet.

FIG. 11 is a diagram showing an example in which the sample is set to Data unit and the pre-slice segment data and the slice segment are packetized as a fragment of Data unit.

The pre-slice segment data and the slice segment are divided into five fragments from fragment # 1 to fragment # 5. Each fragment is stored in a separate MMT packet. At this time, the values of the Fragmentation indicator and the Fragmentation counter included in the header of the MMT packet are as shown in the figure.

For example, Fragment indicator is a binary 2-bit value. The Fragment indicator of MMT packet # 1 at the beginning of Data unit, the Fragment indicator of MMT packet # 5 at the end, and the Fragment indicator from MMT packet # 2 to MMT packet # 4 which are packets in between are different from each other. Set to a value. Specifically, the Fragment indicator of the MMT packet # 1 at the beginning of the Data unit is set to 01, the Fragment indicator of the final MMT packet # 5 is set to 11, and from the MMT packet # 2 which is a packet in between. The Fragment indicator up to MMT packet # 4 is set to 10. If the Data unit contains only one MMT packet, the Fragment indicator is set to 00.

Further, the Fragment counter is 4 which is a value obtained by subtracting 1 from 5 which is the total number of fragments in MMT packet # 1, decreases by 1 in order in subsequent packets, and is 0 in the final MMT packet # 5. be.

Therefore, the receiving device 200 can identify the MMT packet storing the pre-slice segment data by using either the Fragment indicator or the Fragment counter. Further, the receiving device 200 can identify the MMT packet containing the Nth slice segment by referring to the Fragment counter.

The header of the MMT packet separately includes the sequence number in the MPU of the Movie Fragment to which the Data unit belongs, the sequence number of the MPU itself, and the sequence number in the Movie Fragment of the sample to which the Data unit belongs. By referring to these, the demultiplexing unit 203 can uniquely determine the sample to which the Data unit belongs.

Further, since the demultiplexing unit 203 can determine the index number of the fragment in the Data unit from the Fragment counter or the like, the slice segment stored in the fragment can be uniquely specified even when packet loss occurs. For example, even if the fragment # 4 shown in FIG. 11 cannot be acquired due to packet loss, the demultiplexing unit 203 knows that the fragment received next to the fragment # 3 is the fragment # 5, so that the fragment # 5 is used. The slice segment 4 stored in can be correctly output to the decoding unit 204D instead of the decoding unit 204C.

When a transmission line that guarantees no packet loss is used, the demultiplexing unit 203 refers to the header of the MMT packet and refers to the type of data stored in the MMT packet or the slice segment. The arriving packet may be processed periodically without determining the index number of. For example, when the access unit is transmitted by the pre-slice data and a total of five MMT packets of four slice segments, the receiving device 200 determines the pre-slice data of the access unit to start decoding. By processing the received MMT packets in order, the pre-slice data and the data of the four slice segments can be acquired in order.

Hereinafter, a modified example of packetization will be described.

The slice segment does not necessarily have to be divided in both the horizontal direction and the vertical direction in the plane of the access unit, and the access unit may be divided only in the horizontal direction as shown in FIG. However, it may be divided only in the vertical direction.

Also, if the access unit is divided only in the horizontal direction, tiles do not need to be used.

Further, the number of in-plane divisions in the access unit is arbitrary and is not limited to four. However, the area size of slice segments and tiles is H. It must be at least the lower limit of a coding standard such as 265.

The transmission device 100 may store identification information indicating an in-plane division method in the access unit in an MMT message, a TS descriptor, or the like. For example, information indicating the number of divisions in the horizontal direction and the vertical direction in the plane may be stored. Alternatively, the identification information peculiar to the division method, such as being divided into two equal parts in the horizontal direction and the vertical direction as shown in FIG. 3, or being divided into four equal parts in the horizontal direction as shown in FIG. May be assigned. For example, when the access unit is divided as shown in FIG. 3, the identification information indicates mode 1, and when the access unit is divided as shown in FIG. 1, the identification information indicates mode 1. ..

Further, the multiplexing layer may contain information indicating restrictions on the coding conditions related to the in-plane division method. For example, information indicating that one slice segment is composed of one tile may be used. Alternatively, the reference block for motion compensation when decoding a slice segment or tile is limited to the slice segment or tile at the same position on the screen, or the block within a predetermined range in the adjacent slice segment. Information indicating that, etc. may be used.

Further, the transmission device 100 may switch whether or not to divide the access unit into a plurality of slice segments according to the resolution of the moving image. For example, the transmission device 100 does not perform in-plane division when the moving image to be processed has a resolution of 4K2K, and may divide the access unit into four when the moving image to be processed is 8K4K. good. By prescribing the division method for the 8K4K moving image in advance, the receiving device 200 determines the presence / absence of in-plane division and the dividing method by acquiring the resolution of the received moving image, and decodes the image. The operation can be switched.

Further, the receiving device 200 can detect the presence or absence of in-plane division by referring to the header of the MMT packet. For example, when the access unit is not divided and the Data unit of MMT is set in the sample, the fragment of Data unit is not performed. Therefore, the receiving device 200 can determine that the access unit is not divided when the value of the Fragment counter included in the header of the MMT packet is always zero. Alternatively, the receiving device 200 may detect whether or not the value of the Fragmentation indicator is always 01. The receiving device 200 can determine that the access unit is not divided even when the value of the Fragmentation indicator is always 01.

Further, the receiving device 200 can handle a case where the number of divisions in the plane of the access unit and the number of decoding units do not match. For example, if the receiving device 200 includes two decoding units 204A and 204B capable of decoding 8K2K encoded data in real time, the demultiplexing unit 203 may refer to the decoding unit 204A with 8K4K encoded data. Outputs two of the four slice segments that make up.

FIG. 12 is a diagram showing an operation example when the MMT packetized data as shown in FIG. 8 is input to the two decoding units 204A and 204B. Here, it is desirable that the receiving device 200 can integrate and output the decoding results of the decoding units 204A and 204B as they are. Therefore, the demultiplexing unit 203 selects the slice segment to be output to each of the decoding units 204A and 204B so that the decoding results of the decoding units 204A and 204B are spatially continuous.

Further, the demultiplexing unit 203 may select the decoding unit to be used according to the resolution or frame rate of the coded data of the moving image. For example, when the receiving device 200 includes four 4K2K decoding units, if the resolution of the input image is 8K4K, the receiving device 200 performs the decoding process using all four decoding units. Further, if the resolution of the input image is 4K2K, the receiving device 200 performs the decoding process using only one decoding unit. Alternatively, if the demultiplexing unit 203 can decode 8K4K in real time by a single decoding unit even if the in-plane is divided into four, the demultiplexing unit 203 integrates all the division units into one decoding unit. Output to.

Further, the receiving device 200 may determine the decoding unit to be used in consideration of the frame rate. For example, when the receiving device 200 includes two decoding units whose upper limit of the frame rate that can be decoded in real time is 60 fps when the resolution is 8K4K, there is a case where 120 fps coded data is input at 8K4K. be. At this time, assuming that the in-plane is composed of four division units, the slice segment 1 and the slice segment 2 are input to the decoding unit 204A, and the slice segment 3 and the slice segment 4 are combined, as in the example of FIG. It is input to the decoding unit 204B. Since each of the decoding units 204A and 204B can decode up to 120 fps in real time if the resolution is 8K2K (the resolution is half of 8K4K), the decoding process is performed by these two decoding units 204A and 204B.

Further, even if the resolution and the frame rate are the same, the profile or level in the coding method, or H.I. 264 or H. If the coding method itself such as 265 is different, the processing amount will be different. Therefore, the receiving device 200 may select a decoding unit to be used based on this information. If the receiving device 200 cannot decode all the coded data received by broadcasting or communication, or if all the slice segments or tiles constituting the area selected by the user cannot be decoded, the decoding unit Decryptable slice segments or tiles may be automatically determined within the processing range of. Alternatively, the receiving device 200 may provide a user interface for the user to select an area to be decoded. At this time, the receiving device 200 may display a warning message indicating that all the areas cannot be decoded, or may display information indicating the number of decodeable areas, slice segments, or tiles.

Further, the above method can be applied to a case where an MMT packet storing a slice segment of the same coded data is transmitted and received using a plurality of transmission paths such as broadcasting and communication.

Further, the transmission device 100 may perform coding so that the regions of the slice segments overlap each other in order to make the boundaries of the division units inconspicuous. In the example shown in FIG. 13, the 8K4K picture is divided into four slice segments 1 to 4. Each of the slice segments 1 to 3 is, for example, 8K × 1.1K, and the slice segment 4 is 8K × 1K. Also, adjacent slice segments overlap each other. By doing so, at the boundary in the case of being divided into four, which is indicated by the dotted line, motion compensation at the time of coding can be efficiently executed, so that the image quality of the boundary portion is improved. In this way, deterioration of image quality at the boundary portion is reduced.

In this case, the display unit 205 cuts out an 8K × 1K area from the 8K × 1.1K area and integrates the obtained area. In addition, the transmission device 100 separately transmits information indicating whether or not the slice segments are overlapped and encoded and the range of the overlap is included in the multiplexing layer or the encoded data. May be good.

The same method can be applied when tiles are used.

Hereinafter, the operation flow of the transmission device 100 will be described. FIG. 14 is a flowchart showing an operation example of the transmission device 100.

First, the coding unit 101 divides the picture (access unit) into a plurality of slice segments (tiles) which are a plurality of regions (S101). Next, the coding unit 101 encodes each of the plurality of slice segments so that they can be independently decoded, thereby generating coded data corresponding to each of the plurality of slice segments (S102). The coding unit 101 may encode a plurality of slice segments with a single coding unit, or may perform parallel processing by the plurality of coding units.

Next, the multiplexing unit 102 multiplexes the plurality of coded data by storing the plurality of coded data generated by the coding unit 101 in a plurality of MMT packets (S103). Specifically, as shown in FIGS. 8 and 9, the multiplexing unit 102 stores a plurality of coded data so that the coded data corresponding to different slice segments is not stored in one MMT packet. Store in MMT packet. Further, as shown in FIG. 8, the multiplexing unit 102 stores control information commonly used for all decoding units in the picture, and a plurality of MMT packets # 2 to # in which a plurality of coded data are stored. It is stored in MMT packet # 1, which is different from 5. Here, the control information includes at least one of the access unit delimiter, SPS, PPS and SEI.

The multiplexing unit 102 may store the control information in the same MMT packet as any one of the plurality of MMT packets in which the plurality of coded data are stored. For example, as shown in FIG. 9, the multiplexing unit 102 stores the control information in the first MMT packet (MMT packet # 1 in FIG. 9) among the plurality of MMT packets in which the plurality of coded data are stored. You may.

Finally, the transmission device 100 transmits a plurality of MMT packets. Specifically, the modulation unit 103 modulates the data obtained by the multiplexing, and the transmission unit 104 transmits the modulated data (S104).

FIG. 15 is a block diagram showing a configuration example of the receiving device 200, and is a diagram showing in detail the configuration of the demultiplexing unit 203 and the subsequent stage shown in FIG. 7. As shown in FIG. 15, the receiving device 200 further includes a decoding instruction unit 206. Further, the demultiplexing unit 203 includes a type discrimination unit 211, a control information acquisition unit 212, a slice information acquisition unit 213, and a decoding data generation unit 214.

Hereinafter, the operation flow of the receiving device 200 will be described. FIG. 16 is a flowchart showing an operation example of the receiving device 200. Here, the operation for one access unit is shown. When the decryption process of a plurality of access units is executed, the process of this flowchart is repeated.

First, the receiving device 200 receives, for example, a plurality of packets (MMT packets) generated by the transmitting device 100 (S201).

Next, the type determination unit 211 acquires the type of coded data stored in the received packet by analyzing the header of the received packet (S202).

Next, the type determination unit 211 determines whether the data stored in the received packet is the pre-slice segment data or the slice segment data based on the type of the acquired coded data (S203). ..

When the data stored in the received packet is the pre-slice segment data (Yes in S203), the control information acquisition unit 212 acquires the pre-slice segment data of the access unit to be processed from the payload of the received packet, and the slice concerned. The pre-segment data is stored in the memory (S204).

On the other hand, when the data stored in the received packet is slice segment data (No in S203), the receiving device 200 uses the header information of the received packet to display a plurality of data stored in the received packet. It is determined which region of the regions is the coded data. Specifically, the slice information acquisition unit 213 acquires the index number Idx of the slice segment stored in the received packet by analyzing the header of the received packet (S205). Specifically, the index number Idx is an index number in the Movie Fragment of the access unit (sample in MMT).

The process of step S205 may be collectively performed in step S202.

Next, the decoding data generation unit 214 determines a decoding unit for decoding the slice segment (S206). Specifically, the index number Idx and the plurality of decoding units are associated with each other in advance, and the decoding data generation unit 214 decodes the slice segment of the decoding unit corresponding to the index number Idx acquired in step S205. Determine the decoding unit to be used.

As described in the example of FIG. 12, the decoding data generation unit 214 has a resolution of the access unit (picture), a method of dividing the access unit into a plurality of slice segments (tiles), and a plurality of reception devices 200. The decoding unit that decodes the slice segment may be determined based on at least one of the processing capabilities of the decoding unit. For example, the decoding data generation unit 214 determines the method of dividing the access unit based on the identification information in the descriptor such as the message of MMT or the section of TS.

Next, the decoding data generation unit 214 includes control information commonly used for all decoding units in the picture contained in any of the plurality of packets, and each of the plurality of coded data of the plurality of slice segments. By combining with and, a plurality of input data (combined data) to be input to a plurality of decoding units is generated. Specifically, the decoding data generation unit 214 acquires slice segment data from the payload of the received packet. The decoding data generation unit 214 generates input data to the decoding unit determined in step S206 by combining the pre-slice segment data stored in the memory in step S204 and the acquired slice segment data. (S207).

After step S204 or S207, if the data of the received packet is not the final data of the access unit (No in S208), the processing after step S201 is performed again. That is, the above process is repeated until the input data to the plurality of decoding units 204A to 204D corresponding to all the slice segments included in the access unit are generated.

The timing at which the packets are received is not limited to the timing shown in FIG. 16, and a plurality of packets may be received in advance or sequentially and stored in a memory or the like.

On the other hand, when the data of the received packet is the final data of the access unit (Yes in S208), the decoding command unit 206 outputs a plurality of input data generated in step S207 to the corresponding decoding units 204A to 204D. (S209).

Next, the plurality of decoding units 204A to 204D generate a plurality of decoded images by decoding a plurality of input data in parallel according to the DTS of the access unit (S210).

Finally, the display unit 205 generates a display image by combining the plurality of decoded images generated by the plurality of decoding units 204A to 204D, and displays the displayed image according to the PTS of the access unit (S211).

The receiving device 200 acquires the DTS and PTS of the access unit by analyzing the payload data of the MMT packet that stores the header information of the MPU or the header information of the Movie Fragment. Further, when TS is used as the multiplexing method, the receiving device 200 acquires the DTS and PTS of the access unit from the header of the PES packet. When RTP is used as the multiplexing method, the receiving device 200 acquires the DTS and PTS of the access unit from the header of the RTP packet.

Further, the display unit 205 may perform a filter process such as a deblock filter at the boundary of adjacent division units when integrating the decoding results of the plurality of decoding units. Since the filtering process is not required when displaying the decoding result of a single decoding unit, the display unit 205 performs the processing depending on whether or not the filtering process is performed on the boundary of the decoding result of the plurality of decoding units. It may be switched. Whether or not the filtering process is necessary may be predetermined depending on the presence or absence of division and the like. Alternatively, information indicating whether filtering is necessary may be separately stored in the multiplexing layer. In addition, information necessary for filter processing such as filter coefficients may be stored in the SPS, PPS, SEI, or slice segment. The decoding units 204A to 204D or the demultiplexing unit 203 acquire these information by analyzing the SEI, and output the acquired information to the display unit 205. The display unit 205 performs filtering processing using these information. When these information are stored in the slice segment, it is desirable that the decoding units 204A to 204D acquire the information.

In the above description, an example is shown in which the types of data stored in the fragment are two types, the pre-slice segment data and the slice segment, but the data types may be three or more. In this case, the case classification according to the type is performed in step S203.

Further, the transmission device 100 may fragment the slice segment and store it in the MMT packet when the data size of the slice segment is large. That is, the transmission device 100 may fragment the pre-slice segment data and the slice segment. In this case, if the access unit and the Data unit are set to be equal as in the packetization example shown in FIG. 11, the following problems occur.

For example, when the slice segment 1 is divided into three fragments, the slice segment 1 is divided into three packets having a Fragment counter value of 1 to 3 and transmitted. Further, in the slice segment 2 and later, the Fragment counter value becomes 4 or more, and the value of the Fragment counter cannot be associated with the data stored in the payload. Therefore, the receiving device 200 cannot specify the packet that stores the head data of the slice segment from the information in the header of the MMT packet.

In such a case, the receiving device 200 may analyze the data of the payload of the MMT packet to specify the start position of the slice segment. Here, H. 264 or H. In 265, as a format for storing the NAL unit in the multiplexing layer, a format called a byte stream format in which a start code consisting of a specific bit string is added immediately before the NAL unit header, and a field indicating the size of the NAL unit are added. There are two types, a format called the NAL size format.

The byte stream format is used in MPEG-2 systems and RTPs and the like. The NAL size format is used in MP4, as well as DASH and MMT using MP4.

When the byte stream format is used, the receiving device 200 analyzes whether the head data of the packet matches the start code. If the start data of the packet matches the start code, the receiving device 200 obtains the type of the NAL unit from the subsequent NAL unit header, and whether the data included in the packet is the slice segment data. You can detect if.

On the other hand, in the case of the NAL size format, the receiving device 200 cannot detect the start position of the NAL unit based on the bit string. Therefore, in order to acquire the start position of the NAL unit, the receiving device 200 needs to shift the pointer by reading data by the size of the NAL unit in order from the head NAL unit of the access unit.

However, if the size of the subsample unit is indicated in the header of the MPU or Movie Fragment in the MMT and the subsample corresponds to the pre-slice data or the slice segment, the receiving device 200 is based on the size information of the subsample. The starting position of each NAL unit can be specified. Therefore, the transmitting device 100 may include information indicating whether or not the subsample unit information exists in the MPU or Movie Fragment in the information acquired by the receiving device 200 at the start of data reception, such as the MPT in the MMT. ..

The MPU data is an extension based on the MP4 format. In MP4, H. 264 or H. There is a mode in which the parameter set such as 265 SPS and PPS can be stored as sample data, and a mode in which the parameter set cannot be stored. In addition, information for identifying this mode is shown as the entry name of SampleEntry. When a storable mode is used and the parameter set is included in the sample, the receiver 200 acquires the parameter set by the method described above.

On the other hand, when the non-storable mode is used, the parameter set is stored as a Decoder Specific Information in SampleEntry, or is stored using a stream for the parameter set. Here, since the stream for the parameter set is not generally used, it is desirable that the transmitting device 100 stores the parameter set in the Decoder Special Information. In this case, the receiving device 200 analyzes the SampleEntry transmitted as the metadata of the MPU or the metadata of the Movie Fragment in the MMT packet, and acquires the parameter set referred to by the access unit.

When the parameter set is stored as sample data, the receiving device 200 can acquire the parameter set necessary for decoding by referring only to the sample data without referring to SampleEntry. At this time, the transmission device 100 does not have to store the parameter set in the SampleEntry. By doing so, the transmission device 100 can use the same SampleEntry in different MPUs, so that the processing load of the transmission device 100 at the time of MPU generation can be reduced. Further, there is an advantage that the receiving device 200 does not need to refer to the parameter set in the SampleEntry.

Alternatively, the transmission device 100 may store one default parameter set in the SampleEntry and store the parameter set referred to by the access unit in the sample data. In the conventional MP4, since the parameter set is generally stored in the SampleEntry, if the parameter set does not exist in the SampleEntry, there may be a receiving device that stops the reproduction. This problem can be solved by using the above method.

Alternatively, the transmitter 100 may store the parameter set in the sample data only if a parameter set different from the default parameter set is used.

Since it is possible to store the parameter set in the SampleEntry in both modes, the transmitting device 100 may always store the parameter set in the VisualSampleEntry, and the receiving device 200 may always acquire the parameter set from the VisualSampleEntry.

In the MMT standard, the header information of MP4 such as Moov and Moof is called MPU meta, but the transmission device 100 does not necessarily have to transmit the MPU meta. Further, the receiving device 200 determines whether or not SPS and PPS are stored in the sample data based on the service of the ARIB (Association of Radio Industries and Businesses) standard, the type of the asset, the presence or absence of transmission of the MPU meta, and the like. It is also possible to judge.

FIG. 17 is a diagram showing an example in which the pre-slice segment data and each slice segment are set to different Data units.

In the example shown in FIG. 17, the pre-slice segment data and the data sizes from slice segment 1 to slice segment 4 are Length # 1 to Length # 5, respectively. The Field values of Fragmentation indicator, Fragment counter, and Offset included in the header of the MMT packet are as shown in the figure.

Here, Offset indicates the bit length (offset) from the beginning of the coded data of the sample (access unit or picture) to which the payload data belongs to the first byte of the payload data (encoded data) included in the MMT packet. Offset information. Although the Fragment counter value is described as starting from a value obtained by subtracting 1 from the total number of fragments, it may start from another value.

FIG. 18 is a diagram showing an example when the Data unit is fragmented. In the example shown in FIG. 18, the slice segment 1 is divided into three fragments and stored in MMT packet # 2 to MMT packet # 4, respectively. Also in this case, assuming that the data size of each fragment is Length # 2_1 to Length # 2_3, the values of each field are as shown in the figure.

As described above, when the data unit such as the slice segment is set to Data unit, the head of the access unit and the head of the slice segment can be determined as follows based on the field value of the MMT packet header.

The head of the payload in the packet whose Offset value is 0 is the head of the access unit.

The beginning of the payload of the packet whose Offset value is different from 0 and whose Fragmentation indicator value is 00 or 01 is the beginning of the slice segment.

Further, when the data unit fragmentation does not occur and the packet loss does not occur, the receiving device 200 is stored in the MMT packet based on the number of slice segments acquired after detecting the head of the access unit. You can identify the index number of the slice segment.

Further, even when the data unit of the data before the slice segment is fragmented, the receiving device 200 can similarly detect the access unit and the head of the slice segment.

Further, even when packet loss occurs or when SPS, PDU, and SEI included in the pre-slice segment data are set to different Data units, the receiving device 200 slices based on the analysis result of the MMT header. By identifying the MMT packet that stores the start data of the segment and then analyzing the header of the slice segment, the start position of the slice segment or tile in the picture (access unit) can be identified. The amount of processing related to the analysis of the slice header is small, and the processing load does not matter.

As described above, each of the plurality of coded data of the plurality of slice segments is associated one-to-one with the basic data unit (Data unit) which is a unit of data stored in one or more packets. Further, each of the plurality of coded data is stored in one or more MMT packets.

The header information of each MMT packet includes Fragmentation indicator (identification information) and Offset (offset information).

The receiving device 200 determines that the receiving device 200 is the head of the encoded data of each slice segment at the head of the payload data included in the packet having the header information including the Fragmentation indicator having a value of 00 or 01. .. Specifically, the head of the payload data included in the packet having the header information including the Offset whose value is not 0 and the Fragmentation indicator whose value is 00 or 01 is the head of the coded data of each slice segment. Is determined.

Further, in the example of FIG. 17, the head of the Data unit is either the head of the access unit or the head of the slice segment, and the value of the Fragmentation indicator is 00 or 01. Further, the receiving device 200 refers to the type of the NAL unit and determines whether the head of the Data Unit is the access unit delimiter or the slice segment. It is also possible to detect the beginning of the slice segment or the beginning of the slice segment.

In this way, the transmitting device 100 packetizes the NAL unit so that the beginning of the NAL unit always starts from the beginning of the payload of the MMT packet, so that the pre-slice segment data is divided into a plurality of Data units. The receiving device 200 can detect the head of the access unit or the slice segment by analyzing the Packet indicator and the NAL unit header. The type of NAL unit exists in the first byte of the NAL unit header. Therefore, the receiving device 200 can acquire the type of the NAL unit by additionally analyzing one byte of data when analyzing the header portion of the MMT packet. In the case of audio, the receiving device 200 may detect the head of the access unit, and may determine based on whether the value of the Fragmentation indicator is 00 or 01.

Further, as described above, when the coded data encoded so as to be able to be divided and decoded is stored in the PES packet of the MPEG-2 TS, the transmitting device 100 can use the data alignment descriptor. be. Hereinafter, an example of a method of storing the coded data in the PES packet will be described in detail.

For example, in HEVC, the transmitting device 100 can use the data alignment descriptor to indicate whether the data stored in the PES packet is an access unit, a slice segment, or a tile. The types of alignment in HEVC are defined as follows:

Alignment type = 8 indicates a slice segment of HEVC. Alignment type = 9 indicates a slice segment or access unit of HEVC. Alignment type = 12 indicates a slice segment or tile of HEVC.

Therefore, the transmitting device 100 can indicate that the data of the PES packet is either the slice segment or the pre-slice segment data by using, for example, type 9. Since the type indicating a slice instead of the slice segment is also specified separately, the transmission device 100 may use the type indicating a slice instead of the slice segment.

Further, the DTS and PTS included in the header of the PES packet are set only in the PES packet including the head data of the access unit. Therefore, in the receiving device 200, if the type is 9 and the DTS or PTS field is present in the PES packet, the entire access unit or the first division unit in the access unit is stored in the PES packet. It can be judged.

Further, even if the transmitting device 100 uses a field such as transport_priority indicating the priority of the TS packet that stores the PES packet including the head data of the access unit, the receiving device 200 can distinguish the data included in the packet. good. Further, the receiving device 200 may determine the data included in the packet by analyzing whether or not the payload of the PES packet is the access unit delimiter. Also, the data_alignnment_indicator in the PES packet header indicates whether data is stored in the PES packet according to these types. If this flag (data_alignnment_indicator) is set to 1, the data stored in the PES packet is guaranteed to follow the type shown in the data alignment descriptor.

Further, the transmission device 100 may use the data alignment descriptor only when the PES packet is formed in a unit that can be divided and decoded such as a slice segment. As a result, the receiving device 200 can determine that the coded data is PES packetized in units that can be divided and decoded when the data alignment descriptor exists, and if the data alignment descriptor does not exist, the code is coded. It can be determined that the converted data is converted into a PES packet for each access unit. If data_alignnment_indicator is set to 1 and the data alignment descriptor does not exist, it is specified in the MPEG-2 TS standard that the unit of PES packetization is an access unit.

If the data alignment descriptor is included in the PMT, the receiving device 200 determines that the PES packet is formed in units that can be divided and decoded, and inputs to each decoding unit based on the packetized unit. Data can be generated. Further, when the receiving device 200 does not include the data identification descriptor in the PMT and it is determined that parallel decoding of the coded data is necessary based on the program information or other descriptor information, the receiving device 200 does not include the data alignment descriptor. , The input data to each decoding unit is generated by analyzing the slice header of the slice segment. When the coded data can be decoded by a single decoding unit, the receiving device 200 decodes the data of the entire access unit by the decoding unit. When the information indicating whether or not the coded data is composed of a unit that can be divided and decoded such as a slice segment or a tile is separately indicated by a PMT descriptor or the like, the receiving device 200 uses the descriptor of the descriptor. It may be determined whether or not the coded data can be decoded in parallel based on the analysis result.

Further, since the DTS and PTS included in the header of the PES packet are set only in the PES packet including the head data of the access unit, when the access unit is divided into PES packets, the second and subsequent PESs are set. The packet does not contain information indicating the DTS and PTS of the access unit. Therefore, when the decoding processes are performed in parallel, the decoding units 204A to 204D and the display unit 205 use DTS and PTS stored in the header of the PES packet including the head data of the access unit.

Although the transmitting device, the receiving device, the transmitting method and the receiving method according to the embodiment have been described above, the present invention is not limited to this embodiment.

Further, each processing unit included in the transmitting device and the receiving device according to the above embodiment is typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them.

Further, the integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of circuit cells inside the LSI may be used.

In each of the above embodiments, each component may be configured with dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

In other words, the transmitting device and the receiving device include a processing circuit and a storage device electrically connected to the processing circuit (accessible from the control circuit). The processing circuit includes at least one of dedicated hardware and a program execution unit. Further, when the processing circuit includes a program execution unit, the storage device stores the software program executed by the program execution unit. The processing circuit uses the storage device to execute the transmission method or the reception method according to the above embodiment.

Further, the present invention may be the software program or a non-temporary computer-readable recording medium in which the program is recorded. Needless to say, the above program can be distributed via a transmission medium such as the Internet.

In addition, the numbers used above are all exemplified for the purpose of specifically explaining the present invention, and the present invention is not limited to the exemplified numbers.

In addition, the division of functional blocks in the block diagram is an example, and multiple functional blocks can be realized as one functional block, one functional block can be divided into multiple, and some functions can be transferred to other functional blocks. You may. Further, the functions of a plurality of functional blocks having similar functions may be processed by a single hardware or software in parallel or in a time division manner.

Further, the order in which the steps included in the above transmission method or reception method are executed is for exemplifying the present invention in detail, and may be an order other than the above. Further, a part of the above steps may be executed simultaneously with other steps (parallel).

Although the transmitting device, the receiving device, the transmitting method and the receiving method according to one or more aspects of the present invention have been described above based on the embodiments, the present invention is not limited to the present embodiments. No. As long as it does not deviate from the gist of the present invention, one or more of the present embodiments may be modified by those skilled in the art, or may be constructed by combining components in different embodiments. It may be included within the scope of the embodiment.

The present invention can be applied to a device or device that performs media transport such as video data and audio data.

100 Transmitter 101 Coding section 102 Multiplexing section 103 Modulation section 104 Transmitting section 200 Receiver device 201 Tuner 202 Demodulation section 203 Demultiplexing section 204A, 204B, 204C, 204D Decoding section 205 Display section 206 Decoding command section 211 Type discrimination section 212 Control information acquisition unit 213 Slice information acquisition unit 214 Decoding data generation unit

Claims (4)

A split step that splits a picture into multiple areas,
A coding step of generating coded data corresponding to each of the plurality of regions by encoding each of the plurality of regions so that they can be independently decoded.
A packetization step of storing the generated plurality of the coded data in a plurality of packets, and
Including a transmission step of transmitting the plurality of packets.
Each of the plurality of coded data is associated one-to-one with a basic data unit, which is a unit of data stored in one or more packets.
Each of the plurality of coded data is stored in the one or more packets.
The header information of each packet includes (1) only the packet in the basic data unit, (2) a plurality of packets in the basic data unit, and the packet is the head of the basic data unit. It is a packet, (3) a plurality of packets are included in the basic data unit, and the packet is a packet other than the beginning and the end of the basic data unit, and (4) a plurality of packets in the basic data unit. It contains identification information indicating whether the packet is included and the packet is the last packet of the basic data unit.
In the packetization step, the control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets in which the plurality of coded data are stored, and the header information of the packet is the said. Contains identification information indicating that only the packet is included in the basic data unit,
The header information of the packet includes offset information indicating a bit length from the beginning of the coded data of the picture to the beginning of the coded data included in the packet.
Sending method. It is a receiving method in a receiving device including a plurality of decoding units.
A plurality of regions obtained by dividing a picture are encoded so that they can be independently decoded, and a plurality of encoded data obtained by packetizing the plurality of encoded data. The receive step to receive the packet and
The plurality of decoding units include a decoding step of decoding the plurality of coded data in parallel.
Each of the plurality of coded data is associated one-to-one with a basic data unit, which is a unit of data stored in one or more packets.
Each of the plurality of coded data is stored in the one or more packets.
The header information of each packet includes (1) only the packet in the basic data unit, (2) a plurality of packets in the basic data unit, and the packet is the head of the basic data unit. It is a packet, (3) a plurality of packets are included in the basic data unit, and the packet is a packet other than the beginning and the end of the basic data unit, and (4) a plurality of packets in the basic data unit. It contains identification information indicating whether the packet is included and the packet is the last packet of the basic data unit.
The control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets in which the plurality of coded data are stored, and the header information of the packet is stored in the basic data unit. Contains identification information indicating that only the packet is included
The header information of the packet includes offset information indicating a bit length from the beginning of the coded data of the picture to the beginning of the coded data included in the packet.
Receiving method. A division that divides the picture into multiple areas,
A coding unit that generates coded data corresponding to each of the plurality of regions by encoding each of the plurality of regions so that they can be decoded independently.
A packetization unit that stores the generated plurality of the coded data in a plurality of packets, and
It is provided with a transmitter for transmitting the plurality of packets.
Each of the plurality of coded data is associated one-to-one with a basic data unit, which is a unit of data stored in one or more packets.
Each of the plurality of coded data is stored in the one or more packets.
The header information of each packet includes (1) only the packet in the basic data unit, (2) a plurality of packets in the basic data unit, and the packet is the head of the basic data unit. It is a packet, (3) a plurality of packets are included in the basic data unit, and the packet is a packet other than the beginning and the end of the basic data unit, and (4) a plurality of packets in the basic data unit. It contains identification information indicating whether the packet is included and the packet is the last packet of the basic data unit.
The packetizing unit stores the control information used for the decoding unit in the picture in one packet different from the plurality of packets in which the plurality of coded data are stored, and the header information of the packet is the said. Contains identification information indicating that only the packet is included in the basic data unit,
The header information of the packet includes offset information indicating a bit length from the beginning of the coded data of the picture to the beginning of the coded data included in the packet.
Transmitter. A plurality of regions obtained by dividing a picture are encoded so that they can be independently decoded, and a plurality of encoded data obtained by packetizing the plurality of encoded data. The receiver that receives the packet and
It is provided with a plurality of decoding units that decode the plurality of coded data in parallel.
Each of the plurality of coded data is associated one-to-one with a basic data unit, which is a unit of data stored in one or more packets.
Each of the plurality of coded data is stored in the one or more packets.
The header information of each packet includes (1) only the packet in the basic data unit, (2) a plurality of packets in the basic data unit, and the packet is the head of the basic data unit. It is a packet, (3) a plurality of packets are included in the basic data unit, and the packet is a packet other than the beginning and the end of the basic data unit, and (4) a plurality of packets in the basic data unit. It contains identification information indicating whether the packet is included and the packet is the last packet of the basic data unit.
The control information used for the decoding unit in the picture is stored in one packet different from the plurality of packets in which the plurality of coded data are stored, and the header information of the packet is stored in the basic data unit. Contains identification information indicating that only the packet is included
The header information of the packet includes offset information indicating a bit length from the beginning of the coded data of the picture to the beginning of the coded data included in the packet.
Receiver.

Images