JP2001144720A - Device and method for multiplexing and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid a DTS regulation violation of a PES of a channel whose priority is low in a congestion state and also to suppress data discard quantity. SOLUTION: In a step S1, whether or not the TB and MB of a T-STD are normally simulated is verified. In a step S2, whether or not an unerected multiplexing inhibition flag exists among multiplexing inhibition flags corresponding to each channel is decided. When the unerected multiplexing inhibition flag is decided as existing, and an advance is made to a step S4, and a channel in which the value of a multiplex priority function f2 (lefttime, Pk (k)) is minimum is selected and multiplexed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multiplexing apparatus and method, and a recording medium, and is suitable for use, for example, when multiplexing a plurality of data streams for which priorities are set to generate a transport stream. Multiplexing apparatus and method, and a recording medium.

[0002]

2. Description of the Related Art For example, SkyPerfect TV! Digital satellite broadcasting services such as (Trademark)
A transport stream of MPEG (Moving Picture Experts Group) 2 system (Transport Stream, hereinafter referred to as TS) is used for a broadcast signal. An example of the configuration of a digital satellite broadcasting system that realizes a digital satellite broadcasting service will be described with reference to FIG.

[0003] This digital satellite broadcasting system is installed in a broadcasting station or the like, and transmits a TS as a broadcast signal to a communication satellite 2, a communication satellite 2 which relays a TS from the transmission device 1, and T relayed by communication satellite 2
The receiving device 3 receives S and reproduces the AV signal of the selected program.

[0004] The PES stream generation unit 11- of the transmission device 1
1 to PES stream generation units 11-n generate an elementary stream (hereinafter, referred to as ES) by compression-encoding the AV signal of the program according to the MPEG2 system, and convert the ES into packets at a predetermined data length. Into a packetized elementary stream (Packetiz
An ed Elementary Stream (hereinafter, referred to as PES) is generated and output to the multiplexer 12.

[0005] As shown in FIG. 2, a PES packet constituting a PES corresponds to an ES divided into a predetermined data length.
A PES packet header is added to the PES payload data. In the PES header, ESCR (Elementary Stream Clock Reference) indicating the reference time used to synchronize the reference time of the transmitting side and the receiving side when transmitting the PES, decoding and displaying at the timing intended by the transmitting side at the receiving side DTS (Decode Time Stam)
p) and PTS (Presentation Time Stamp).

The multiplexing device 12 generates a TS by time-division multiplexing the n PES streams generated by the PES stream generation units 11-1 to 11-n, and outputs the TS to the error correction unit 13. I do. The TS is composed of a fixed-length (188 bytes) TS packet as shown in FIG. The TS packet has a PCR (Pr
The TP header includes a TP header including a gram clock reference) and payload data storing the divided PES packets.

[0007] The error correction section 13 performs encoding for error correction on the TS generated by the multiplexing apparatus 12 and performs modulation on the TS.
4 is output. The modulation unit 14 performs digital modulation processing such as QPSK (Quadrature Phase Shift Keying) on the TS on which the error correction processing has been performed, and outputs the resultant to the antenna 15. The antenna 15 amplifies the modulation signal from the modulation unit 14 and outputs R
The signal is transmitted to the communication satellite 2 as an F signal.

The antenna 21 of the receiving device 3 is connected to the communication satellite 2
Receives the RF signal relayed by the
D (Integrated Receiver Decoder) 22 is output. IRD
A demodulation process, an error correction process, and the like are performed on the IF signal from the antenna 21, a PES of a program selected by the user is extracted from the obtained TS, and an AV signal obtained by MPEG decoding is output to the monitor 23. Output.

FIG. 4 shows an example of a detailed configuration of the multiplexing device 12 of the transmitting device 1. The input processing unit 31
The n PESs input from the stream generators 11-1 to 11-n are converted into the corresponding multiplex buffers 34-1 to 34-34.
−n and extract attribute data described in a PES header or the like of a PES packet constituting each PES. The input processing unit 31 also reproduces the reference time of the stream using the decoded value of the ESCR and the captured timing for the ESCR among the extracted attribute data, and transmits the obtained reference time to the control data generation unit 32. Supply. The extracted other attribute data (Stream ID or the like) is supplied to the multiplexing control unit 37 and the control data generation unit 32.

[0010] The control data generation unit 32 includes an input processing unit 31.
Using the reference time information and the attribute data supplied from, the control data necessary for reproducing the program on the receiving side is generated and output to the switch. The generated control data includes Program Specific Infomat
service information (Service In) which is various information about the whole or a part of the program.
formation, hereinafter referred to as SI), and PCR indicating a program time reference value unique to each program.

The null packet generation unit 33 generates a null packet (a packet having no meaning as information) to be multiplexed to satisfy the TS bit rate, and
6 is output.

The multiplexing buffers 34-1 to 34-n are used to temporarily store the PES input from the input processing unit 31.
An O-format buffer that stores the stored PES in the multiplexing control unit 3.
7 is output to the switch 36 in accordance with the control from.

The TP header generation unit 35 includes a multiplex buffer 3
When PESs are read from 4-1 to 34-n, a TP header to be added to the head of the PES is generated and output to the switch 36.

The switch 36 corresponds to the control by the multiplex control unit 37, and controls the control data from the control data generation unit 32,
The null packet from the null packet generator 33, the PES from the multiplex buffers 34-1 to 34-n, or the TP header from the TP header generator 35 are switched and output to the output controller 38.

The multiplexing control unit 37 includes a PES, control data,
Alternatively, the switch 36 is switched so that the null packet is divided into a certain data length, and a TP header is added to each. The control method will be described later.

The output control unit 38 outputs a TS packet containing control data, a null packet or a PES packet, which is input from the switch 36, to the subsequent error correction unit 13 as a TS.

Here, T output from the output control unit 38
S will be described. It is indispensable for the TS to observe the restrictions specified in MPEG2 Systems. The rules of MPEG2 Systems have restrictions on data delay. Specifically, a virtual decoder model as shown in FIG. 5 (hereinafter, T-STD (Transport Stream System Target Decorder))
Is defined, and it is stipulated that the transmitting side must send encoded data, synchronization management information, and control data so that all buffers in T-STD operate without failure. .

The PCR arranged in the TS specifies that the last byte of the PCR describes the estimated time of arrival at the input stage of the T-STD.
The arrival time of all the bits of the T-STD at the input stage can be known.

It is assumed that the input TS is demultiplexed into a system for video data, a system for audio data, or a system for system control data in order to reproduce a program.

As shown in FIG. 5, a system for T-STD video data includes a transport buffer (TB ₁ ) 41-1, a multiplex buffer (MB ₁ ) 42, and an elementary buffer (E).
B ₁ ) 43, a video decoder (D ₁ ) 44, and a rearrangement buffer (O ₁ ) 45. System for audio data, Trang port buffer (TB _n) 41-2, a main buffer (B _n) 46-1, and consists of an audio decoder (D _n) 47. The control data system includes a transport buffer (TB _sys ) 41-3, a main buffer (B _sys ) 46-2, and a system decoder (D _sys ) 48.

The transport buffers 41-1 to 41-3 of each system buffer a bursty, high-speed data stream immediately after demultiplexing. Each of the transport buffers 41-1 to 41-3 has a fixed storage capacity (for example, 512 bytes).
The remainder can be stored. When no data byte exists in the transport buffers 41-1 to 41-3, the transfer of the data byte is not performed. It is stipulated that the transport buffer must not cause an overflow in the configuration of the bit stream.

In the MPEG2 Systems, in order to prevent normal decoding from being performed, the AU is decoded by a decoding time indicated by a DTS of a coding unit (hereinafter referred to as an AU (Access Unit)). Therefore, it is specified that the data output must be completed so that all of the data necessary for the data arrive at the elementary buffer 43. Hereinafter, this rule will be described as a restriction of DTS. Further, a parameter indicating how much time the AU has been multiplexed with respect to the decoding time indicated by the DTS is referred to as a DTS margin and is used below.

In other words, the DTS margin indicates the lapse of time from the AU multiplexing completion time to the decoding time. FIG. 6 shows DTS margins AU [n], A of AUs sequentially output when a stream composed of a plurality of AUs is multiplexed.
An example of the transition of U [n + 1],... In the figure, only the DTS margin AU [n + 7] is a negative value,
This indicates that the constraint of has not been observed.

Note that a plurality of data streams (PES, etc.)
When a single data stream is converted to a TS instead of generating a TS by multiplexing the data, there is almost no data delay that dynamically varies depending on the content of the data. The DTS margin obtained under such a condition (a state in which the data delay due to multiplexing is 0) is referred to as an original DTS margin hereinafter.

Returning to FIG. 4, the multiplexing control unit 37 of the multiplexing device 12 controls which of the control data, the null packet, and the plurality of PES streams is output (multiplexed) at the subsequent stage. Since data is very important data for program reproduction, control is performed such that data is multiplexed with priority over the PES stream. Therefore, when there is no control data to be multiplexed, multiple PESs
And one of a null packet is selected.

There are various methods of selecting one PES from a plurality of PESs. Here, a method of setting priorities to be multiplexed to each PES in advance and selecting the PESs in descending order of priority (hereinafter, referred to as PES) is described. , Stream prioritized multiplexing or conventional).

The number of input PES channels is n, and the priority order of the PES of channels 1, 2,.
, 2nd,..., Nth. In this case, first, the data storage amount of the multiplex buffer 54-1 corresponding to the channel 1 is measured, and when it is determined that one or more packets are stored, the channel 1
Is selected, and it is determined that no data is stored in the multiplex buffer 54-1, the data storage amount of the multiplex buffer 54-2 corresponding to the channel 2 is measured. Similarly, the data storage amounts of the multiplexing buffers 54-3 to 54-n are sequentially measured until a channel to be multiplexed is selected. If it is determined that no data is stored in the multiplexing buffer 54-n corresponding to the channel n with the lowest priority, a null packet is selected.

According to the stream priority multiplexing method, if the output bit rate of the generated TS is higher than the bit rate of the PES of channel 1 having the first priority, the PES of channel 1 is multiplexed without delay. Is done. For the PES of channel 2 with the second priority, T
If the value obtained by subtracting the bit rate of the PES of channel 1 from the output bit rate of S is larger than the bit rate of the PES of channel 2, multiplexing can be performed with a small amount of delay. At this time, the PES of channel 1 is multiplexed without delay.

Similarly, for the channel k having the k-th priority, the value obtained by subtracting the bit rate of each PES of the channels 1 to k-1 from the output bit rate of the TS is the bit rate of the PES of the channel k. If it is larger than the PES of the channel k-1, the multiplexing is possible although the delay amount is increased.

Therefore, if the sum of the input bit rates of the n PESs corresponding to channels 1 to n is smaller than the output human rate of the TS to be generated,
It is possible to multiplex PES. However, the delay time of the PES increases as the priority decreases. Also, in this case, if the original DTS margin of each PES is relatively large, even if a delay occurs in a channel of lower priority, the DT
It is possible to keep the S margin constraint.

However, if the sum of the input bit rates of the n PESs corresponding to channels 1 to n is greater than the output bit rate of the generated TS (hereinafter referred to as a congestion state), all PESs are If an attempt is made to multiplex and output without discarding, the PES of a channel with a lower priority is likely to cause synchronization failure.

[0032]

According to the above-described stream priority multiplexing method, a plurality of streams whose priorities are set in advance are determined.
When multiplexing PES, when the plurality of PESs are in a congested state or when the original DTS margin of the plurality of PESs is small, synchronization failure or data discard is likely to occur in the PES of a channel with a lower priority. There was a problem.

For example, the sum of the average values of the input bit rates of each PES is equal to the output bit rate of the TS to be generated, but the input bit rate of the PES of the channel having a higher priority instantaneously increases. Assume a situation where the sum of the input bit rates of each PES exceeds the output bit rate of the TS. In this situation, the PES delay of the lower priority channel increases significantly, and the DTS margin decreases significantly. In some cases, the PES of a channel with a lower priority is not output by the decoding time, and is discarded or a synchronization failure occurs.

Further, for example, assuming a situation in which the input PES is added and the congestion state is steadily increased, the PES of a channel with a low priority has a significantly increased frequency of synchronization failure or is discarded. The data volume increases.

The present invention has been made in view of such a situation, and a plurality of PESs in which priorities are set in advance are set.
When multiplexing, the channel to be multiplexed is selected in consideration of the priority order and the remaining time until the decoding time, so that the
This is to avoid violation of the DTS constraint of S and to suppress the amount of data discard.

[0036]

SUMMARY OF THE INVENTION A multiplexer according to the present invention comprises:
Input means for inputting priority information indicating the priority set for each data stream, obtaining means for obtaining decoding time information corresponding to each data stream, and extracting means for extracting reference time information from each data stream Holding means for temporarily holding each data stream, and calculating means for calculating the remaining time until the decoding time at the current time of the data stream held by the holding means, based on the decoding time information and the reference time information, Selecting means for selecting one data stream from the plurality of data streams based on the priority information and the remaining time; and reading means for reading and multiplexing the data stream selected by the selecting means from the holding means. It is characterized by.

[0037] The selecting means may preferentially select a data stream having a minimum value obtained by subtracting the value indicated by the priority information from the remaining time.

[0038] The multiplexing apparatus of the present invention may further include a measuring means for measuring the holding amount of each data stream held by the holding means, wherein the selecting means includes the priority information, the remaining time, and the holding time. The data stream can be selected based on the value determined by the multiple priority function with the quantity as a parameter.

[0039] The multiplexing device of the present invention can further include a separating means for separating the transport stream into a plurality of packetized elementary streams.

The multiplexing method of the present invention includes an input step of inputting priority information indicating a priority set for each data stream; an obtaining step of obtaining decoding time information corresponding to each data stream; An extraction step of extracting reference time information from the data stream; a holding step of temporarily holding each data stream; and a current time of the data stream held in the processing of the holding step based on the decoding time information and the reference time information. , A calculation step of calculating the remaining time until the decoding time, a selection step of selecting one data stream from a plurality of data streams based on the priority information and the remaining time, and a selection step. Read and multiplex the data stream from the buffer used for the holding step Characterized in that it comprises a look out step.

According to the recording medium program of the present invention, an input step of inputting priority information indicating a priority set for each data stream; an obtaining step of obtaining decoding time information corresponding to each data stream; An extracting step of extracting reference time information from each data stream; a holding step of temporarily holding each data stream; and a current step of holding the data stream held in the processing of the holding step based on the decoding time information and the reference time information. A calculation step of calculating the remaining time until the decoding time at the time, a selection step of selecting one data stream from the plurality of data streams based on the priority information and the remaining time, and a selection step. Read the data stream from the buffer used for the holding step Characterized in that it comprises a reading step of duplicating.

In the multiplexing apparatus, multiplexing method, and recording medium program of the present invention, priority information indicating the priority set for each data stream is input, and decoding time information corresponding to each data stream is input. Is obtained, reference time information is extracted from each data stream, and each data stream is temporarily held. Further, the remaining time until the decoding time at the current time of the held data stream is calculated based on the decoding time information and the reference time information, and one of the plurality of data streams is calculated based on the priority information and the remaining time. Are selected, and the selected data stream is read and multiplexed.

[0043]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A multiplexer 50 according to the present invention.
Will be described with reference to FIG. This multiplexing device 50 constitutes the transmitting device 1 instead of the multiplexing device 12 of FIG. The multiplexing device 50 has n
PES and priority information corresponding to each PES (a value indicating the priority in multiplexing, the higher the value, the higher the priority) are input.

The input processing unit 51 of the multiplexer 50 writes the inputted n PESs into the corresponding multiplexing buffers 54-1 to 54-n, and describes the PESs in the PES headers of the PES packets constituting each PES. Extract attribute data that has been set. The input processing unit 51 also reproduces the reference time of the stream using the decoded value of the ESCR and the captured timing for the ESCR among the extracted attribute data, and outputs the obtained reference time to the control data generation unit 52 and It is supplied to the multiplexing control unit 57. The other extracted attribute data (Stream ID or the like) is also supplied to the multiplexing control unit 57 and the control data generation unit 52.

The control data generation unit 52 includes an input processing unit 51
Using the reference time information and attribute data supplied from, the control data necessary for reproducing the program on the receiving side is generated and output to the switch 56. The generated control data includes PSI, SI, and PCR.

The null packet generator 53 generates a null packet to be multiplexed to satisfy the bit rate of the TS, and outputs it to the switch 56.

The multiplexing buffers 54-1 to 54-n are provided in the FIF for temporarily storing the PES input from the input processing unit 51.
O-type buffer that stores the stored PES in the multiplexing control unit 5
7 is output to the switch 56 in accordance with the control from.

The TP header generation unit 55
When PESs are read from 4-1 to 54-n, a TP header to be added to the head of the PES is generated and output to the switch 56.

The switch 56 corresponds to the control by the multiplex control unit 57, and controls the control data from the control data generation unit 52,
A TS packet is generated by switching the null packet from the null packet generation unit 53, the PES from the multiplex buffers 54-1 to 54-n, or the TP header from the TP header generation unit 55, and a multiplex control unit 57 in the subsequent stage. And output to the output control unit 58.

The multiplexing control unit 57 controls the switch 56 so as to divide the PES, control data, or null packet into fixed data lengths based on the priority information and the like, and to add a TP header to each of them. The multiplexing control unit 57 also controls the drive 100 so that the magnetic disk 101,
The control program stored in the optical disk 102, the magneto-optical disk 103, or the semiconductor memory 104 is read, and each unit of the multiplexer 50 is controlled based on the read control program.

The output control unit 58 outputs a TS packet containing control data, a null packet, or a PES packet, which is input from the switch 56, as a TS to a subsequent stage.

FIG. 8 shows a detailed configuration example of the multiplexing control unit 57. The multiplexing control unit 57 includes a buffer output and switch control unit 61, a multiplex channel (CH) selection unit 6
2 and a virtual decoder buffer verification unit 63.

Buffer output and switch control unit 61
Outputs the multiplexed buffer read address to the multiplexed channel selector 62, receives the multiplexed channel information returned corresponding to the multiplexed buffer read address, and selects any of the multiplexed channel information (control data, null packet, and n PESs). The control data generator 52, the null packet generator 53, and the multiplex buffers 54-1 to 54-
A data read control signal is output to any one of n, and a switch control signal is output to the switch 56. When multiplexing PES, the multiplex channel information includes P
It contains data of the ES payload length. In this case, the buffer output and switch control unit 61 outputs TP header information such as a stuffing byte length calculated from the PID and the PES payload length to the TP header generation unit 55.

The multiplex channel selection section 62 receives the control data multiplex request signal input from the control data generation section 52, the multiplex buffer write address and buffer output from the multiplex buffers 54-1 to 54-n, and the switch control section 61. Buffer occupancy calculated from the read address of, AU delimiter, TP payload length, and AU data length obtained by analyzing output data (TS packet) from switch 56, reference time data from input processing unit 51 And the remaining time left until the decode time of each of the first AUs stored in the multiplex buffers 54-1 to 54-n calculated using the decode time data.
The channel to be multiplexed is selected based on the time, the priority information of each channel input in advance, and the multiplexing inhibition flag input from the virtual decoder buffer verification unit 63, and the buffer output and control unit 61 as multiplexed channel information. Output to The multi-channel selection unit 62 also
The TP payload length and the AU data length obtained by analyzing the output data (TS packet) from the switch 56 are output to the virtual decoder buffer verification unit 63.

The virtual decoder buffer verification unit 63 uses the TP payload length and the AU data length of the TP output from the multiplex buffers 54-1 to 54-n from the multiplex channel selection unit 62, and The operation of the transport buffer (such as the transport buffer 41-1 of the T-STD shown in FIG. 5) and the multiplexing buffer (such as the multiplexing buffer 42 of FIG. 5) are simulated, and the transport buffer (hereinafter, TB) is When the multiplexing buffer (hereinafter referred to as MB) is about to fail, a multiplexing prohibition flag corresponding to the channel is set and output to the multiplexing channel selector 62.

That is, a predetermined threshold value close to the buffer size is set for each of TB and MB, and TB or M
When the accumulated amount of B exceeds the threshold value, it is determined that the buffer overflow is imminent, and a multiplexing inhibition flag is set and output to the multiplex channel selection unit 62.

It should be noted that MPEG2 Systems stipulates that it is prohibited to hold data continuously for one second or longer for TB and MB. Therefore, the virtual decoder buffer verification unit 63 provides a timer for each buffer to be simulated, resets the timer each time the buffer becomes empty, and when the timer value exceeds a predetermined threshold determined by the leak rate of the buffer, It is determined that the data continuous holding time is imminent to one second, and also at this time, the multiplex prohibition flag corresponding to the channel is set and output to the multiplex channel selection unit 62.

Next, the operation of the multiplex channel selector 62 will be described with reference to the flowchart of FIG. Data to be multiplexed includes data such as video and audio, PSI and P
Although control data such as CR can be roughly classified, since control data is data necessary for reproducing a program, control is performed so as to be multiplexed with priority over data such as video and audio. Therefore, at the timing when control data is not multiplexed, control is performed so as to multiplex any of the PESs corresponding to data such as video and audio or null packets.

More specifically, in step S 1, the multiplex channel selection unit 62 verifies whether the virtual decoder buffer verification unit 63 simulates the TB and MB of the T-STD normally. In step S2,
The multi-channel selection unit 62 includes a virtual decoder verification unit 63
Of the multiplex prohibition flags corresponding to the respective channels input from, whether or not there is a multiplex prohibition flag that is not set. If it is determined that there is a multiple prohibition flag that has not been set, the process proceeds to step S3.

In step S3, the multiplex channel selection unit 62 determines, for each channel, the remaining time lefttime until the decoding time of the first AU stored in the multiplex buffers 54-1 to 54-n of each channel, and A multi-priority function f1 (lefttime, P (k)) is calculated using the channel priority information P (k) as a parameter, and the multi-priority function f1 (lefttime, P (k)) is minimized. Select a channel as a multiplex channel. Multiple priority function f1 (lefttime, P (k)) = lefttime-P (k)

However, the priority information P (k) is the channel k
The higher the priority of the multiplexed channel, the larger the value is set. In addition, the remaining time lefttime decreases as time elapses. Therefore,
The multiple priority function f1 (lefttime, P (k)) evaluates that the smaller the value is, the higher the priority of the corresponding channel is.

If it is determined in step S2 that there is no multiplex prohibition flag that has not been set, the multiplex channel selector 62 selects a null packet as a multiplex channel in step S4.

Next, the result of the operation test of the multiplexer 50 will be described with reference to FIGS. In the operation test, different 6 second video scenes were MPEG
Channel 3 using three PESs encoded in two systems
, The PES of channel 2 has the first priority, the PES of channel 2 has the second priority, and the PES of channel 3 has the third priority. The input bit rate of each PES is 5.87 Mbps, 4.75 Mbps, and the average rate in order from channel 1.
5.87 Mbps, each of which is a variable bit rate having a fluctuation range of ± 100%. However, it is assumed that the input bit rate fluctuation timing between the channels is independent. The bit rate assigned to the output of the generated TS (hereinafter referred to as the output available rate) includes the
16.49M which is equal to the sum of the average input bit rates of S
bps (test 1) and 1 less by 1Mbps
Two values of 5.49 Mbps (test 2) were used.

FIG. 10 shows that the possible output rate is 16.49 Mb.
The graph is a graph obtained by normalizing the input bit rate of the PES of each channel, which changes with time, in the test 1 set to ps with the outputtable rate. As is clear from the figure, the added value of the average input bit rate of each PES instantaneously exceeds the available output rate. Therefore, in the test 1, the test is performed under the condition of instantaneous congestion.

FIG. 11 shows that the possible output rate is 15.49 Mb.
The graph is a graph obtained by normalizing the input bit rate of the PES of each channel that changes with time in the test 2 set to ps, with the output possible rate. As is clear from the figure, the added value of the average input bit rate of each PES almost constantly exceeds the available output rate.
Therefore, in the test 2, the test is performed under the condition that the congestion state is almost constant.

In an instantaneous or steady congestion state, if it is determined that the AUs are not multiplexed in time when the AUs are multiplexed in order so that the data does not break down, the AU data is Are discarded from the multiplex buffer without being multiplexed.

In the operation tests 1 and 2, the unit of the remaining time lefttime which is a parameter of the above-mentioned priority multifunction f1 (lefttime, P (k)) is set to milliseconds, and the priority P (1) of the channel 1 is set. = 4, priority P (2) of channel 2 =
2. The priority P (3) of channel 3 is set to 0.

FIG. 12 shows the time transition of the data discard amount in Test 1. For comparison with the result according to the present embodiment (hereinafter, referred to as present method 1), a conventional stream priority multiplexing method (hereinafter, referred to as conventional method) is used.
The results from are also shown.

As is apparent from FIG.
In the conventional method, data was discarded only in the PES of channel 3. When comparing the transition of the data discard amount shown in FIG. 12 with the transition of the input bit rate shown in FIG. 10, it is clear that the method 1 is more resistant to the DTS constraint violation in the instantaneous congestion state than the conventional method. It can be judged that it is better than

FIG. 13 shows the time transition of the data discard amount in Test 2. FIG. 13 also shows the result of the conventional method. As is clear from FIG. 6, in Test 2, in the method 1 and the conventional method, data was discarded only in the PES of the channel 3. Compared with the transition of the data discard amount shown in FIG. 13 and the transition of the input bit rate shown in FIG. 11, in the steady congestion state, the timing at which the data discard amount takes the local peak value is larger in this method 1. There may be a delay, but this allows a data delay for the high-priority PES when the remaining time lefttime of the high-priority PES has a relatively large margin,
This is because control is performed such that PESs with lower priorities are multiplexed.

The total amount of data discarded is 3677 packets in the present method 1 and 41 in the conventional method.
90 packets. Therefore, it can be determined that the method 1 is more resistant to the DTS constraint violation than the conventional method and can suppress the data discard amount of the PES with a low priority.

Next, an example of the configuration of a multiplexer 70 to which the present invention is applied will be described with reference to FIG. The multiplexing device 70 also has the same configuration as the multiplexing device 50 of FIG.
Of the multiplexing device 12 and the multiplexing device 5
The input processing unit 51 and the multiplexing control unit 57 are replaced with an input processing unit 71 and a multiplexing control unit 73, and a code flag memory 72 is added.

The input processing unit 71 of the multiplexer 70 writes the input n PESs into the corresponding multiplexing buffers 54-1 to 54-n, and describes the PESs in the PES headers of the PES packets constituting each PES. Extract attribute data that has been set. The input processing unit 71 also reproduces the reference time of the stream using the decoded value of the ESCR and the captured timing for the ESCR among the extracted attribute data, and outputs the obtained reference time to the control data generation unit 52. It is supplied to the multiplexing control unit 73. The other extracted attribute data (Stream ID or the like) is also supplied to the control data generator 52 and the multiplexing controller 73.

Further, the input processing section 71 checks the AU of each PES.
Search start code and PES start code,
When a start code of the AU is detected, for example, 10 is output to the code flag memory 72 as code flag data, and when a PES start code is detected, for example, 20 is output to the code flag memory 72 as code flag data. When the code and the PES start code are not detected, for example, 00 is output to the code flag memory 72 as code flag data.

The code flag memory 72 has the same number (= n) of sub-areas as the number of the multiplex buffers 54, and
As shown in FIG. 5, each sub area stores a code flag corresponding to the data held by the multiplexing buffer 54 on a one-to-one basis.

The multiplexing control unit 73 includes a PES, control data,
Alternatively, the switch 56 is controlled so that a null packet is divided into a certain data length and a TP header is added to each of them.

FIG. 16 shows a detailed configuration example of the multiplexing control unit 73. The multiplexing control unit 73 includes a buffer output and switch control unit 81, a multiplex channel selection unit 82,
And a virtual decoder buffer verifying unit 83.

Buffer output and switch control unit 81
Outputs the multiplexed buffer read address to the multiplexed channel selector 82, receives the multiplexed channel information returned in response to the multiplexed buffer readout address, and outputs any of the multiplexed channel information (control data, null packet, and n PESs). The control data generator 52, the null packet generator 53, and the multiplex buffers 54-1 to 54 correspond to the information to be selected).
A data read control signal is output to any one of −n and a switch control signal is output to the switch 56. When multiplexing PES, the multiplex channel information includes P
It contains data of the ES payload length. In this case, the buffer output and switch control unit 81 outputs TP header information such as a stuffing byte length that can be calculated from the PID and the PES payload length to the TP header generation unit 55.

The multiplex channel selection unit 82 receives the control data multiplex request signal input from the control data generation unit 52, the multiplex buffer write address from the multiplex buffers 54-1 to 54-n, the buffer output, and the switch control unit 81. Of the multiplexed buffer calculated from the read address of the multiplexed buffer, the data length occupied by the first AU in the multiplexed buffer 54 obtained by analyzing the code flag stored in the code flag memory 72, and the reference from the input processing unit 71. Selecting a channel to be multiplexed based on the remaining time lefttime up to the decoding time of each AU calculated using the time data and the decoding time data, and the multiplexing inhibition flag input from the virtual decoder buffer verification unit 83; Output to the buffer output and control unit 81 as multiplex channel information The multiplex channel selection unit 82 also outputs to the virtual decoder buffer verification unit 83 the TP payload length and AU data length of the channel multiplexed immediately before, which are obtained by analyzing the code flags stored in the code flag memory 72. .

The virtual decoder buffer verification unit 83 simulates the operation of the TB and MB for each PES using the TP payload length and AU data length of the channel multiplexed immediately before from the multiplex channel selection unit 82. Then, when the TB or MB is about to fail, a multiplex prohibition flag corresponding to the channel is set and output to the multiplex channel selection unit 82.

Next, the operation of the multiplex channel selector 82 will be described with reference to the flowchart of FIG.
Data to be multiplexed can be roughly classified into data such as video and audio, and control data such as PSI and PCR.
Since the data is necessary for reproducing the program, the data is controlled so as to be multiplexed with priority over data such as video and audio. Therefore, at the timing when control data is not multiplexed, control is performed so as to multiplex any of the PESs corresponding to data such as video and audio or null packets.

More specifically, in step S11, the multiplex channel selection unit 82 verifies whether the virtual decoder buffer verification unit 83 simulates the TB and MB of the T-STD normally. In step S12, the multiplex channel selection unit 82 determines whether or not a multiplex prohibition flag that has not been set exists among the multiplex prohibition flags corresponding to each channel input from the virtual decoder verification unit 83. If it is determined that there is a multiple prohibition flag that has not been set, the process proceeds to step S13.

In step S13, the multiplex channel selecting unit 82 determines, for each channel, the remaining time lefttime until the decoding time of the first AU stored in the multiplex buffers 54-1 to 54-n of each channel, Priority information P (k) and multiplex buffer 5
4, the occupation amount oc of the leading AU among the stored data is obtained, and the remaining time lefttime, the priority information P (k), and the occupation amount oc are used as parameters.
efttime, oc, P (k)) and calculate the multiple priority function f2 (lef
The channel having the largest value of (ttime, oc, P (k)) is selected as the multiplexed channel. Multiple priority function f2 (lefttime, oc, P (k)) = oc-α (lefttime
− (P (k) + β))

Here, β is a positive constant, and α is a coefficient for setting the weight of the occupation amount oc and the remaining time lefttime, which are the parameters of the multiple priority function f2 (lefttime, oc, P (k)). If the time lefttime is smaller than P (k) + β, α
By switching from α0 to α1 which is larger than α0, a multi-priority function f2 (lefttime, oc, P (k)) whose dependency on the remaining time lefttime is increased can be obtained. α = α0 (lefttime ≧ P (k) + β) = α1 (lefttime <P (k) + β) α0 ≦ α1

The multiple priority function f2 (lefttime, oc, P (k)) monotonically increases as the occupancy oc increases, and the remaining time lefttime
Increases monotonically with decreasing over time.

FIG. 18 shows an example of a graph of the multiple priority function f2 (lefttime, oc, P (k)). However, in the figure, the unit of the remaining time lefttime is the packet cycle, and the unit of the occupancy oc is the TP data size. Further, priority information P (k) = 10, β = 3, α0 = 1/3, and α1 = 1.

If it is determined in step S12 that there is no multiplex prohibition flag that has not been set, the multiplex channel selection unit 82 selects a null packet as a multiplex channel in step S14.

Next, the result of the operation test of the multiplexer 70 is shown in FIG. The operation test was performed under the same conditions as in Test 3 described above. Note that the priority multiple function f2 (lefttim
e, oc, P (k)) is the priority of channel 1 P (1)
= 100, channel 2 priority P (2) = 50, channel 3 priority P (3) = 0, β = 3, α0 = 1/3, α1 =
It was set to 1. FIG. 19 shows the time change of the data discard amount. As in FIG. 12, for comparison with the result of the present embodiment (hereinafter referred to as the present method 2), the result of the conventional method is also shown. are doing.

FIG. 19 shows the transition of the data discard amount shown in FIG.
In comparison with the transition of the input bit rate shown in FIG. 1, in the steady congestion state, the timing at which the data discard amount reaches the local peak value may be delayed in the method 2 in accordance with the priority. Time lefttim with high PES
This is because when e has a relatively large margin, data delay is allowed for a PES with a high priority, and control is performed such that a PES with a low priority is multiplexed.

The total amount of data discarded is 3626 packets in the present system 2 and 41 in the conventional method.
90 packets. Therefore, the method 2 is more resistant to DTS constraint violation than the method 1 (total amount of data discarded = 3677 packets) and the conventional method, and suppresses the data discard amount of the PES with low priority. Can be determined to be possible.

The multiplexing device 5 according to the present embodiment
Although the input to 0 and 70 is based on the PES, a TS in which a plurality of PESs are multiplexed may be input.

As in the multiplexer 90 shown in FIG. 20, a TS-PES converter 91 for converting an input TS into a PES format is provided at a stage preceding the input processor 71 of the multiplexer 70 in FIG. It may be provided. Note that the TS-PES conversion unit 91
Is output to the input processing unit 71,
The reference time of the TS is reproduced, and the reproduced reference time is output to the control data generator 52 and the multiplexing controller 73.

Further, in the present embodiment, a multiplexing apparatus that inputs a PES and outputs a TS is described. However, the present invention provides that the output of each coding unit is output before the decoding time of the coding unit. The present invention can be applied to a device that outputs any bit stream having a constraint that the completion is completed.

Incidentally, the above-described series of processes can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a recording medium into a possible general-purpose personal computer or the like.

As shown in FIG. 7, the recording medium is a magnetic disk 101 (including a floppy disk) on which the program is recorded, which is distributed separately from the computer to provide the program to the user, and an optical disk 10
2 (CD-ROM (Compact Disc-Read Only Memory), DVD (Dig
Ital Versatile Disc), magneto-optical disc 103
(Including MD (Mini Disc)) or semiconductor memory 1
In addition to being constituted by a package medium such as a CD-ROM 04, the ROM is provided to a user in a state of being incorporated in a computer in advance, and is constituted by a ROM in which a program is recorded, a hard disk included in a storage unit, and the like.

In this specification, the steps for describing the program recorded on the recording medium are not limited to the processing performed in chronological order according to the described order, but are not necessarily performed in chronological order. Alternatively, it also includes individually executed processing.

[0097]

As described above, according to the multiplexing apparatus, the multiplexing method, and the program of the recording medium of the present invention, decoding of the held data stream at the current time is performed based on the decoding time information and the reference time information. The remaining time up to the time is calculated, and one data stream is selected and multiplexed from a plurality of data streams based on the priority information and the remaining time. Avoid PES DTS constraint violations,
In addition, the amount of data discard can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of a configuration of a conventional digital satellite broadcasting system.

FIG. 2 is a diagram showing a data structure of a PES packet.

FIG. 3 is a diagram showing a data structure of a TS packet.

FIG. 4 is a block diagram illustrating an example of a configuration of the multiplexer 12 of FIG.

FIG. 5 is a block diagram illustrating a configuration example of a T-STD.

FIG. 6 is a diagram for explaining a DTS margin.

FIG. 7 is a block diagram illustrating a configuration example of a multiplexing device 50 according to an embodiment of the present invention.

8 is a block diagram illustrating a detailed configuration example of a multiplexing control unit 57 in FIG.

FIG. 9 is a flowchart illustrating an operation of the multiplex channel selection unit 62 of FIG. 8;

FIG. 10 is a diagram showing a transition of an input bit rate of a PES used for an operation test 1;

FIG. 11 is a diagram showing a transition of an input bit rate of a PES used for an operation test 2;

FIG. 12 is a diagram showing a data discard amount in an operation test 1 of the multiplexing device 50;

13 is a diagram showing a data discard amount in the operation test 2 of the multiplexing device 50. FIG.

FIG. 14 shows a multiplexer 70 according to an embodiment of the present invention.
FIG. 3 is a block diagram illustrating a configuration example of FIG.

FIG. 15 is a diagram for explaining a code flag memory 72 of FIG. 14;

16 is a block diagram illustrating a detailed configuration example of a multiplexing control unit 73 in FIG.

FIG. 17 is a flowchart illustrating the operation of the multiplex channel selection unit 82 of FIG.

FIG. 18 is a diagram illustrating a multiple priority function f2.

19 is a diagram illustrating a data discard amount in an operation test of the multiplexing device 70. FIG.

FIG. 20 shows a multiplexer 90 according to an embodiment of the present invention.
FIG. 3 is a block diagram illustrating a configuration example of FIG.

[Description of Signs] 50 multiplexer, 51 input processing unit, 52 control data generation unit, 53 null packet generation unit, 54 multiplex buffer, 55 TP header generation unit, 56 switch, 57 multiplex control unit, 58 output control unit , 6
1 buffer output and switch control unit, 62 multiplex channel selection unit, 63 virtual decoder buffer verification unit, 70 multiplexer, 71 input processing unit, 72
Code flag memory 73 multiplexing control unit 81
Buffer output and switch control unit, 82 multiplex channel selection unit, 83 virtual decoder buffer verification unit, 90 multiplexer, 91 TS-PES conversion unit

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) KA02 KA19 KX28 LA15 LA18 LC11 LD01 LD07 LD18 LE06 9A001 BB04 CC03 CC05 DD10 GG01 HH32 JJ12 KK56

Claims (7)

[Claims] 1. A multiplexing device for multiplexing a plurality of data streams for which priorities are set, an input unit for inputting priority information indicating the priorities set for each of the data streams; Acquiring means for acquiring decoding time information corresponding to each data stream; extracting means for extracting reference time information from each data stream; holding means for temporarily holding each data stream; Based on the reference time information,
Calculating means for calculating the remaining time until the decoding time at the current time of the data stream held by the holding means; and one data stream from the plurality of data streams based on the priority information and the remaining time And a reading unit that reads the data stream selected by the selecting unit from the holding unit and multiplexes the data stream. 2. The multiplexing apparatus according to claim 1, wherein said selecting means preferentially selects the data stream having a minimum value obtained by subtracting the value indicated by the priority information from the remaining time. Device. 3. The information processing apparatus according to claim 1, further comprising: a measuring unit configured to measure a holding amount of each data stream held by the holding unit, wherein the selecting unit multiplexes the priority information, the remaining time, and the holding amount as parameters. The multiplexing apparatus according to claim 1, wherein the data stream is selected based on a value obtained by the priority function. 4. The multiplexing apparatus according to claim 1, wherein the data stream is a packetized elementary stream or a transport stream composed of a single packetized elementary stream. 5. The apparatus according to claim 1, further comprising a separating unit configured to separate the transport stream into a plurality of packetized elementary streams, wherein the data stream is a transport stream including a plurality of packetized elementary streams. The multiplexing device according to claim 1, wherein 6. A multiplexing method for a multiplexing apparatus for multiplexing a plurality of data streams for which priorities have been set, wherein an input for inputting priority information indicating the priorities set for each data stream. An acquisition step of acquiring decoding time information corresponding to each of the data streams; an extraction step of extracting reference time information from each of the data streams; a holding step of temporarily holding each of the data streams; Based on the decoding time information and the reference time information,
An operation step of calculating the remaining time until the decoding time at the current time of the data stream held in the processing of the holding step; and one of the plurality of data streams based on the priority information and the remaining time. A multiplexing method, comprising: selecting a data stream from the buffer; and reading out and multiplexing the data stream selected in the processing of the selecting step from the buffer used in the processing of the holding step. Method. 7. A multiplexing program for multiplexing a plurality of data streams for which priorities are set, an input step of inputting priority information indicating the priorities set for each of the data streams. An acquisition step of acquiring decoding time information corresponding to each data stream; an extraction step of extracting reference time information from each data stream; a holding step of temporarily holding each data stream; Based on the time information and the reference time information,
An operation step of calculating the remaining time until the decoding time at the current time of the data stream held in the processing of the holding step; and one of the plurality of data streams based on the priority information and the remaining time. A selecting step of selecting the data stream of the above, and a reading step of reading and multiplexing the data stream selected in the processing of the selecting step from the buffer used in the processing of the holding step. A recording medium on which a readable program is recorded.