JP3078990B2 - Low delay mode image decoding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an SP @ ML in an image decoding method and apparatus for decoding an encoded image signal.
(Simple profile at main level: Sim
ple Profile at Main Lebe
l) Specification, MP @ LL (Main Profile at Low Level: Main Profile at Low Level)
Level specification or low delay mode (Low Del.)
those concerning the decoding how in ay Mode).

[0002]

2. Description of the Related Art When transmitting or storing digitally expressed image data, encoding is performed to reduce the amount of data. As a coding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information.

As a method utilizing the temporal correlation, there are methods of encoding a difference between two consecutive screens (frames) and detecting motion of an image to perform motion compensation.
As a method using spatial correlation, an image is divided into blocks of a predetermined size (for example, 8 pixels each in the vertical direction and the horizontal direction), data in the blocks are orthogonally transformed, and transform coefficients are scan-converted. Some (for example, rearrange from low-frequency components to high-frequency components) and perform variable-length codes. MPEG (Moving PictureExpe
An image coding method (hereinafter, abbreviated as MPEG2), which is being standardized by rts Group, uses the above two methods in combination. The provisional recommendation for MPEG2 is G
eneric Coding of Moving P
ictures and associated Au
ISO / IEC 13818-2 entitled "dio".

FIG. 7 shows an example of the configuration of an image decoding apparatus for decoding data encoded by such a method. In FIG. 7, reference numeral 10 denotes a memory storage unit, which comprises a buffer memory control unit 11 and a buffer memory 51. Decoding means 20 comprises a variable length decoder 21, a scan converter 22, an inverse quantizer 23, an inverse DCT unit 24, a motion compensation image reproducing unit 25, and a prediction frame memory 26. Also, 52-54
Denotes a frame memory, 100 denotes an input bit stream representing an encoded image, and 200 denotes a reproduced image. Note that 50 indicates the entire memory.

Next, the operation will be described.

The input bit stream 100 is stored in the buffer memory 51 under the control of the buffer memory controller 11. The data read from the buffer memory 51 is subjected to variable-length decoding by the variable-length decoder 21.

Although not all data are variable-length coded, fixed-length codes are also decoded by the variable-length decoder 21. Next, after the scan converter 22 rearranges the order of the data, the data is inversely quantized by the inverse quantizer 23. Next, the inverse DCT unit 24 performs an inverse discrete cosine transform. When receiving data encoded in a frame, the motion-compensated image reproducing unit 25 writes the received data into the predicted frame memory 26 as it is. The reproduced image 200 is reproduced as described above. When an inter-frame difference is received, the prediction data 40 is read from the prediction frame memory 26, added to the reception data from the inverse DCT unit 24, and then the decoded data 41 is stored in the prediction frame memory 26.
Write to.

In forward (one-sided) prediction, a frame (here, P frame) temporally later than the previous frame (here, I frame) is predicted. Therefore, to reproduce the P frame, the prediction data 4 of the I frame that has been decoded in advance is used.
0 must be read. The P frame is reproduced by the prediction data 40 and the prediction error output from the inverse DCT unit 24, and written into the prediction frame memory 26 as decoded data 41. Further, the written data is read out as display data 42, and a reproduced image 200 is output from the motion compensation image reproducing unit 25.

In addition, Dual Prime (Dual P)
(rim) prediction requires predicting a field by averaging predictions of two fields of a reference I frame or P frame.

[0010] In bidirectional prediction of MPEG2, a temporally intermediate frame (here, a B frame) is used from both frames of a temporally preceding frame (here, an I frame) and a temporally later frame (here, a P frame). ). For this reason, in order to reproduce the B frame, it is necessary to recall previously decoded I frame and P frame prediction data 40 from the prediction frame memory 26. (In MPEG2, a temporally later P frame is prior to the B frame. Decrypted). The B frame is reproduced by the prediction data 40 and the output of the inverse DCT unit 24, that is, the prediction error,
The data is written to the memory as bidirectional decoded data 41.
Each of the I, P, and B frames in the frame memories (52 to 54) is read from the memory as display data 42 in a predetermined order, and a reproduced image 200 is output.

In addition, a low-delay mode is prepared for communication using real-time images such as a videophone and a video conference system. When the low delay flag in the input bit stream 100 is "1", it indicates that the mode is the low delay mode. In the low-delay mode, prediction of an intermediate frame (here, a B frame) from temporally preceding and succeeding frames is prohibited, and the encoder can skip one or more frames (skip function). Called a picture).

[0012]

In ordinary bidirectional prediction, a delay of 1.5 frames occurs from the start of decoding processing to the start of display processing. This is because, when performing the decoding process, the decoded data that has been bi-directionally predicted is once written in the frame memory and then read out for display again, which causes a delay. For this reason, it has been found that this becomes a major obstacle in communication applications such as videophones and video conference systems that require low delay.

The operation in the low delay mode is the same as the operation in the forward (one-way) prediction. In forward prediction, the next frame is predicted while referring to the information of the previous frame. For this reason, in order to perform normal writing and reading (when there is no overtaking control of the memory), the predicted frame memory 26 needs to be at least 1.5 frames or more of a frame image to be handled (two frames in a normal MP @ ML decoding device). Is required. In particular, PAL (Phase Alt)
NTSC in the “ernation by Line” method
(National Television System
em Committee) system
Double memory capacity is required, which is a major economic disadvantage.

In MPEG2, Simple, Main, SNR (Signal to Noise Ret)
io) Scalable, S (Spatial) Scalable and high are classified into five profiles and high, high 1440, main and low are combined in four levels. For example, a combination of a simple profile and a main level is expressed as SP @ ML. Is a 11 kind of specifications can be used in MPEG2, SP @ ML, MP @ H L, MP @ H1440, MP
＠ML, MP ＠ LL, SNR ＠ ML, SNR ＠ LL, S
SP @ P1440, HP @ HL, HP @ H1440, H
P ＠ ML .

Also, MP @ ML (main profile,
At Main Level: Main Profile at
When switching from the Main Level) specification to the low-delay mode or the SP @ ML specification, the memory prepared as the MP @ ML specification is not used and remains, so that the system user can maximize the performance of the system. I didn't . The present invention solves the above-described problem in the conventional image decoding device that performs bidirectional prediction,
In the low-delay mode, instead of the conventional storage method using a memory of 1.5 to 2 or more frames, a predetermined memory address space is provided in addition to one frame memory, and overwriting on the same memory is performed. It is an object of the present invention to provide an image decoding method and an image decoding apparatus that increase the use efficiency of a memory (minimize the memory capacity).

[0016]

According to the image decoding method of the present invention, when data is written to a predicted frame memory, eight slices are inserted between a frame signal and the next frame signal.
The same frame memory is overwritten while providing the above memory address space, and the written frame signal is read and decoded.

[0017] Then, a decoding method when having a low delay mode only of a plurality of image decoding device or a plurality of image decoding apparatus, multiple simultaneous decoding or low delay mode, and simple profile-at low delay mode The main-level specification and low-delay mode are combined with the main profile at low-level specification, and the above- described low-delay mode image decoding method is used simultaneously for the above- mentioned decoding within the range of the memory capacity.

[0018]

According to the low-delay mode image decoding how the the present invention,
Each profile (classification in the encoding system) of the specifications defined by MPEG2 utilizes the fact that the vertical motion compensation range is a maximum of 8 slices. In addition to the memory for the frame, more than 8 slices, ie, NTSC / P
In the AL method, by providing a memory address space of 360 macroblocks or more, only the minimum information necessary for motion compensation is used, and motion compensation of the subsequent frame is performed, and the minimum memory (1 frame + 360 macroblocks or more) is provided. Memory) in the decoding process. This is applied in the low delay mode in each profile defined in MPEG2.

Then , a low-delay mode multiple simultaneous decoding using a plurality of image decoding devices, or a low-delay mode and MP @ M
A system that realizes simultaneous decoding in a specification combining L and SP @ ML is made possible.

[0020]

DETAILED DESCRIPTION FIG. 1 the basic structure of the [Basic Configuration] The present invention will be described by FIGS.

FIG. 1 shows a low-delay mode image decoding method according to the present invention.
FIG. 2 is a block diagram showing a basic configuration of an apparatus for realizing the method , FIG. 2 is a diagram showing a method of writing to a predicted frame memory, and FIG. 3 is a diagram showing timings of writing / reading to the predicted frame memory.

In FIG. 1, the same reference numerals as those in FIG. 7 indicate the same parts, 27 indicates an address space of 8 slices or more according to the present invention, and 50 indicates the buffer memory 12, the prediction frame memory 26 having the address space 27, and For example, an 8 Mbit dynamic RAM is used. Reference numeral 30 denotes an address control unit. Also, 25A
Indicates overwriting means and 25B indicates reading means.

Next, the operation of the embodiment shown in FIG. 1 will be described.

First, the header is analyzed from the input bit stream 100 by the variable length decoder 21, and it is checked whether the low delay flag is "0" or "1". If the low-delay flag = "1" is detected, the low-delay mode is set by the external CPU or the internal logic controller.
Thereby, the method of writing to and reading from the predicted frame memory 26 is changed.

In the low delay mode, since no B frame is included, only the I frame and the P frame are decoded. This is the same operation as the forward (one-way) prediction of SP (simple profile).

That is, the input bit stream 100 passes through the buffer memory control unit 11 and the buffer memory 12, undergoes various processes, and reaches the motion compensated image reproduction unit 25. The decoded data 41 of the motion-compensated frame is written to the prediction frame memory 26. Then, each field data is sent from the head of this frame to the motion-compensated image reproducing unit 25 as the display data 42, and is output as the reproduced image 200.

Next, the written decoded data 41 is read as prediction data 40.

At this time, each profile defined by MPEG2 utilizes the fact that the vertical motion compensation range is a maximum of 8 slices, and a memory space 27 (360 macros) of 8 slices or more is stored in the prediction frame memory 26. (The block space: an address space of 1105920 bits or more) is provided, and the decoded data 41 of the next frame created by motion compensation while referring to the first 8 slices of the previous frame is at least 8 slices provided on the predicted frame memory 26 Is written by the overwriting means 25A in the memory space 27 of the memory.

Similarly, motion compensation is performed sequentially with reference to the prediction data 40 of the previous frame, and decoded data 41 of the next frame is generated. The decoded data of the frame is written continuously while shifting by more than the slice. As described above, the decoded data 4 of the frame after the motion compensation is performed.
1 is overwritten on the predicted frame memory 26 while shifting the address of the data of the previous frame by 8 slices or more.

As a result, the prediction data 40 for eight slices for the reference of the previous frame, which is the maximum vertical motion compensation range required for motion compensation, is secured without being lost by overwriting, and Is performed. Similarly, in subsequent frames, writing and reading are continuously repeated while shifting the address space 27 for eight slices or more.

FIG. 2 shows an address space 2 of 8 slices or more.
7 is written to the prediction frame memory 26 while shifting
FIG. 3 shows an operation of continuously repeating reading, and FIG. 3 shows a timing relationship among the prediction data 40, the decoded data 41, and the display data 42 in FIG. The motion-compensated decoded data 41 of the first frame is written to the prediction frame memory 26. The odd (ODD) and even (EVEN) field data from the beginning of the first frame are sent to the motion-compensated image reproducing unit 25 as the display data 42 and output as the reproduced image 200. Next, the written decoded data 41 is read out as prediction data 40. At this time, a memory space 27 of 8 slices or more is provided on the prediction frame memory 26,
The decoded data 41 of the second frame created by motion compensation while referring to the first eight slices of the first frame is written into the memory space 27 for eight slices or more provided on the predicted frame memory 26. Further, the motion compensation is performed with reference to the prediction data of the next first frame, and the decoded data of the second frame is continuously written while the memory addresses of the previous frame and the subsequent frame are shifted by 8 slices or more. Similarly, the decoded data 41 of the third frame after the motion compensation is overwritten on the predicted frame memory 26 while shifting the address of the data of the second frame by 8 slices or more.

At the time of writing, an 8-Mbit dynamic R
Writing is performed from the start address of the prediction frame memory 26 mapped on the AM, and when a predetermined address set to one frame + 8 slices or more is reached, the process returns to the start of the prediction frame memory 26 and writing is repeated.

[0033] To use the memory cyclically as described above, is provided an address control unit 30 between the motion compensation image reproduction unit 25 prediction frame memory 26 as shown in FIG. 1, and stores the head address of each frame, Used to generate the address of the next frame.

According to the above method, the memory capacity for one frame + eight slices is 4147200 + 1 in the NTSC system.
105920 bits, 4976640+ in PAL format
It becomes 1105920 bits. The maximum buffer capacity of the profile defined by MPEG2 is 18350
7088128b in NTSC format even if 08 bit is added
It is 7917568 bits in PAL format, and MP
In the low-delay mode in each profile defined by EG2, the buffer memory 12 and the prediction frame memory 26 can be constructed on one 8-Mbit dynamic RAM in both the NTSC and PAL systems.

[0035] Example 1 of Example 1 present invention will be described with reference to FIG.

Normally, in an image decoding apparatus which satisfies the MP @ ML specification of MPEG2, for example , as shown in FIG.
t is divided into a buffer memory 51 and three frame memories (52, 53, 54) and used. By using the above [ Basic configuration ] and connecting a plurality of image decoding devices, the same memory capacity is obtained. In low-latency mode (or S
Simultaneous decoding of MP @ LL specifications in addition to P @ ML specifications is possible. In the decoding device 1, the low delay mode is set, and the input bit stream 100 is set to the low delay mode (or SP
An example of a configuration for decoding according to (＠ML specification) will be shown. The operation is exactly the same as the basic configuration . Further, in the decoding device 2, the MP @ LL specification is set, and the input bit stream 101
An example of a configuration for decoding according to the L specification is shown. The operation is MP ＠ M in FIG.
This is exactly the same as the L specification, and the number of pixels to be handled is reduced to 1/4 and the buffer memory size is changed to 489472 bits.

The memory 50 is common to both decoding devices. The decoding device 1 according to the first embodiment has a buffer memory of +1 frame + 8 slices of buffer memory 60 (7088128 bits or more in NTSC specification, 79175 in PAL specification).
68 bits or more). The remaining buffer memory 61 is allocated to the decoding device 2.

As a result, simultaneous decoding of MP @ LL becomes possible in addition to the low delay mode (or SP @ ML specification) by effectively using the memory capacity of 16 Mbits required for the MP @ ML specification.

[0039] Example 2 [Example 2] The present invention will be described with reference to FIG.

Normally, in an image decoding apparatus that satisfies the MP @ ML specification of MPEG2, a 16-Mbit memory 50 is divided into a buffer memory 51 and three frame memories (52, 53, 54) as shown in FIG. Is the above [ base
This configuration ] is used to connect a plurality of image decoding devices, so that the same memory capacity can be used for low-delay mode or SP @ M
L can be simultaneously decoded. In the decoding device 1, a low-delay mode (or SP @ ML specification) is set, and a configuration example in which the input bit stream 100 is decoded in the low-delay mode (or SP @ ML specification) is shown. The operation is exactly the same as the basic configuration . In the decoding device 2, the SP @ ML specification or the low delay mode is set, and the input bit stream 101
The following is an example of a configuration for decoding the data according to the SP @ ML specification (or the low delay mode). The operation is exactly the same as in the first embodiment.

The memory 50 is common to both decoding devices, and the decoding device 1 has a buffer memory + 1 frame + 8 slices of buffer memory 60 (7088128 bits or more in NTSC specification, 79175 in PAL specification) according to the basic configuration.
68 bits or more). The buffer memory 61 is similarly assigned to the decoding device 2.

Thus, with the memory capacity required for the MP @ ML specification of 16 Mbit, simultaneous decoding of the low delay mode and the SP @ ML specification, or the two screens of the low delay mode, or the SP
同時 Simultaneous decoding of two screens of the ML specification becomes possible.

[0043] Example 3 of the Example 3 the present invention will be described with reference to FIG.

Normally, in an image decoding apparatus that satisfies the MP @ ML specification of MPEG2, a 16-Mbit memory 50 is divided into a buffer memory 51 and three frame memories (52, 53, 54) as shown in FIG. But the above basics
By using the configuration and connecting a plurality of image decoding apparatuses, simultaneous decoding with the low delay mode or the SP @ ML specification can be performed with the same memory capacity. In the decoding device 1, a low-delay mode (or SP @ ML specification) is set, and a configuration example in which the input bit stream 100 is decoded in the low-delay mode (or SP @ ML specification) is shown. The operation is exactly the same as the basic configuration . In the decoding device 2, the low delay mode (or SP @ ML specification) is set, and the input bit stream 1
1 shows an example of a configuration for decoding 01 in a low-delay mode (or SP @ ML specification).

[0045] Memory 50 is common to both the decoding device, the basic configuration to the decoding device 1 buffer memory +1 frame +8 slices of the buffer memory 60 (NTSC specification 7088128bit above, in the PAL specification 79175
68 bits or more). The remaining buffer memory 61 is allocated to the decoding device 2. At this time, a memory capacity of 1.5 frames can be secured by the NTS.
Only C specification.

As a result, the memory capacity of 16 Mbit required for the MP @ ML specification and the low delay mode (or the SP @ M
L specification).

[0047]

By the low delay mode image decoding how the present invention according to the present invention <br/> lever, 8 slices during the beginning of the frame signal and the next frame signal to write data to the prediction frame memory
Since the same frame memory is overwritten and read out while providing the above memory address space, the motion compensation of the subsequent frame is performed using only the minimum information necessary for motion compensation, and the minimum memory Thus, the memory storage operation in the decoding process becomes possible.

[0048] As a method of decoding when provided with a plurality of image decoding device or a plurality of image decoding apparatus dedicated low-delay mode, multiple simultaneous decoding of low delay mode or low delay mode and SP @ ML specification, low delay Since decoding is performed using the above-described decoding method within the range of the memory capacity in the combination of the mode and the MP @ LL specification, simultaneous decoding can be realized with a small memory capacity.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration diagram of a basic configuration of the present invention.

FIG. 2 is a diagram illustrating a method of writing to a prediction frame memory having a basic configuration .

FIG. 3 shows the basic configuration of writing / writing to a prediction frame memory.
FIG. 4 is a diagram showing read timing.

FIG. 4 is a block diagram of a first embodiment illustrating an example of a configuration of an image decoding system to which the basic configuration of the present invention is applied.

FIG. 5 is a block diagram of a second embodiment illustrating a configuration example of an image decoding system to which the present invention has been applied.

FIG. 6 is a block diagram of a third embodiment illustrating a configuration example of an image decoding system to which the first embodiment of the present invention is applied;

FIG. 7 is a block diagram illustrating a configuration example of a conventional image decoding device.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Memory storage means 11 Buffer memory control part 12 Buffer memory 20 Decoding means 21 Variable length decoder 22 Scan converter 23 Inverse quantizer 24 Inverse DCT unit 25 Motion compensation image reproduction part 26 Predicted frame memory 27 Address of 8 slices or more Space 30 Address control unit 40 Predicted data 41 Decoded data 42 Display data 50 Memory 52 I frame memory 53 P frame memory 54 B frame memory 100 Input bit stream 101 Input bit stream 200 Playback image 201 Playback image

──────────────────────────────────────────────────続 き Continued on the front page (72) Takayuki Kobayashi Inventor Graphics Communication Laboratories, 4-36-19 Yoyogi, Shibuya-ku, Tokyo (72) Inventor Shigeru Komatsu Yoyogi, Shibuya-ku, Tokyo 4-36-19, in Graphics Communication Laboratories, Inc. (72) Inventor Ryuji Saito 4-36-19, Yoyogi, Shibuya-ku, Tokyo, Japan In Graphics Communication Laboratories (72) Inventor Tomoyuki Shindo 4-36-19 Yoyogi, Shibuya-ku, Tokyo Inside Graphics Communication Laboratories Co., Ltd. (72) Inventor Norihiko Nagai 4-36-19 Yoyogi, Shibuya-ku, Tokyo Graphics Communication Co., Ltd.・ Laboratories (56) References JP-A-6-12485 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N ^7/ 24-7/68

Claims (1)

(57) [Claims] 1. A profile defined by MPEG2
To a plurality of image decoding devices dedicated to the low-delay mode, or to an entire device having a plurality of image decoding devices.
Input bit stream representing an image
A low-delay mode multiple simultaneous decoding, or a combination of a low-delay mode and a simple-profile at main-level specification, or a low-delay mode and a main-profile at low-level specification. to the decoding in the range, the low-delay mode image decoding method that uses a decoding method in the low delay mode at the same time,
The decoding method in the low-delay mode includes a prediction frame method.
When writing data to the memory,
Memory address of 8 slices or more between frame signals
A space is provided to store the previous frame signal and the rear frame signal.
Memory addresses are continuously shifted with a shift of 8 slices or more.
Overwrites one frame memory and writes the written frame
Read and decode the system signal.
Low delay mode image decoding method.