KR100903863B1 - Interface method between overlapped transform and deblocking filter of wmv decoder - Google Patents

Abstract

The present invention provides an interface method between an overlap converter and a deblocking filter of a WMV decoder, which can provide a WMV decoder having a reduced memory bandwidth by changing an interface structure between the overlap converter and a deblocking filter that occupies a large portion of the memory bandwidth. Regarding, The method comprising: determining whether the overlap converter and the deblocking filter is activated; And apply an adaptive pipeline structure according to the above-described activation. If both the overlap converter and the deblocking filter are activated, deblocking filtering is performed using map data of YCbCr format stored in the external memory. It comprises a step of performing.

Windows Media Video, Overlap Transform, Deblocking Filter, Hardware, Bandwidth, WMV9, VC-1, Overlapped Transform, Deblocking filter, bandwidth

Description

Interface method between overlap converter and deblocking filter of FM decoder {INTERFACE METHOD BETWEEN OVERLAPPED TRANSFORM AND DEBLOCKING FILTER OF WMV DECODER}

1 is a functional block diagram of a typical WMV decoder.

FIG. 2 is a diagram illustrating an execution time point of overlap transform and deblocking filtering in a 176 × 144 (QCIF) sized image.

3 and 4 are diagrams illustrating the relationship between the delay time and the memory buffer in the pipeline structure.

5 is a flowchart illustrating an interface method between an overlap converter and a deblocking filter of a WMV decoder according to an embodiment of the present invention.

6 is a diagram illustrating an external memory structure according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating a pipeline structure when the overlap transform unit is deactivated and the deblocking filter is activated.

8 is a diagram illustrating a pipeline structure when both an overlap converter and a deblocking filter are activated.

9 is a diagram illustrating a macroblock unit overlap conversion process according to an embodiment of the present invention.

10 is a diagram illustrating a macroblock-based deblocking filtering process according to an embodiment of the present invention.

FIG. 11 is a diagram illustrating an external memory map in an overlap conversion process according to an embodiment of the present invention. FIG.

<Description of the symbols for the main parts of the drawings>

10: variable length decoder 12: inverse transform / inverse quantization unit

14: predictive decoding unit / motion compensation unit 16: overlap conversion unit

18: Deblocking filter 22: External memory

The present invention relates to a hardware system configuration of a Windows Media Video (WMV) decoder, and more particularly, to change an interface structure between an overlapped transform and a deblocking filter that occupy a large portion of a memory bandwidth. Accordingly, the present invention relates to an interface method between an overlap conversion unit and a deblocking filter of a WMV decoder capable of providing a WMV decoder having a reduced memory bandwidth.

The video decoder is a device for restoring a video coded in a compressed form, and the MPEG-based decoder used as an international standard and the WMV (VC-1) -based decoder widely used as a market standard are preferred. As demand for multimedia contents increases, the importance of hardware implementation of the video decoder is increasing.

In general, when implementing a video decoder in hardware, the most performance-sensitive part can be seen in terms of the frequency of external memory access and circuit size by direct memory access (DMA) calls connected to an external bus.

In particular, WMV series decoders have higher compression efficiency than MPEG-4 with compression efficiency close to the widely used H.264 codec, and have a comparative advantage in performance compared to H.264 in complexity. Is evaluated.

1 is a functional block diagram of a typical WMV decoder. For hardware implementation of the WMV decoder, a variable length decoder (VLD) 10, an inverse transform / inverse quantizer (IG / IDCT) 12, and a predictive decoder / motion compensation are shown. The MC (14), the overlap converter (16), and the deblocking filter (18) should be operated in a pipeline manner in macroblock units.

In addition, functional units such as the variable length decoder 10, the inverse transform / inverse quantizer 12, the predictive decoder / motion compensator 14, the overlap transform unit 16, and the deblocking filter 18 are provided in the system bus. The external memory 22 is accessed using the reference numeral 20.

The external memory 22 stores information in macroblock units or block units. Accordingly, the variable length decoder 10, the inverse transform / inverse quantizer 12, the predictive decoder / motion compensation unit 14, the overlap transform unit 16, and the deblocking filter 18 are spaced adjacent to each other. If block or block information is needed, each memory 22 is accessed via a system bus 20.

In general, the variable length decoder 10 (function unit 1), the inverse transform / inverse quantizer 12 (function unit 2), and the predictive decoder / motion compensation unit 14 (function unit 3) are implemented in a macroblock implementation. However, the overlap transform unit 16 (function unit 4) and the deblocking filter 17 (function unit 5) are not easy to implement a macroblock due to a structural feature of converting certain data around the macroblock.

Specifically, the functional units 1 to 3 refer to the N-1 macroblock data in processing the Nth macroblock but do not convert the N-1th macroblock data. On the contrary, the function unit 4 and the function unit 5 convert the value of the N-th macroblock in processing the N-th macroblock. This is because the end point of the actual N-1 macroblock processing is delayed by 1 macroblock processing time in the horizontal direction and by 1 macroblock line in the vertical direction.

If only one of the functional units 4 and 5 is performed, the data for which the execution of the functional units 1 to 3 is completed is used, and the functional units 1 to 3 are used for the functional units 4 and 5. The delay does not cause any structural problems in the pipeline because the data is not received again. However, if all of the functions 4, 5 are performed, the function 5, i.e., the deblocking filter 18, should receive and process the result of the function 4, i. Therefore, when both the functions 4 and 5 are to be performed, the delay time becomes longer, and at this time, the processing of the function 4 causes a delay of 1 macroblock line vertically and 1 macroblock horizontally. Problem occurs.

FIG. 2 is a diagram illustrating an execution time point of overlap transform and deblocking filtering in a 176 × 144 (QCIF) sized image.

Referring to FIG. 2, the deblocking filtering may perform MB0 when the overlap transform performs MB12. That is, a delay of 12 macroblocks occurs based on QCIF when performing overlap transform and deblocking filtering (1 macroblock line + 1 macroblock). This delay increases as the image size increases. For example, a delay of 23 macroblocks in a CIF (352 * 288) image and 46 macroblocks in an HD (720 * 480) image occurs.

3 and 4 are diagrams illustrating the relationship between the delay time and the memory buffer in the pipeline structure.

First, Figure 3 shows a pipeline structure without a delay.

Referring to FIG. 3, for example, if there are processes 0, 1, and 2 that have no delay, process 0 executes first, and then executes 0, 1 (receiving previous 0 data) at the next execution time. Proceed to how to do. Thus, it can be seen that there must be two buffers between processes. You need a buffer to store the data of the process that is currently being processed and to store the data that was processed just before (the data the next processor will receive).

On the other hand, Figure 4 shows a pipeline structure with a delay.

Referring to FIG. 4, processes 0 and 1 are the same as in FIG. 3, but delays of two processes occur between processes 1 and 2. In other words, it shows a pipeline structure problem in which the size of the buffer must be increased by the delay time. It can be seen that there are four buffers between process 1 and process 2 in the lower part of FIG. 4.

If there is always 2 delays as in the case of FIG. 4, two additional buffers may be solved. However, if the variable and the delay time are very long, such as an image, the internal buffer (register or internal memory) cannot be used and the general external memory is used. There is no choice but to use.

This may lead to deterioration of the performance of the entire system, and therefore, there is a need to improve the interface structure between the overlap change unit and the deblocking filter in order to minimize the deterioration of the performance.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to improve the interface structure of an overlapping block and a deblocking filter having high access frequency of an external memory in hardware implementation of a WMV decoder widely used as a market standard. Accordingly, the present invention provides an interface method between the overlap converter of the WMV decoder and the deblocking filter, which can minimize the memory bandwidth.

Another object of the present invention is to use the memory map applied between the overlap transform unit and the deblocking filter and to adaptively change the pipeline structure according to the execution condition of the overlap transform unit and the deblocking filter. It is to provide an interface method between the overlap conversion unit and the deblocking filter of the WMV decoder to reduce the P.

Other objects and advantages of the invention will be described below and will be appreciated by the embodiments of the invention. In addition, the objects and advantages of the invention may be realized by the means and combinations indicated in the claims.

In order to achieve the above object, an interface method between an overlap converter and a deblocking filter of a WMV decoder according to the present invention includes a variable length decoder (functional unit 1) and an inverse transform / inverse quantizer (functional unit 2) having a pipeline structure. ), A WMV including a plurality of functional units consisting of a predictive decoding unit / motion compensation unit (functional unit 3), an overlap transform unit (functional unit 4), and a deblocking filter (functional unit 5), and capable of accessing an external memory. An interface method between an overlap converter and a deblocking filter, the decoder comprising: a first step of determining whether the overlap converter and the deblocking filter are activated; And an adaptive pipeline structure according to the activation determined in the first step. When both the overlap converter and the deblocking filter are activated, the map data of YCbCr format stored in the external memory is used. And a second step of performing deblocking filtering.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the annexed drawings, detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention will be omitted. In general, image data is input in a large picture unit and a small pixel unit. In the present invention, an input is accepted using a macroblock unit, which is a medium size, as a reference processing unit.

As described above with reference to FIG. 1, in the hardware implementation of the WMV decoder, the variable length decoder 10 (function unit 1), the inverse transform / inverse quantizer 12 (function unit 2), and the predictive decoder / motion compensation unit (14) (Function 3) are easy to implement a macroblock, but overlap conversion section 16 (Function 4) and deblocking filter 17 (Function 5) is required to convert certain data around the macroblock. Due to structural features, macroblock implementation is not easy.

Therefore, in the present invention, in order to reduce the frequency of external memory access required by the overlap transform unit and the deblocking filter, an interface map (modified Y, Cb, Cr map structure) having YUV data in macroblock units is applied, and a frame is also applied. By applying the adaptive pipeline structure according to overlap transform and deblocking filtering defined in the header part, the WMV decoder provides optimized performance.

FIG. 5 is a flowchart illustrating an interface method between an overlap converter and a deblocking filter of a WMV decoder according to an embodiment of the present invention, and the interfacing between the overlap converter and the deblocking filter includes an overlap converter and a deblocking filter. Is determined, and an adaptive pipeline structure is applied according to the above-mentioned activation. If both the overlap converter and the deblocking filter are activated, the YCbCr format map data stored in the external memory is used. This is done by performing deblocking filtering. This will be described in detail below.

First, it is determined whether the overlap converter and the deblocking filter are activated (S10).

In general, whether overlapping conversion and deblocking filtering is performed is defined in a frame header portion having frame information, and thus can be known through frame header decoding of the WMV decoder. Specifically, it is processed according to the overlap conversion flag and loop filter flag obtained from the entry-point header. In particular, when both flags are turned on, the external memory is accessed. Deblocking filtering is performed.

Whether the overlap converter and the deblocking filter are activated is determined in four cases as follows.

Case1:

In the process of checking whether the overlap converter is activated (S12) and the process of checking whether the deblocking filter is activated (S14), the overlap converter and the deblocking filter are both deactivated. The macroblock data is processed by applying a three-stage pipeline structure.

Case2:

When the overlap converter is activated in step S12 of checking whether the overlap converter is activated, the deblocking filter is deactivated in step S18 of checking whether the overlap converter is activated. The overlap conversion is performed by applying a four-stage pipeline structure (S16).

Case3:

When the overlap converter is deactivated in step S12 of checking whether the overlap converter is in an activated state, and the deblocking filter is activated in step S18 of checking whether the deblocking filter is in an activated state, the function units 1 to 3, And deblocking filtering by applying a four-stage pipeline structure including a functional unit 5 (S20).

Case4:

In the process of checking whether the overlap converter is activated (S12) and the process of checking whether the deblocking filter is activated (S18), the overlap converter and the deblocking filter are both activated. The overlap transform (S16) and the deblocking filtering (S20) are performed by applying a five-stage pipeline structure. At this time, the deblocking filtering is performed by accessing the external memory and using map data of the YCbCr format stored in the external memory.

FIG. 6 is a diagram illustrating an external memory structure according to an embodiment of the present invention. FIG. 6A shows a logical memory structure in units of macroblocks, and FIG. 6B shows a physical memory location.

The amount of data can be reduced by using a format in which a color image is divided into a luminance component and a chrominance component, and this format is called a YCbCr format.

In the present invention, the Y component as the luminance component and the position of the YUV image data represented by the chrominance components Cb and Cr are configured in macroblock units or in a predetermined processing unit size, that is, the Y component, Cb and Cr component data are continuously formed. The memory bandwidth is reduced by allocating to read sequentially.

This generally takes into account the compatibility of the interface with other external output devices by keeping the Y, Cb, and Cr areas separate when processing or representing YUV data, but in the present invention, the modified Y, Cb, Cr maps between the overlap transform and the deblocking filtering are considered. By using the architecture, we were able to reduce the system's memory bandwidth by about 10% or more.

FIG. 7 is a diagram illustrating a pipeline structure when the overlap transform unit is deactivated and the deblocking filter is activated. That is, it is a pipeline structure of the WMV decoder with no delay associated with the case of the overlapped transform flag = 0 and the loop filter flag = 1.

Of course, even when Overlapped Transform Flag = 1 and Loop Filter Flag = 0, the same structure is used as an overlap transform instead of the process of deblocking filtering of FIG. 5.

However, when the overlapped transform flag = 0 and the loop filter flag = 0, the part called deblocking filtering of FIG. 5 is excluded.

8 is a diagram illustrating a pipeline structure when both an overlap converter and a deblocking filter are activated. That is, the pipelined structure of the WMV decoder with a delay with respect to the case of the overlapped transform flag = 1 and the loop filter flag = 1.

This will vary depending on the size of the image, but as described above with reference to FIG. 2, a delay time D of one macroblock line + one macroblock is obtained.

In the delayed pipeline structure of FIG. 8, a buffer corresponding to the delay time D is required. Since the size of the buffer is variable according to an image and its capacity is large, an external memory is used.

For reference, the difference between the external memory and the internal buffer shows a difference in access time. In the case of registers or internal memory used as internal buffers, data can be read / written in one cycle (or clock) without waiting for a certain unit of access, but external memory has a latency. It is not constant, so it is very difficult to load data at the desired time.

For example, in FIG. 1, only an internal memory (not shown) and an interface exist between the functional units 1 to 3, and an interface of the external memory 22 through a memory controller (not shown) in the functional units 4 and 5. Is present. In order to access the external memory 22, data is transmitted and received through the system bus 20 under the control of the memory controller, since the system bus 20 assigns whether the system bus 20 is available or in a standby state. When a request for data in memory is received, actual data may be shortly waited for several cycles or tens of cycles.

Thus, reducing the frequency of access of external memory provides the effect of lowering the overall bandwidth of the system. Here, the access frequency refers to access of non-contiguous addresses from 1D physical memory locations.

For example, requesting data at addresses 0, 10, 20, and 30 from external memory differs greatly from requesting data at 0, 1, 2, and 3.

In the former case, the waiting time (L0) + data processing time (No. 0) + waiting time (L1) + data processing time (No. 1) + waiting time (L2) + data processing time (No. 2) + waiting time (L3) ) + Data processing time (3 times) = Wait time * 4 + Data processing time * 4.

However, in the latter case, waiting time (L0) + data processing time (No. 0) + data processing time (No. 1) + data processing time (No. 2) + data processing time (No. 3) = waiting time + data processing time * 4 is needed.

In other words, since the external memory uses an external bus using a DMA controller, a delay in response time occurs according to a system state. Discontinuous data requests also cause delays in response time. This results in a slowdown in the processing of high frequency memory accesses, which is a factor in overall hardware performance degradation.

In order to solve this problem, the present invention provides a map of the external memory as shown in FIG. 6, and when the external memory use authority is requested to the DMA controller and receives a response, the memory access frequency is obtained by bringing as much data as possible. Is lowering. That is, when the data of the luminance component (Y) is imported, the color difference components (Cb, Cr) are also continuously imported to reduce the memory bandwidth.

Meanwhile, FIG. 9 is a diagram illustrating a macroblock unit overlap conversion process according to an embodiment of the present invention. The vertical boundary overlap conversion of the macroblock N is performed at N time, and the vertical boundary overlap conversion is performed at N-1 time. Then, the overlap conversion is performed for every macroblock by the horizontal overlap conversion of the stored macroblock N-1.

FIG. 10 is a diagram illustrating a macroblock-based deblocking filter process according to an embodiment of the present invention, and enables macroblock processing by delaying 8x8 blocks to the left and 8x8 blocks to the left, and the deblocking filter is applied to the horizontal filtering result. Again, the filtering priority is important because it's a vertical filtering method.

(1) Perform 8x8 horizontal filtering

(2) Perform 4x4 horizontal filtering on the result of (1) above

(3) Perform 8x8 vertical filtering on the results of (1) and (2) above

(4) Perform 4x4 vertical filtering on the results of (1), (2) and (3) above.

By the above-described filtering priority, as shown in FIG.

FIG. 11 is a diagram illustrating an external memory map in an overlap conversion process according to an embodiment of the present invention, wherein the overlap conversion output has a map data structure in YCbCr format according to the present invention. Thereafter, deblocking filtering is performed using the map data of the YCbCr format.

While the above has been illustrated and described with respect to the preferred embodiments of the present invention, the present invention is not limited to the specific embodiments described above, it is usually in the art to which the subject innovation belongs without departing from the spirit of the invention claimed in the claims. Various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

As described above, in the present invention, since the memory bandwidth between the overlap converter and the deblocking filter, which is a performance problem when the WMV decoder is implemented in hardware, can be reduced, the processing speed and power consumption of the entire decoder can be reduced. .

In addition, the adaptive pipeline structure according to whether the overlap transform and the deblocking filtering is activated according to the present invention has an advantage of providing optimized performance according to the bitstream configuration.

Claims (8)

Variable length decoder (function unit 1), inverse transform / dequantizer (function unit 2), predictive decoder / motion compensation unit (function unit 3), overlap transform unit (function unit 4), deblocking filter of the pipeline structure In the WMV decoder including a plurality of functional units constituted by (functional unit 5) and capable of accessing an external memory, the interface method between the overlap converter and the deblocking filter, Determining whether the overlap converter and the deblocking filter are activated; And A five-stage pipe including the functional unit 1 to the functional unit 5 is applied to the adaptive pipeline structure depending on whether the activation is determined in the first step, and when the overlap converter and the deblocking filter are both activated. And a second step of performing overlap transform and deblocking filtering by applying a line structure. Variable length decoder (function unit 1), inverse transform / dequantizer (function unit 2), predictive decoder / motion compensation unit (function unit 3), overlap transform unit (function unit 4), deblocking filter of the pipeline structure In the WMV decoder including a plurality of functional units constituted by (functional unit 5) and capable of accessing an external memory, the interface method between the overlap converter and the deblocking filter, Determining whether the overlap converter and the deblocking filter are activated; And An adaptive pipeline structure is applied according to whether the activation is determined in the first step, and when the overlap converter and the deblocking filter are both deactivated, a three-stage pipe including the functional units 1 to 3 is provided. And a second step of processing macroblock data by applying a line structure to the overlapping block and the deblocking filter of the WMV decoder. Variable length decoder (function unit 1), inverse transform / dequantizer (function unit 2), predictive decoder / motion compensation unit (function unit 3), overlap transform unit (function unit 4), deblocking filter of the pipeline structure In the WMV decoder including a plurality of functional units constituted by (functional unit 5) and capable of accessing an external memory, the interface method between the overlap converter and the deblocking filter, Determining whether the overlap converter and the deblocking filter are activated; And A four-stage pipe including the functional unit 1 to the functional unit 4 is applied if the adaptive pipeline structure is applied according to the activation determined in the first step, and the overlap converter is activated and the deblocking filter is deactivated. And a second step of performing an overlap transformation by applying a line structure. 2. The interface method between the overlap transform unit and the deblocking filter of the WMV decoder. Variable length decoder (function unit 1), inverse transform / dequantizer (function unit 2), predictive decoder / motion compensation unit (function unit 3), overlap transform unit (function unit 4), deblocking filter of the pipeline structure In the WMV decoder including a plurality of functional units constituted by (functional unit 5) and capable of accessing an external memory, the interface method between the overlap converter and the deblocking filter, Determining whether the overlap converter and the deblocking filter are activated; And An adaptive pipeline structure is applied according to whether the activation is determined in the first step, but when the overlap converter is deactivated and the deblocking filter is activated, the functional units 1 to 3, and the functional unit 5. And a second step of performing deblocking filtering by applying a four-stage pipeline structure including a second stage pipeline structure including an overlapping conversion unit and a deblocking filter of the WMV decoder. The method of claim 1, The deblocking filtering is performed by using the map data of the YCbCr format stored in the external memory. The interface method between the overlap converter and the deblocking filter of the WMV decoder. The method of claim 5, And the map data in the YCbCr format is sequentially allocated to read Y component, Cb and Cr component data so that the map data of the YCbCr format can be sequentially read. The method of claim 6, And the Y component, Cb, and Cr component data are allocated on a macroblock basis. The interface method between the overlap converter and the deblocking filter of the WMV decoder. The method according to any one of claims 1 to 7, Whether the overlap converter and the deblocking filter are activated is obtained by performing frame header decoding of the decoder. The interface method between the overlap converter and the deblocking filter of the WMV decoder.

Images