KR20160125712A - Simplified Rate-Distortion calculation method considering quantization parameters for a hardware-based sample adaptive offset in hevc encoder - Google Patents

Abstract

The present invention relates to a method for calculating an aldel value considering a quantization value for Esoio of a hardware based H.264 codec, A method for calculating an Al-Dy value considering a quantization value for a S-io of a hardware-based H.264 codec, the method comprising: a first step of calculating an offset for RDO of a current CTU; A second step of determining optimal SAO parameters through RDO operation; And a third step of applying SAO to a current CTU having an optimal SAO parameter. According to the ALD optimization method according to the present invention, hardware complexity can be reduced by not employing CABAC when designing a hardware-based HEVC encoder, and a CTU schedule shifted between DF and SAO is adopted to smoothly perform pipeline operation This is possible.

Description

[0001] The present invention relates to an Al-Dy calculation method considering a quantization value for Esio of a hardware-based H.264 codec,

More particularly, the present invention relates to a method for calculating Al-Dy considering a quantization value for S-IE in a hardware-based H.264 codec, and more particularly, The present invention relates to a method for calculating an al-Dey calculation considering a quantization value for Esoio of a hardware based H-Viteron coder designed considering a SAO operation in a shifted CTU schedule.

In general, image compression technology has attracted attention in various industrial fields due to explosive demand for high-definition (HD) image contents as well as video applications such as video conferencing, streaming, image storage and video communication. At this time, H.264 / AVC is the latest and most widely used image coding technology.

Recently, HEVC (High Efficiency Video Coding), which is known as a next generation image coding technology standard, has been proposed. The HEVC may comprise a flexible block structure, an increased intra-coding direction, sophisticated interpolation filters, various in-loop filters, Several new forms have been introduced, such as enhanced entropy coding schemes.

The HEVC (High Efficiency Video Coding) standard is a next-generation video compression standard developed by the joint collaborative team on video coding (JCT-VC), and has a coding efficiency provided by H.264 / AVC (Advanced Video Coding) To be doubled. Like H.264 / AVC, the HEVC is an in-phase and out-of-phase method to reduce artifacts such as blocking, ringing, color biases, blurring, Loop filtering is adopted. Compared to H.264 / AVC, HEVC provides a larger conversion size and an increased tab in interpolation. These altered facets can cause even more severe ringing artifacts. In order to reduce these artifacts, not only conventional DF (deblocking filter) but also SAO (sample adaptive offset) and ALF (adaptive loop filter) have been proposed. Due to the high complexity of the ALF among the above three schemes, DF and SAO have recently been included in the HEVC standard.

On the other hand, an image encoder used in HEVC (High Efficiency Video Coding) is capable of encoding an input image in an intra mode or an inter mode and outputting a bit stream. A SAO processing device of a video compression system of -0098481 and a Sample Adaptive Offset (SAO) which is one of in-loop filters as shown in the method are essentially provided.

The SAO improves picture quality by dividing the reconstructed sample obtained from the deblocking filter into several categories according to the criteria and then adding different offsets to each sample in each category to improve the picture quality. %, Respectively.

On the other hand, the improvement of image quality through SAO is achieved by finding the optimal parameter for the reconstructed sample. In this case,

RD Optimization (RDO) must be performed using the bit allocation of each parameter in order to select. In this case, the delta RD cost used in the SAO is calculated by using the distortion and the bit rate. Although the delta RD cost is an integer calculation, since the bit rate is a real number calculation, there is a difficulty in hardware implementation.

Reference Document: Published Patent No. 10-2013-0098481

Accordingly, the present invention has been made to solve the above problems, and it is an object of the present invention to provide a hardware-based HEVC encoder having no CABAC in order to reduce the complexity of hardware, a CTU schedule shifted between DF and SAO And a method of estimating the SAO operation in a shifted CTU schedule and carefully designing the sample loading order. The object of the present invention is to provide an ALD calculation method considering a quantization value for Esoio in a hardware based HVC encoder .

In order to solve such a technical problem,

A method for calculating an Al-Dy value considering a quantization value for a S-io of a hardware-based H.264 codec, the method comprising: a first step of calculating an offset for RDO of a current CTU; A second step of determining optimal SAO parameters through RDO operation; And a third step of applying the SAO to the current CTU having the optimal SAO parameter. The hardware-based HVC encoder according to the present invention provides an al-Dy calculation method considering the quantization values for SIEO.

Here, the first step is calculated by adding the difference between the pre-SAO sample and the source sample for each band of the BO in each category of EO in the statistical calculation unit of the SAO module, For each band, the difference between the pre-SAO sample and the source sample is summed and stored in the statistics_table together with the number of samples belonging to each category and band.

In the second step, the optimal parameter of the SAO is determined by comparing the delta RD cost calculated from the delta_distortion_calculator and the bit_rate_estimator in the RDO part of the SAO module, and the optimal parameter information determined by the RDO part is determined by the line_memory and stored in the best_parameter_buffer.

The third step is a step of obtaining the optimal SAO parameter and the pre-SAO samples from the external_memory, line_memory, pre-SAO_sample_buffer and best_parameter_buffer in the SAO module of the SAO module.

According to the Al-Dy calculation method considering the quantization value for Esoio of the hardware-based H.264 codec according to the present invention, not only hardware complexity can be reduced since CABAC is not adopted in designing a hardware-based HEVC encoder Employing a shifted CTU schedule between DF and SAO, smooth pipeline operation is possible.

1 is a hardware configuration diagram for image processing according to the present invention.
FIG. 2 shows four one-dimensional directivity patterns for EO sample classification.
Figure 3 is a diagram showing five categories for EO sample categorization.
Fig. 4 is a diagram showing the type determination in RDO. Fig.
5 is a table showing average bit sizes for BAC SAO parameters in regular mode.
6 is a table showing estimated bit sizes for SAO parameters.
Figure 7 is a diagram illustrating shifted CTU processing for DF and SAO.
8 is a view showing a SAO structure according to the present invention.
9 is a table showing the results of RD execution of the SAO module according to the present invention.

Hereinafter, the features of the hardware-based H.264 codec according to the present invention will be described in detail with reference to the accompanying drawings.

Prior to this, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms, and the inventor should appropriately interpret the concepts of the terms appropriately It should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention based on the principle that it can be defined.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It should be understood that various equivalents and modifications may be present.

Referring to FIG. 1, an encoder 1, which compresses and uses an image in HEVC (High Efficiency Video Coding), encodes an input image in an intra mode or an inter mode, can do.

The video decoder 2 receives the bit stream outputted from the encoder 1 and decodes the video stream into the intra mode or the inter mode, and outputs the reconstructed video, that is, the reconstructed video. At this time, in the intra mode, the switch is switched to the intra mode, and in the inter mode, the switch can be switched to the inter mode.

At this time, the HEVC encoder 1 essentially includes a sample adaptive offset (SAO) 100, which is one of the in-loop filters. The SAO 100 is provided with a deblocking filter The resulting reconstructed sample is divided into several categories according to the criteria, and the offset is added to the sample for each category to improve the image quality. The improvement of the image quality through the SAO (100) is achieved by finding the optimal parameter for the reconstructed sample. To select the optimal value among the multiple parameter candidates, RDO (RD Optimization) must be performed using the bit allocation amount of each parameter.

More specifically, in HEVC, SAF is executed after DF is applied first during in-loop filter processing. The SAO's basic concept is to reduce the average distortion of the samples by adding offsets to the reconstructed sample, which is performed by the coding tree unit (CTU). In this case, there are two design changes to be considered for hardware-based SAO.

First, in block-based processing, the DF for the right and bottom samples in the current CTU can not be completed. Because the right and bottom samples request invalid samples of the next neighboring CTUs. The SAO should be executed using the samples for which DF has already been processed. Thus, the SAO can not be executed for the right and bottom samples of the current CTU.

Second, the SAO requires rate-distortion optimization (RDO) to determine the optimal SAO parameters taking into account the bits of the SAO parameters and the distortion reduced by the SAO. To calculate the bit size required for RDO, context adaptive binary arithmetic coding (CABAC) is used in the HM reference software. This means that independent CABAC hardware logic is required for the SAO module.

Therefore, the present invention proposes two schemes for hardware-based SAO. In order to reduce hardware complexity, we propose a simplified bit rate estimator without CABAC. For smooth pipeline operation, we adopt a shifted CTU schedule between DF and SAO and carefully design sample loading order considering SAO operations within the shifted CTU schedule.

Hereinafter, the RDO flow will be described in detail in the SAO standard and the HM reference software in the HEVC.

In SAO parameters, SAO has three types of offsets: NO (None Offset), EO (Edge Offset) and BO (Band Offset), and each luma coding tree block (CTB) and chroma CTB Type.

At this time, the type NO indicates that the SAO is not applied to the current CTB.

FIG. 2 is a diagram showing four one-dimensional directivity patterns for EO sample classification. The horizontal direction (EO class = 0), vertical (EO class = 1), 135.degree. Diagonal (EO class = EO class = 3). Referring to this, the type EO has four classes of horizontal, vertical, 135 ° diagonal, and 45 ° diagonal. Each square represents a sample. The method for applying the offset depends on the edge direction of the samples of the coding tree block (CTB). Sample c represents the current sample, while sample a, b represent neighboring samples, and each class is divided into five different categories. Depending on the categories, positive or negative offsets are applied to the current sample.

Next, FIG. 3 shows five categories for EO sample categorization. The horizontal axis identified in FIG. 1 represents the sample index, while the vertical axis represents the sample value. As can be seen in Fig. 3, the categories are determined by the relative relationship between the two neighboring samples (a, b) and the current sample (c), and no offset is applied to the sample with category '0' . The BO type tries to add similar offsets to similar-level samples, where the sample range is divided into 32 bands for 8-bit samples ranging from 0 to 255, and each band contains 8 sample levels. Of the 32 bands, four consecutive bands are selected.

As described above, in order to obtain the bits used in the SAO parameters, the HEVC adopts the SAO merging scheme that shares the SAO parameters of the neighboring CTUs for the current CTU. Since the SAO parameters for the current CTU are not transmitted to the decoder, if the current CTU and the neighboring CTU are merged, the bit size will be effectively stored.

Next, we explain the offset determination process of SAO based on RDO (R-D Optimization).

For RDO (R-D Optimization), new parameters are applied every hour, and distortion between original and redonstructed samples must be calculated repeatedly. This increases computational complexity and memory accesses.

Therefore, in order to reduce computational complexity, a fast distortion estimation scheme is proposed in advance and is included in the HM reference software. The basic idea of this scheme is that distortion is replaced by delta distortion between pre-SAO and post-SAO samples. The negative delta distortion shows that the distortion is reduced after SAO, whereas the positive delta distortion shows that the distortion increases when the current parameters are used. With delta distortion (ΔD), the delta RD cost (ΔJ) is calculated as in equation (1).

? J =? D +? R Equation (1)

Where lambda is the Lagrangian multiplier and R is the bit size of the SAO parameters. According to equation (1), the RDO of the SAO module must determine the SAO parameters that minimize the delta RD cost (ΔJ). SAO's RDO consists of two stages. In the first step, the offsets of EO and BO are determined, and the optimal SAO type is selected in the second step.

First of all, looking at the offset determination, EO has four classes, each class has four categories that require non-zero offset, so there are 16 offsets for EO and BO has 32 offsets for 32 bands Are needed. A total of 48 optimal offsets must therefore be determined. For distortion, the optimal offset is the mean difference between the original and reconstructed samples, denoted DAVG. However, considering both distortion and bit size, offsets from zero to DAVG should be searched. Although a particular offset is not optimal for distortion, its RD cost may be less than that of DAVG because a small offset in bit size is advantageous.

Next, looking at the type determination, Figure 4 shows the flow of RDO for type determination after 48 offsets have been determined for EO and BO. The white rectangle R1 represents the RDO decision for each luma, Cb and Cr CTBs and the light gray rectangle R2 represents the RDO decision for the luma and chroma (Cb + Cr) CTBs, while the dark gray rectangle (R3) represents the RDO decision for most CTBs. For example, luma, Cb, and Cr CTBs can have independent bands in BO, whereas luma CTBs and chroma CTBs can have independent EO classes, but Cb and Cr CTBs must have the same class do. There is only one kind of merging for most CTUs. Between the four EO classes, one class with a minimum delta RD cost (DELTA J) is selected. The delta RD cost in the BO is calculated for each band. Thereafter, four consecutive bands with the smallest sum of delta RD costs are selected. Next, the optimal EO type and the optimal BO type are compared with the NO type, and the optimal type with the minimum delta RD cost (ΔJ) is selected from the three. The delta RD cost (ΔJ) of the NO type can be easily obtained from the required bits because NO has zero delta distortion. Finally, the optimal type of delta RD cost (ΔJ) so far can be compared to the delta RD cost of Merge_Left and Merge_Up. The delta RD cost (DELTA J) of the merge type is computed from the delta distortion with the bits for coding the parameters of the neighboring CTU and the merge flags.

The proposed schemes for hardware implementation of SAO are described below.

First, let's look at the simplified RDO.

Direct implementation of RDO in HM reference software in hardware has difficulties. For calculation of delta RD cost (ΔJ), CABAC requires measurement of the number of bits of SAO parameters of R in Eq. (1). For RDO, it is inefficient to include complex CABAC modules in SAO hardware for only bit counting. CABAC has a coding mode of 2, a regular mode in which the BAC is performed in a context model and a bypass mode in which the context is not used to perform the BAC.

The regular mode is mainly used for coding syntax elements having a strong dependence on the current coding context, while the bypass mode is used for coding the syntax elements having weak relevance to the current coding context It is used to encode syntax elements, but the bypass mode is faster than regular mode, but has no compression gain.

In the bit counts generated, the number of bins of the syntax element is equal to the number of bits encoded by the BAC when the bypass mode is used, while the regular mode is the context model After the BAC with the context model, it generates compressed bits from the bin of syntax elements.

In the present invention, a simplified bit estimator is proposed to estimate the number of bits without independent CABAC. The number of bits for the bypass mode can be easily obtained from the number of bin of syntax elements. The number of bits for regular mode is estimated from statistical data obtained from experiments with various video sequences and quantization parameters (QPs). The table shown in FIG. 5 shows average bits of SAO parameters encoded in a regular mode in the BAC. The 'Basketball Drive' and the 'BQTerrace' having a resolution of 1920 × 1080 and a resolution of 832 × 480 Four video sequences of 'PartyScene' and 'Flowervase' are used for evaluation. The number of frames is '50', and four quantization parameters (QPs) 12, 22, 32 and 42 are used.

The first column of the table shown in FIG. 5 represents the video sequence and the second column represents the QP, while the average of the bits generated for each SAO parameter appears in the third and fourth columns. In low QPs, the bit size shows a somewhat irregular dependence on the video sequences. However, since the influence of the bit size R of the SAO parameters in equation (1) is very small, this variation can be tolerated in low QPs and the bit sizes in the middle and high QPs are fairly consistent regardless of the video sequences.

The estimated bits of each SAO parameter based on the table observations of FIG. 5 are shown in the table of FIG. The first column of the table shown in FIG. 6 represents the SAO parameter, while the second column represents the CABAC coding mode for the corresponding SAO parameter. The third through the middle columns represent estimates of the bits for encoding each SAO parameter that is dependent on the QP. The estimated bits for the BO band position, the EO class, and the offset sine offset Abs, which are encoded and provided by the bypass mode, are exactly the same as the actual CABAC coding, while the estimation bits for the SAO merging and type May differ from the bit sizes of the actual CABAC coding. Such inaccuracies can lead to erroneous decisions in RDO processing. However, the degradation of RD execution caused by the proposed estimation bits is insignificant. The results of this experiment can be confirmed from the experimental results described later.

Next, the recorded sample loading process in the shifted CTU process will be described.

For DF execution for samples at the boundary of two CTUs, the prediction and reconstruction must be completed for the two CTUs. In this way, in CTU-based processing, DF can not be applied to the boundary samples of the current CTU before the prediction and reconstruction of all neighboring CTUs is finished.

At the CTU boundary for the luma elements, DF is applied to the four samples on the right and left sides, whereas for CTU boundaries for the chroma elements, DF is performed on the two samples on the right and left sides. Thus, in the current CTU phase, DF processing is not possible for the right and bottom samples of the luma elements and the right and bottom samples of the chroma elements. The problem is that SAO should be applied to samples processed by DF. In addition, as shown in FIG. 2, the EOs of the current samples require neighboring samples. If neighboring samples a and b are not valid, sample c can not be SAO. Thus, in the current CTU phase, the SAO for the right and bottom samples of the luma elements and the right and bottom samples of the chroma elements can not end. In the previous work for the HEVC decoder, it is proposed that the pipeline schedule between DF and SAO assume that the DF is processed through the shifted positions in the left and upper directions, as in the four samples. The SAO is then processed through the shifted positions in the left and upper directions, as in the case of the five samples.

In the present invention, a shifted CTU pipeline schedule as shown in Fig. 7 is applied for the encoder 1 between DF and SAO. The solid line in FIG. 7 represents the CTU boundary, the light gray region represents the samples to which DF is applied, while the dark gray region represents the samples to which SAO and DF are applied. The black region represents only the samples to which the SAO is applied. Unlike the SAO in the decoder 2, for the SAO 100 in the encoder 1, statistical calculation and RDO for obtaining the optimal parameters of the current CTU are required. For SAO processing, it is imperative to provide a statistical calculation module having samples of the current CTU to determine optimal parameters. In this way, the samples of the current CTU are loaded first, and the statistics computation and RDO are performed. Then, SAO processing is executed for SAO appearing as black and dark gray color as shown in Fig. At this time, in order to execute the SAO for the right and bottom of the current CTU in the next CTU phase, the optimal SAO parameters and pre-SAO samples of the current CTU are stored in the internal buffer and line memory.

Hereinafter, a hardware-based SAO structure according to the present invention will be described with reference to FIG. According to this, the SAO hardware encoder is divided into three parts. In the statistic calculating unit 110, an offset is calculated for RDO of the current CTU. In the RDO unit 120, the optimal SAO parameters are determined through the RDO operation. In the SAO unit 130, the SAO calculates the optimal SAO parameters Currently applied to CTU.

More specifically, the statistic calculation unit 110 calculates the sum of the difference between the pre-SAO sample and the source sample for each category of EO and each band of the BO. The sum computed by the statistic calculation unit 110 is stored in the statistics_table 112 along with the number of samples belonging to each category and band.

The RDO unit 120 compares the delta RD cost calculated from the delta_distortion_calculator 122 and the bit_rate_estimator 124 to determine the optimal parameter of the SAO. The optimal parameter information determined in the RDO unit 120 is stored in the line_memory 140 and the best_parameter_buffer 132 for the next CTU.

The SAO unit 130 performs SAO in four different CTU areas at one time due to the shifted CTU process as shown in FIG. Therefore, the optimal SAO parameters and pre-SAO samples are obtained from external_memory (150), line_memory (140), pre-SAO_sample_buffer (134) and best_parameter_buffer (132).

The experimental result according to the above conditions is the same as the RD execution table of the proposed SAO module of FIG. At this time, in the present invention, the simplified RDO and the SAO module to which the shifted DTU is applied are executed in the Verilog RTL. The table of FIG. 9 shows RD execution of the proposed RDO, in which nine video sequences are used for the experiment. At this time, the number of frames is 50, and its arrangement is lowdelay_main (LD) and randomaccess_main (RA).

The first and second columns of the table shown in FIG. 9 represent video sizes and test sequences, and the third through fourth columns indicate the Bjontegaard Delta Bitrate (BD- BR) and Bjontegaard Delta PSNR (BD-PSNR), while the seventh to tenth columns denote BD-BR and BD-PSNR with high QP values of 32, 37, 42 and 47, respectively. As shown in the table of FIG. 9, for LD and RA, the increase in BD-BR is less than 0.30% in the mean with almost 0.011 dB of BD-PSNR with various QPs.

The SAO module according to the present invention synthesizes 65 nm technology. The gate counts are 158K, while the maximum operating clock frequency is 300MHz. For shifted CTU processing, a 64Kbit line memory is used for pre-SAO sample data and optimal parameters. The proposed SAO supports 32 × 32 CTUs and its throughput is about 350000 CTU / s at 300 MHz.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The scope of protection of the present invention should be construed under the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

1: Encoder 2: Decoder
100: SAO 110: Statistical calculation unit
120: RDO unit 130: SAO unit

Claims (6)

A method for calculating an Al-Dy value considering a quantization value for Esoio of a hardware-based H.264 codec,
A first step of calculating an offset for RDO of the current CTU; A second step of determining optimal SAO parameters through RDO operation; And a third step of applying SAO to a current CTU having an optimal SAO parameter. The method of claim 1,
The method according to claim 1,
Wherein the first step is calculated by adding the difference between the pre-SAO sample and the source sample for each band of the BO in each category of EO in the statistical calculation unit of the SAO module. Al - Dy calculation method considering quantization value for Ao.
3. The method of claim 2,
The difference between the pre-SAO sample and the source sample for each band of the BO in each category of the EO is summed and stored in the statistics_table together with the number of samples belonging to each category and band. Al - Dy calculation method considering the quantization value for Esoio of the.
The method according to claim 1,
Wherein the second step is a step of comparing the delta RD cost calculated from the delta_distortion_calculator and the bit_rate_estimator in the RDO part of the SAO module to determine the optimal parameter of the SAO module. Al-Dy calculation method considering value.
5. The method of claim 4,
And the optimal parameter information determined by the RDO unit is stored in a line_memory and a best_parameter_buffer for the next CTU, and a simplified algorithm optimization method for hardware-based SIA.
The method according to claim 1,
Wherein the third step is a step of obtaining the optimal SAO parameter and pre-SAO samples from an external_memory, a line_memory, a pre-SAO_sample_buffer and a best_parameter_buffer in the SAO module of the SAO module. Lt; RTI ID = 0.0 > a < / RTI > quantization value.

Images