CN118301370B - Wavelet fast transformation method for JPEG-XS encoding and decoding - Google Patents

Abstract

The invention relates to a wavelet rapid transformation method for JPEG-XS encoding and decoding, which comprises the following steps: dividing frame data to generate a wavelet unit; vertical horizontal wavelet transform; distinguishing and respectively processing; a half-image width horizontal wavelet transform; three wavelet transforms. The invention reclassifies the data of a JPEG-XS coding image according to the characteristics of the coding flow, and carries out vertical-horizontal wavelet transformation on all image data in the original coding process; and then carrying out vertical-horizontal wavelet transformation or horizontal wavelet transformation on the 1/4 image, and optimizing the flow of carrying out three times of horizontal wavelet transformation. The buffer memory of the intermediate transformation result is reduced by adopting a blocking mode, the reading and writing of intermediate data are also reduced, and the coding efficiency is greatly improved. The data driving DWT mode can further reduce the buffer memory and the read-write of intermediate data, thereby improving the coding efficiency.

Description

A fast wavelet transform method for JPEG-XS encoding and decoding

Technical Field

The invention relates to a wavelet fast transform method for JPEG-XS encoding and decoding, a data transmission method for a computer network, and a method for processing and transmitting network ultra-high definition video data.

Background Art

JPEG-XS is the latest international standard based on the shallow compression domain of wavelet compression technology, which can generate higher quality images with low latency. JPEG-XS can record 8K (7680×4320), 4K (3840×2160) and HD (1920×1080) 422 10-bit videos at a reasonable bit rate, with a compression ratio of 10:1-6:1. It has the characteristics of visual lossless quality, good multi-generation replication characteristics, low encoding and decoding complexity and low encoding and decoding latency, which meets the dual requirements of low latency and high quality for radio and television programs, and can significantly reduce the pressure of ultra-high-definition services on data storage, streaming transmission and video processing calculations. JPEG-XS coding uses wavelet transform, quantization, entropy coding and other coding techniques. Wavelet transform first performs vertical transformation on the column data of the image, then stores the intermediate results of column processing, and then performs horizontal transformation on the row data, which requires at least one frame of intermediate storage space for data. The required storage space is about 200Mb under 8K 422 10bit. This requires a large-capacity memory to cache the column transformation results, which wastes a lot of storage resources and increases the overhead of reading and writing data. The increase in storage overhead reduces the utilization of hardware and limits the speed of encoding and decoding. On the other hand, with the continuous development of information technology, the advantages of parallel processing of massive data in memory bandwidth are becoming more and more obvious. However, due to the vertical wavelet transform of JPEG-XS coding, this parallel processing is difficult to carry out. At the same time, the vertical transform operates on the column data, which cannot fully utilize the characteristics of unrelated encoding between slices, does not give full play to the potential of parallel coding, and affects the coding efficiency. Therefore, it is urgent to find an efficient implementation method for DWT/IDWT of JPEG-XS coding and decoding.

Summary of the invention

In order to overcome the problems of the prior art, the present invention proposes a wavelet fast transform method for JPEG-XS encoding and decoding. The method is based on the characteristics of the JPEG-XS frame structure, adopts a block-based approach to reduce the intermediate transform result cache, and also reduces the reading and writing of intermediate data, greatly improving the encoding efficiency. The use of a data-driven DWT approach can further reduce the caching and reading and writing of intermediate data, thereby improving the encoding efficiency.

The object of the present invention is achieved by: a wavelet fast transform method for JPEG-XS encoding and decoding, the steps of the method are as follows:

Step 1, frame data is divided into wavelet transform units: according to the encoding parameters, the frame data is divided into a number of pixel blocks of frame_width×K×2 ^v as wavelet transform units, where frame_width is the image pixel width, v is the number of vertical wavelet decompositions, K is a positive integer with a maximum value of frame_width/2 ^v , frame_height is the image pixel height, and 2 ^v is the pixel height of precinct;

Step 2, first or second vertical-horizontal wavelet transform: divide the wavelet transform unit in step 1 into 8×8 pixel blocks, perform vertical wavelet transform from right to left and from top to bottom, and use the transform result as input data; then check whether the input data is sufficient for horizontal wavelet transform, if not, return to repeat vertical wavelet transform, if sufficient, perform horizontal wavelet transform; after two transformations, the wavelet coefficients generated by each 8×8 pixel block can be divided into four sub-bands, namely LL, HL, LH, and HH; LL represents the low-frequency components of the image in the horizontal and vertical directions, HH represents the high-frequency components of the image in the horizontal and vertical directions, LH represents the low-frequency components of the image in the horizontal direction and the high-frequency components in the vertical direction, and HL represents the high-frequency components of the image in the horizontal direction and the low-frequency components in the vertical direction; classify and save all four sub-band coefficients, output the LL sub-band coefficient, and prepare to continue the wavelet transform;

Step 3, identify and process separately: if it is Main Profile, jump directly to step 4; if it is High Profile, repeat step 2 with the LL wavelet coefficients calculated in step 2, then skip step 4 and go directly to step 5;

Step 4, horizontal wavelet transform of half image width: check whether the LL data is sufficient for a horizontal wavelet transform. If not, return to step 2, otherwise perform a horizontal transform. The transform result can be divided into two sub-bands, LL and HL. All LL sub-band data and HL sub-bands are saved in sequence, and the LL sub-band coefficients are output, ready to continue the wavelet transform.

Step 5, three-level horizontal wavelet transform: check whether the input data is sufficient for three-level horizontal wavelet transform. If not, return to step 2, otherwise perform horizontal transform. The transform result can be divided into two sub-bands, LL and HL. All LL sub-band data and HL sub-bands are saved in sequence. The LL sub-band data is used as the input for the next transform, and the output still needs to be classified and saved in sequence. Repeat three times to finally obtain the transform data required for entropy coding.

The advantages and beneficial effects of the present invention are as follows: the present invention re-divides the data of a JPEG-XS coded image according to the characteristics of the coding process, and optimizes the original coding process from first performing vertical-horizontal wavelet transform on all image data; then performing vertical-horizontal wavelet transform (High Profile) or horizontal wavelet transform (Main Profile) on 1/4 image, and then performing three horizontal wavelet transforms. The optimized coding process can encode the minimum coding unit, i.e., slice, respectively, avoiding the storage of a large number of intermediate results, reducing hardware overhead, and improving hardware utilization. The wavelet transform method of the prior art caches a large amount of intermediate transformation data, and introduces the reading and writing of these data, which is the main bottleneck of the current CPU architecture encoding performance. The present invention adopts a block-by-block method to perform DWT on image pixels. The block-by-block method is used to reduce the intermediate transformation result cache, and also reduces the reading and writing of intermediate data, greatly improving the coding efficiency. The present invention also adopts a data-driven method to perform DWT, and the data output from each transformation is used as the input data of the next level DWT transformation. When the next level DWT transformation detects that there is enough input data, it starts to transform and output until all data are transformed. The data-driven DWT approach can further reduce the caching and reading and writing of intermediate data, thereby further improving coding efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

FIG1 is a flow chart of the JPEG-XS encoding process;

FIG2 is a schematic diagram of the wavelet transform process of JPEG-XS;

Figure 3 is a schematic diagram of wavelet decomposition of High Profile;

FIG4 is a schematic diagram of a JPEG-XS frame structure;

FIG5 is a flow chart of the method according to an embodiment of the present invention.

DETAILED DESCRIPTION

Example:

This embodiment is a wavelet fast transform method for JPEG-XS encoding and decoding.

JPEG-XS uses coding techniques such as DWT, quantization, and entropy coding. The encoding and decoding of each frame does not need to refer to other frames. The encoding process flow chart is shown in Figure 1. In order to allow different levels of delay and complexity, JPEG-XS defines different profiles. The maximum number of vertical wavelet decompositions for each profile is defined as follows:

Light Profile has 5 layers of decomposition horizontally and 0 layers of decomposition vertically

Main Profile is decomposed into 5 layers horizontally and 1 layer vertically

High Profile has 5 horizontal layers and 2 vertical layers

Since LightProfile does not require vertical transformation and does not have the above problem, the data division of this profile in step 1 is relatively simple, but the benefits described in the present invention can still be obtained by using a data-driven approach only in the horizontal transformation process. The wavelet transform in the JPEG-XS encoding process is shown in Figure 2. Taking HighProfile as an example, all column data are first vertically transformed in sequence, and all rows of the transformation results are horizontally transformed again. The LL band that has undergone a vertical transformation and a horizontal transformation respectively repeats the vertical transformation and the horizontal transformation once, and then each row in the LL band result is horizontally transformed three times. A total of 2 vertical transformations and 5 horizontal transformations were performed, and the final result of the transformation is shown in Figure 3. The types of bands obtained by the transformation are shown in Table 1. For the entire image, each pixel will generate a corresponding wavelet coefficient, and the final output and input coefficient matrix of the same size will be used for subsequent encoding quantization, entropy coding, etc.

0 LL _5,2 1 HL _5,2 2 HL _4,2 3 HL _3,2 4 HL _2,2 5 LH _2,2 6 HH _2,2 7 HL _1,1 8 LH _1,1 9 HH _1,1

Table 1 Correspondence between bands after wavelet transformation of High Profile

The smallest coding unit of the JPEG-XS format is a slice, which consists of several precincts. The width of the precinct is generally the width of the image, and the height is 2 ^v (v is the number of vertical decomposition layers). Taking 8K resolution High Profile as an example, the width of the precinct is 7680 pixels and the height is 4 pixels. Within a slice, the precinct below can be predicted using the precinct above; slices are independent of each other and have no dependencies. The frame structure of JPEG-XS is shown in Figure 4. In this diagram, the slice height is 4 precincts, with 5 levels of horizontal decomposition and 2 levels of vertical decomposition. The medium thick line represents the precinct boundary, and the thick line represents the image boundary. The precincts with dotted lines and thin lines belong to different slices.

From the wavelet transform process of JPEG-XS encoding, it can be found that vertical transformation and horizontal transformation need to be calculated on one column and one row of image data respectively, and the calculation results need to be saved to the cache for subsequent transformation. This affects the parallel encoding between slices and does not fully utilize the structural characteristics of JPEG-XS. On the other hand, it also consumes a lot of storage space and read and write time, affecting the encoding efficiency.

This embodiment designs and implements an efficient wavelet transform method based on the characteristics of the JPEG-XS coded image structure, which greatly improves the coding efficiency. At the same time, JPEG-XS decoding is the inverse operation of the encoding process, and the IDWT in decoding is the inverse process of DWT. The transformation method described in this embodiment can also be used for the IDWT of JPEG-XS to improve the decoding efficiency.

The existing wavelet transform first performs vertical and horizontal wavelet transform on all image data, then performs vertical and horizontal wavelet transform (High Profile) or horizontal wavelet transform (Main Profile) on 1/4 of the image, and then performs a third horizontal wavelet transform. Each wavelet transform must be stored, which increases hardware overhead.

This embodiment optimizes this process. The optimized encoding process can encode the smallest encoding unit, namely, slice, respectively, thereby avoiding the storage of a large number of intermediate results, reducing hardware overhead, and improving hardware utilization.

The encoding process described in this embodiment is to first re-divide the data of a JPEG-XS encoded image according to the characteristics of the encoding process in a data-driven manner, dividing it into wavelet transform units and pixel blocks, and then perform wavelet transform on each of the different profiles, thereby saving the access hardware overhead in the wavelet transform process.

The wavelet fast transform method for JPEG-XS codec described in this embodiment includes the following steps, and its flow is shown in FIG5 :

Step 1, frame data is divided into wavelet transform units: according to the encoding parameters, the frame data is divided into several pixel blocks of frame_width×K× ^2v as wavelet transform units, where frame_width is the image pixel width, v is the number of vertical wavelet decompositions, K is a positive integer with a maximum value of frame_height/ ^2v , which means that the entire image is a transform unit, frame_height is the image pixel height, and ^2v is the pixel height of precinct.

The value of K needs to take into account the number of precincts contained in the slice in the coding structure. So, the image size of 8K is 7680×4320, High Profile v＝2, and the maximum value of K is 1080. If each slice contains 4 precincts, the smallest wavelet transform unit is 7680×4×2 ² , that is, 7680×16; 8K Main Profile's v＝1, and the maximum value of K is 2160. If each slice contains 4 precincts, the smallest wavelet transform unit is 7680×4×2 ¹ , that is, 7680×8. Light Profile does not require vertical wavelet decomposition, so it is ignored.

In this step, a frame of image is first divided into multiple pixel blocks, and the division is mainly based on the frame structure of JPEG-XS and the characteristics of CPU. For example, an image can be divided into 10 wavelet units, each wavelet unit contains 7680×432 pixels.

Step 2, first or second vertical-horizontal wavelet transform: divide the wavelet transform unit in step 1 into 8×8 pixel blocks, perform vertical wavelet transform from right to left and from top to bottom, and use the transform result as input data; then check whether the input data is sufficient for horizontal wavelet transform, if not, return to repeat the vertical wavelet transform, if sufficient, perform horizontal transform; after two transforms, the wavelet coefficients generated by each 8×8 pixel block can be divided into four sub-bands, namely LL, HL, LH, and HH; LL represents the low-frequency components of the image in the horizontal and vertical directions, HH represents the high-frequency components of the image in the horizontal and vertical directions, LH represents the low-frequency components of the image in the horizontal direction and the high-frequency components in the vertical direction, and HL represents the high-frequency components of the image in the horizontal direction and the low-frequency components in the vertical direction; all four sub-band coefficients are classified and saved, and the LL sub-band coefficient is output, ready to continue the wavelet transform.

It should be noted that the wavelet transform unit is divided into 8×8 blocks, and the transformation needs to use the pixels of adjacent blocks.

Step 3, identify and process separately: if it is Main Profile, jump directly to step 4; if it is High Profile, repeat step 2 with the LL wavelet coefficients calculated in step 2, then skip step 4 and go directly to step 5.

Step 4, horizontal wavelet transform of half image width: check whether the LL data is sufficient for a horizontal wavelet transform. If not, return to step 2. If sufficient, perform a horizontal transform. The transform result can be divided into two sub-bands, LL and HL. All LL sub-band data and HL sub-bands are saved in sequence, and the LL sub-band coefficients are output, ready to continue the wavelet transform.

If implementing the example for the high profile description, steps 3 and 4 can be skipped.

Step 5, three-times horizontal wavelet transform: Check whether the input data is sufficient for three-times horizontal wavelet transform. If not, return to step 2, otherwise, perform horizontal transform. The transform result can be divided into two sub-bands, LL and HL; all LL sub-band data and HL sub-band are saved in sequence; LL sub-band data is used as the input for the next transform, and the output still needs to be classified and saved in sequence, repeat three times, and finally obtain the transform data required for entropy coding.

The wavelet transform process adopts a data-driven approach to DWT: during the entire transform process, except for the first vertical transform, the output data of each transform is used as the input data for the next level of DWT transform. The next level of DWT transform will not start the transform until it detects that there is enough input data, and will output it until all the data has been transformed.

Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred arrangement scheme, a person skilled in the art should understand that the technical solution of the present invention (such as the block division method, the wavelet transformation method, the sequence of steps, etc.) can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims (1)

1. A wavelet fast transformation method for JPEG-XS codec, characterized by the steps of:

Step 1, a frame data division wavelet transformation unit: dividing frame data according to coding parameters, dividing the frame data into a plurality of pixel blocks with the size of frame_width multiplied by K multiplied by 2 ^v, and taking the frame_width as a wavelet transformation unit, wherein the frame_width is the image pixel width, v is the number of vertical wavelet decomposition times, K is a positive integer, the maximum value of K is frame_height/2 ^v, the frame_height is the image pixel height, and 2 ^v is the pixel height of precinct;

Step 2, first or second vertical-horizontal wavelet transform: dividing the wavelet transformation unit in the step 1 into 8×8 pixel blocks, performing vertical wavelet transformation from right to left and from top to bottom, and taking the transformation result as input data; then checking whether the input data is enough to perform horizontal wavelet transformation, returning to repeat the vertical wavelet transformation if the input data is insufficient, and performing the horizontal wavelet transformation if the input data is enough; the wavelet coefficient generated by each 8×8 pixel block after twice transformation can be divided into four sub-bands, which are LL, HL, LH, HH respectively; wherein LL represents low-frequency components in the horizontal and vertical directions of the image, HH represents high-frequency components in the horizontal and vertical directions of the image, LH represents low-frequency components in the horizontal and vertical directions of the image, and HL represents high-frequency and low-frequency components in the horizontal and vertical directions of the image; classifying and storing all four sub-band coefficients, outputting LL sub-band coefficients, and preparing to continue wavelet transformation;

Step 3, distinguishing and respectively processing: if the map Profile is the Main Profile, directly jumping to the step 4; if the High Profile is High, repeating the step 2 with the LL wavelet coefficient calculated in the step 2, and then skipping the step 4 to directly carry out the step 5;

Step 4, horizontal wavelet transformation of half image width: checking whether the LL data is enough to perform the horizontal wavelet transformation once, if not, returning to the step 2, otherwise, performing the horizontal transformation; the transform result may be divided into two subbands, LL and HL; all LL sub-band data and HL sub-bands are respectively stored in sequence, LL sub-band coefficients are output, and wavelet transformation is prepared to be continued;

Step 5, three times of horizontal wavelet transformation: checking whether the input data is enough to perform three times of horizontal wavelet transformation, returning to the step 2 if the input data is insufficient, otherwise performing the horizontal transformation; the transform result may be divided into two subbands, LL and HL; all LL sub-band data and HL sub-bands are respectively stored in sequence; the LL sub-band data is used as the input of the next transformation, the output still needs to be classified and stored according to the sequence, and the transformation data needed by the entropy coding is finally obtained after repeating for three times.