CN112631597B

CN112631597B - Shuffling method and computing device

Info

Publication number: CN112631597B
Application number: CN201910955639.8A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Cambricon Technologies Corp Ltd
Current assignee: Cambricon Technologies Corp Ltd
Priority date: 2019-10-09
Filing date: 2019-10-09
Publication date: 2024-08-23
Anticipated expiration: 2039-10-09
Also published as: CN112631597A

Abstract

The application relates to a shuffling method and a computing device. The method comprises the following steps: determining at least one first shuffling group corresponding to a data block to be encoded according to a first shuffling grade, wherein the first shuffling group comprises a plurality of first shuffling units, and the first shuffling units comprise at least one encoding stream; determining the coding length of each first shuffling unit; determining an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit; and sending a first preset number of stream blocks to be shuffled in the output shuffling unit to the input shuffling unit as the first stream blocks to be encoded in the input shuffling unit, so as to encode the first stream blocks to be encoded according to a preset encoding algorithm through the input shuffling unit. The application can reduce the coding rate.

Description

Shuffling method and computing device

Technical Field

The present application relates to the field of computer technologies, and in particular, to a shuffling method and a computing device.

Background

At present, in the encoding and decoding process of data in an SOC (System on a Chip), data needs to be repaired, so that the encoding and decoding rate is high. Therefore, a scheme for reducing the codec rate is needed.

Disclosure of Invention

Accordingly, it is desirable to provide a shuffling method and a computing device for solving the above-mentioned problems.

In a first aspect, there is provided a shuffling method comprising:

acquiring a plurality of coded streams corresponding to a data block to be coded;

Dividing the plurality of encoded streams into a plurality of first shuffle units according to a first shuffle level, and dividing the plurality of first shuffle units into at least one first shuffle group, the first shuffle unit comprising at least one encoded stream, the first shuffle group comprising a plurality of first shuffle units;

For each first shuffling unit in each first shuffling group, determining the total coding length of the blocks to be coded in the coding stream contained in the first shuffling unit as the coding length of the first shuffling unit;

Determining an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit;

and sending a first preset number of stream blocks to be shuffled in the output shuffling unit to the input shuffling unit as the first stream blocks to be encoded in the input shuffling unit, so as to encode the first stream blocks to be encoded according to a preset encoding algorithm through the input shuffling unit.

In a second aspect, there is provided a shuffling method comprising:

acquiring a plurality of decoding streams corresponding to a data block to be decoded;

Dividing the plurality of decoded streams into a plurality of first shuffle units according to a first shuffle level, and dividing the plurality of first shuffle units into at least one first shuffle group, the first shuffle unit containing at least one decoded stream, the first shuffle group containing a plurality of first shuffle units;

for each first shuffling unit in each first shuffling group, determining the total coding length of stream blocks to be decoded in the decoded stream contained in the first shuffling unit as the coding length of the first shuffling unit;

And sending a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as the first to-be-decoded stream blocks in the input shuffling unit, so as to decode the first to-be-decoded stream blocks according to a preset decoding algorithm through the input shuffling unit.

In a third aspect, a computing device is provided, the computing device comprising a processor;

the processor is configured to receive source data and encode the source data using the method according to any one of the first aspect, resulting in encoded data.

As an alternative embodiment, the processor includes a control module, an on-chip memory module, and an arithmetic module;

The control module is used for decoding the machine instruction received by the processor into an encoding micro instruction, and controlling the operation module to operate on the source data in the on-chip storage module by utilizing the encoding micro instruction and writing the source data into the off-chip storage module;

the control module comprises an encoding unit, wherein the encoding unit is used for executing an encoding instruction and decoding the encoding instruction into an encoding micro instruction.

In a fourth aspect, a computing device is provided, the computing device comprising a processor;

the processor is configured to receive encoded data and decode the encoded data using the method according to any of the second aspects to obtain source data.

The control module is used for decoding the machine instruction received by the processor into a decoding micro instruction, controlling the operation module to operate on the encoded data in the off-chip storage module by utilizing the decoding micro instruction, and writing the encoded data into the on-chip storage module;

the control module includes a decode unit to execute a decoded instruction, which is decoded into a decoded micro instruction.

The embodiment of the application provides a shuffling method and a computing device. The codec determines at least one first shuffling group corresponding to the data blocks to be encoded according to the first shuffling level. Wherein the first shuffle group includes a plurality of first shuffle units, the first shuffle units including at least one encoded stream. Then, the codec determines the coding length of each first shuffling unit, and determines an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit. And then, the coder and decoder sends a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit to serve as the first to-be-encoded stream blocks in the input shuffling unit, so that the first to-be-encoded stream blocks are encoded according to a preset encoding algorithm through the input shuffling unit. Thus, by shuffling each coded stream of the data block to be coded, the coding length of each coded stream can be ensured to be similar, so that excessive complement caused by alignment requirement among each coded stream is avoided, and the coding rate is reduced. Meanwhile, the deadlock condition among all the coded streams can be avoided.

Drawings

Fig. 1 is a schematic structural diagram of a codec according to an embodiment of the present application;

Fig. 2 is a flow chart of a method for generating an encoded stream according to an embodiment of the present application;

FIG. 3 is a schematic diagram of source data according to an embodiment of the present application;

FIG. 4A is a schematic diagram of a stream block partition according to an embodiment of the present application;

FIG. 4B is a schematic diagram of a stream block partition according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of a shuffling method according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of a determining output/input shuffling unit according to an embodiment of the present application;

FIG. 7A is a schematic diagram of a coded stream before shuffling according to an embodiment of the present application;

FIG. 7B is a schematic diagram of another encoded stream before shuffling according to an embodiment of the present application;

FIG. 8A is a schematic diagram of a shuffling process of encoded streams according to an embodiment of the present application;

FIG. 8B is a schematic diagram of a shuffling process of another encoded stream provided by embodiments of the present application;

FIG. 9A is a schematic diagram of a shuffling completion of an encoded stream according to an embodiment of the present application;

FIG. 9B is a schematic diagram of a shuffling completion of another encoded stream provided by embodiments of the present application;

FIG. 10 is a diagram of encoded data according to an embodiment of the present application;

FIG. 11 is a schematic diagram of source data sub-block splicing according to an embodiment of the present application;

FIG. 12A is a schematic diagram of a sub-block splicing of encoded data according to an embodiment of the present application;

FIG. 12B is a schematic diagram of a sub-block splicing of encoded data according to an embodiment of the present application;

FIG. 12C is a schematic diagram of a sub-block splicing of encoded data according to an embodiment of the present application;

FIG. 12D is a schematic diagram of a sub-block splicing of encoded data according to an embodiment of the present application;

FIG. 13 is a schematic flow chart of a shuffling method according to an embodiment of the present application;

FIG. 14 is a schematic flow chart of a determining output/input shuffling unit according to an embodiment of the present application;

FIG. 15 is a schematic diagram of a decoded stream before shuffling according to an embodiment of the present application;

FIG. 16 is a schematic diagram of a decode stream shuffling process according to an embodiment of the present application;

FIG. 17 is a schematic diagram of a completion of shuffling of a decoded stream according to an embodiment of the present application;

fig. 18 is a schematic diagram of data of a coding terminal according to an embodiment of the present application;

fig. 19 is a schematic diagram of data division when decoding according to an embodiment of the present application;

fig. 20 is a schematic diagram of data division when decoding according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present disclosure will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the disclosure. Based on the embodiments in this disclosure, all other embodiments that may be made by those skilled in the art without the inventive effort are within the scope of the present disclosure.

It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of this disclosure are used for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising" when used in the specification and claims of the present disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the present disclosure is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the present disclosure and claims refers to any and all possible combinations of one or more of the associated listed items, and encompasses such combinations.

As used in this specification and the claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".

The embodiment of the application provides a shuffling method, which can be applied to a coder-decoder (codec). Fig. 1 is a system architecture diagram of a codec according to an embodiment of the present application. As shown in fig. 1, the codec employs a general-purpose processor architecture including at least one memory and a general-purpose processor. The at least one memory is used to store a computer program. The general processor is used for executing a shuffling method according to a computer program stored in a memory, so as to ensure that the coding lengths of all the coded streams are similar, thereby avoiding excessive complements caused by the alignment requirement among all the coded streams and further reducing the coding rate. The general-purpose processor may be an IPU (INTELLIGENT PROCESSING UNIT, intelligent processor) or a CPU (Central Processing Unit ) or a GPU (Graphics Processing Unit, graphics processor).

For easy understanding, the embodiment of the present application preferably describes in detail a method for generating an encoded stream provided by the embodiment of the present application. As shown in fig. 2, the specific steps are as follows:

in step 201, source data to be encoded is acquired.

In practice, when the codec needs to encode the source data, the codec may acquire the pre-stored source data to be encoded. As shown in fig. 3, the format of the source data is a two-dimensional format, the low dimension is X, the high dimension is Y, the start address is a data address (DATA ADDRESS), the size of the source data is a low dimension data size (data size X), the high dimension data size (data size Y), and the stride (stride) of the source data in the low dimension is a low dimension data stride (DATA STRIDE X). The source data may be further divided into a plurality of source data blocks (blocks) uniformly in two dimensions, as shown in fig. 3, each block represents a source data block, the size of the source data block is a low-dimensional source data block size (block size X) Gao Weiyuan data block size (block size Y), the low-dimensional interval between two adjacent source data blocks is a low-dimensional source data block stride (block stride X), and the high-dimensional interval is a Gao Weiyuan data block stride (block stride Y).

Step 202, determining an encoding region corresponding to the encoding instruction in the source data according to the received encoding instruction, wherein the encoding region comprises at least one data block to be encoded, and the data block to be encoded is a source data block or a source data sub-block.

In implementation, after the codec obtains the source data to be encoded, the encoding region corresponding to the encoding instruction may be determined in the source data according to the received encoding instruction. The coding region comprises at least one data block to be coded, and the data block to be coded is a source data block or a source data sub-block. As shown in fig. 3, a dotted line box is an encoded region that can be obtained by specifying a low-dimensional offset (offset X) in the low-dimensional direction and a high-dimensional offset (offset Y) in the Gao Weifang direction in the source data. Wherein the encoded region contains a low-dimensional number of source data blocks (block num X) in the low-dimensional direction and Gao Weiyuan number of source data blocks (block num Y) in the Gao Weifang direction. The start address (START ADDRESS) of the encoded region is the address of the start source data block.

After the codec obtains the coding region corresponding to the coding command, the codec may directly code the source data block for the central source data block (i.e., the source data block indicated by the vertical square in fig. 3). Whereas for the source data blocks of the edges (i.e., the source data blocks represented by diagonal blocks in fig. 3), the codec may determine source data sub-blocks in the source data blocks by specifying four parameters of up (top), down (bottom), left (left), and right (right), and encode the determined source data sub-blocks.

It should be noted that, except for the encoding region, the data in other regions in the source data need not be partitioned, and the sizes of the data in other regions need not be aligned to the size of the source data block, and the low-dimensional offsets, the high-dimensional offsets, the low-dimensional source data block stride, the Gao Weiyuan data block stride, the low-dimensional source data size, the Gao Weiyuan data size, and the low-dimensional source data stride need not be aligned to the low-dimensional source data block size or the Gao Weiyuan data block size. That is, the codec supports arbitrary blocking in the source data and arbitrary fetching of the coding region in the resulting source data block for coding.

In step 203, for each to-be-encoded data block in at least one to-be-encoded data block, the to-be-encoded data block is divided into a plurality of stream blocks, and the plurality of stream blocks are sent to a plurality of encoded streams corresponding to the to-be-encoded data block, and are used as to-be-shuffled stream blocks in the plurality of encoded streams.

In an implementation, in order to improve the throughput of encoding, a codec may encode the same data block to be encoded in parallel by multiple encoding streams. Similarly, when the codec performs shuffling, for each data block to be encoded, the codec may divide the data block to be encoded into a plurality of stream blocks, and send the plurality of stream blocks to a plurality of encoded streams corresponding to the data block to be encoded, as the data block to be shuffled in the plurality of encoded streams. Alternatively, the codec may divide the data block to be encoded into a plurality of row sequences, each row sequence containing at least one sub-block of data to be encoded. Then, for each sub-block of data to be encoded in each row sequence, the codec may divide the sub-block of data to be encoded into a plurality of stream blocks. In this way, it is ensured that the number of stream blocks in each encoded stream is similar. For example, as shown in fig. 4A, the codec divides the data block to be encoded into 4 row sequences. The first line sequence comprises a data sub-block 1 to be encoded, a data sub-block 2 to be encoded, a data sub-block 3 to be encoded and a data sub-block 4 to be encoded; the second row sequence comprises a data sub-block 5 to be encoded, a data sub-block 6 to be encoded, a data sub-block 7 to be encoded and a data sub-block 8 to be encoded; the third row sequence comprises a data sub-block 9 to be encoded, a data sub-block 10 to be encoded, a data sub-block 11 to be encoded and a data sub-block 12 to be encoded; the fourth row sequence comprises a data sub-block 13 to be encoded, a data sub-block 14 to be encoded, a data sub-block 15 to be encoded and a data sub-block 16 to be encoded, and the diagonal line blocks in each row sequence represent non-encoded data in the data block to be encoded. Then, as shown in fig. 4B, for each data sub-block to be encoded in each row sequence, the codec may divide the data sub-block to be encoded into 4 stream blocks, and send the 4 stream blocks to the encoded streams (stream) 1 to 4, respectively, as the stream blocks to be shuffled in each encoded stream. Wherein encoded stream 1 and encoded stream 2 comprise 16 to-be-shuffled stream blocks and encoded stream 3 and encoded stream 4 comprise 12 to-be-shuffled stream blocks.

A specific embodiment will be described below in detail about a shuffling method according to an embodiment of the present application, which is applied to an encoding process. As shown in fig. 5, the specific steps are as follows:

Step 501, a plurality of encoded streams corresponding to the data blocks to be encoded are obtained.

In an implementation, when the codec needs to shuffle a plurality of encoded streams corresponding to the data block to be encoded, the codec may obtain the plurality of encoded streams corresponding to the data block to be encoded.

Step 502, dividing the plurality of encoded streams into a plurality of first shuffle units according to the first shuffle level, and dividing the plurality of first shuffle units into at least one first shuffle group. Wherein the first shuffle unit comprises at least one encoded stream and the first shuffle group comprises a plurality of first shuffle units.

In implementation, the codec may select a corresponding shuffle level from among preset shuffle levels according to a difference in the number of stream blocks included in each encoded stream. For example, three shuffle levels, shuffle level 1, shuffle level 2, and shuffle level 3, may be preset in the codec. After the codec selects a certain shuffling level (i.e. the first shuffling level), at least one first shuffling group corresponding to the data blocks to be encoded may be determined further according to the first shuffling level. Wherein the first shuffle group includes a plurality of first shuffle units, the first shuffle units including at least one encoded stream.

After the codec obtains a plurality of encoded streams corresponding to the data block to be encoded, the codec may divide the plurality of encoded streams into a plurality of first shuffling units according to the first shuffling level, and divide the plurality of first shuffling units into at least one first shuffling group. Wherein the first shuffle unit comprises at least one encoded stream and the first shuffle group comprises a plurality of first shuffle units. For example, the data block to be encoded corresponds to 32 encoded streams, and when the first shuffling level is a shuffling level 1, the codec may divide the 32 encoded streams into 32 first shuffling units, and divide the 32 first shuffling units into 8 first shuffling groups, i.e., shuffle every adjacent 4 encoded streams; when the first shuffling level is shuffling level 2, the codec may divide 32 encoded streams into 8 first shuffling units and divide the 8 first shuffling units into 2 first shuffling groups, i.e. shuffle 4 encoded streams as one shuffling unit out of every adjacent 16 encoded streams; when the first shuffling level is shuffling level 3, the codec may divide 32 encoded streams into 2 first shuffling units and divide the 2 first shuffling units into 1 first shuffling group, i.e. 16 encoded streams among the 32 encoded streams are shuffled as one shuffling unit.

In step 503, for each first shuffling unit in each first shuffling group, the codec may determine the total coding length of the blocks of the stream to be coded in the coded stream contained in the first shuffling unit as the coding length of the first shuffling unit.

In an implementation, after determining a first shuffling group corresponding to a data block to be encoded, the codec may further determine, for each first shuffling unit in each first shuffling group, a total coding length of a block to be encoded in an encoded stream included in the first shuffling unit as a coding length of the first shuffling unit. Optionally, for the case that the shuffling process and the encoding process are performed synchronously, the codec may further determine, as the encoding length of the first shuffling unit, a total encoding length of a to-be-encoded stream block and an encoded stream block in the encoded stream included in the first shuffling unit.

In step 504, an output shuffle unit and an input shuffle unit are determined in each first shuffle unit based on the encoding length of each first shuffle unit.

In an implementation, after determining the coding length of each first shuffling unit, the codec may determine an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit. Wherein, as shown in fig. 6, the codec determines the processing procedure of the output shuffling unit and the input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit as follows:

Step 601, determining a shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit.

In an implementation, after determining the coding length of each first shuffling unit, the codec may determine the shuffling type of each first shuffling unit according to the coding length of each first shuffling unit. The manner of determining the shuffling type of each first shuffling unit by the codec according to the encoding length of each first shuffling unit may be various, and the embodiments of the present application provide two possible manners, specifically as follows:

in one aspect, for each first shuffling unit, the codec may determine a shuffling type of the first shuffling unit based on a first difference between the encoding length of the first shuffling unit and the encoding lengths of other first shuffling units. The specific treatment process is as follows:

step one, for each first shuffling unit of the first shuffling units, a first difference between the coding length of that first shuffling unit and the coding lengths of the other first shuffling units is determined.

In an implementation, after determining the coding length of each first shuffling unit, the codec may further determine, for each first shuffling unit, a first difference between the coding length of that first shuffling unit and the coding lengths of the other first shuffling units. In example one, as shown in fig. 7A, 4 shuffling units are used as a shuffling group to shuffle, each shuffling unit includes an encoded stream as an example, the encoded length of encoded stream 1 is n+5, the encoded length of encoded stream 2 is n+9, the encoded length of encoded stream 3 is N, the encoded length of encoded stream 4 is n+3, N is a positive integer, the first difference value corresponding to encoded stream 1 is { -4,5,2}, the first difference value corresponding to encoded stream 2 is {4,9,6}, the first difference value corresponding to encoded stream 3 is { -5, -9, -3}, and the first difference value corresponding to encoded stream 4 is { -2, -6,3}.

In example two, as shown in fig. 7B, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes an encoded stream as an example, the encoded length of encoded stream 1 is n+15, the encoded length of encoded stream 2 is n+10, the encoded length of encoded stream 3 is n+8, the encoded lengths of encoded stream 4 are N, N are positive integers, the first difference value corresponding to encoded stream 1 is {5,7, 15}, the first difference value corresponding to encoded stream 2 is { -5,2, 10}, the first difference value corresponding to encoded stream 3 is { -7, -2,8}, and the first difference value corresponding to encoded stream 4 is { -15, -10, -8}.

And step two, determining the shuffling type of the first shuffling unit according to a first difference value corresponding to the first shuffling unit.

In an implementation, the codec may have a first preset difference threshold stored in advance. The first preset difference threshold may be set empirically by the skilled person. After the codec obtains the first difference value corresponding to the first shuffling unit, it may be determined whether a target difference value with an absolute value greater than or equal to a first preset difference threshold exists in the first difference value corresponding to the first shuffling unit, and whether the target difference values are all positive numbers. And if the first difference value corresponding to the first shuffling unit has a target difference value with an absolute value larger than or equal to a first preset difference value threshold value and the target difference values are positive numbers, the codec executes the step three. And if the first difference value corresponding to the first shuffling unit has a target difference value with an absolute value larger than or equal to a first preset difference value threshold value and has a target difference value with a negative number, the codec executes the step four. If the first difference value corresponding to the first shuffling unit does not have a target difference value with an absolute value larger than or equal to a first preset difference threshold value. The codec performs step five.

And step three, determining that the shuffling type of the first shuffling unit is a high coding type.

In implementation, if there is a target difference value whose absolute value is greater than or equal to a first preset difference threshold value in the first difference value corresponding to the first shuffling unit, and the target difference values are both positive numbers, the codec may determine that the shuffling type of the first shuffling unit is a high encoding type (hereinafter denoted by H). For example, in the above example one, as shown in fig. 7A, the first preset difference threshold is 7, the shuffle type of the encoded stream 2 is H, and in the above example two, as shown in fig. 7B, the first preset difference threshold is 7, the shuffle type of the encoded stream 1 and the encoded stream 2 is H.

And step four, determining that the shuffling type of the first shuffling unit is a low coding type.

In implementation, if there is a target difference value with an absolute value greater than or equal to a first preset difference threshold value in the first difference value corresponding to the first shuffling unit and there is a target difference value with a negative number in the target difference values, the codec may determine that the shuffling type of the first shuffling unit is a low encoding type (hereinafter denoted by L). For example, in the above example one, as shown in fig. 7A, the first preset difference threshold is 7, the shuffle type of the encoded stream 3 is L, and in the above example two, as shown in fig. 7B, the first preset difference threshold is 7, the shuffle type of the encoded stream 3 and the encoded stream 4 is L.

And fifthly, determining that the shuffling type of the first shuffling unit is a medium encoding type.

In implementation, if there is no target difference value having an absolute value greater than or equal to a first preset difference threshold value among the first difference values corresponding to the first shuffle unit, the codec may determine that the shuffle type of the first shuffle unit is an intermediate code type (hereinafter denoted by M). For example, in the above example one, as shown in fig. 7A, if the first preset difference threshold is 7, the shuffle type of the encoded streams 1 and 4 is M.

In a second manner, for each first shuffling unit, the codec may determine a shuffling type of the first shuffling unit based on a second difference between the encoding length of the first shuffling unit and the average encoding length. The specific treatment process is as follows:

step one, determining an average coding length according to the coding length of each first shuffling unit.

In an implementation, after determining the coding length of each first shuffling unit, the codec may determine an average coding length from the coding lengths of each first shuffling unit. In example three, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes an encoded stream as an example, the encoded length of encoded stream 1 is n+5, the encoded length of encoded stream 2 is n+9, the encoded length of encoded stream 3 is N, the encoded length of encoded stream 4 is n+3, N is a positive integer, and the average encoded length is n+4.25.

In example four, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes an encoded stream as an example, the encoded length of encoded stream 1 is n+15, the encoded length of encoded stream 2 is n+10, the encoded length of encoded stream 3 is n+8, the encoded length of encoded stream 4 is N, N is a positive integer, and the average encoded length is n+8.25.

Step two, for each first shuffling unit in the first shuffling units, determining a second difference value between the coding length of the first shuffling unit and the average coding length.

In an implementation, after determining the average coding length, the codec may further determine, for each first shuffle unit, a second difference between the coding length of the first shuffle unit and the average coding length. For example, in the above example three, the second difference corresponding to the encoded stream 1 is 0.75, the second difference corresponding to the encoded stream 2 is 4.75, the second difference corresponding to the encoded stream 3 is-4.25, and the second difference corresponding to the encoded stream 4 is-1.25; in the fourth example, the second difference value corresponding to the encoded stream 1 is 6.75, the second difference value corresponding to the encoded stream 2 is 1.75, the second difference value corresponding to the encoded stream 3 is-0.25, and the second difference value corresponding to the encoded stream 4 is-8.25.

And thirdly, determining the shuffling type of the first shuffling unit according to the second difference value corresponding to the first shuffling unit.

In an implementation, the codec may have a second preset difference threshold stored in advance. The second preset difference threshold may be set empirically by the skilled person. After the codec obtains the second difference value corresponding to the first shuffle unit, it may be determined whether the absolute value of the second difference value corresponding to the first shuffle unit is greater than or equal to a second preset difference threshold, and whether the second difference value is a positive number. And if the absolute value of the second difference value corresponding to the first shuffling unit is greater than or equal to a second preset difference threshold value and the second difference value is a positive number, the codec executes the fourth step. And if the absolute value of the second difference value corresponding to the first shuffling unit is greater than or equal to a second preset difference threshold value and the second difference value is a negative number, executing a step five by the codec. And if the absolute value of the second difference value corresponding to the first shuffling unit is smaller than a second preset difference threshold value, the codec executes step six.

And step four, determining that the shuffling type of the first shuffling unit is a high coding type.

In implementation, if the absolute value of the second difference value corresponding to the first shuffle unit is greater than or equal to the second preset difference threshold, and the second difference value is a positive number, the codec may determine that the shuffle type of the first shuffle unit is a high coding type (i.e., H). For example, in the above example three, if the second preset difference threshold is 4, the shuffle type of the encoded stream 2 is H; in the fourth example, if the second preset difference threshold is 6, the shuffle type of the encoded stream 1 is H.

And step five, determining that the shuffling type of the first shuffling unit is a low coding type.

In implementation, if the absolute value of the second difference value corresponding to the first shuffle unit is greater than or equal to a second preset difference threshold, and the second difference value is negative, the codec may determine that the shuffle type of the first shuffle unit is a low coding type (i.e., L). For example, in the above example three, if the second preset difference threshold is 4, the shuffle type of the encoded stream 3 is L; in the fourth example, if the second preset difference threshold is 6, the shuffle type of the encoded stream 4 is L.

Step six, determining that the shuffling type of the first shuffling unit is a mid-encoding type.

In an implementation, if the absolute value of the second difference value corresponding to the first shuffle unit is smaller than a second preset difference threshold, the codec may determine that the shuffle type of the first shuffle unit is a mid-code type (i.e., M). For example, in the above example three, the second preset difference threshold is 4, and the shuffle type of the encoded streams 1 and 4 is M; in the fourth example, if the second preset difference threshold is 6, the shuffle type of the encoded streams 2 and 3 is M.

In step 602, according to the shuffling type and preset shuffling rules of each first shuffling unit, an output shuffling unit and an input shuffling unit corresponding to each output shuffling unit are determined in each first shuffling unit.

In implementations, the shuffling rules may be pre-stored in the codec. The shuffling rules may be set empirically by the skilled person. After the codec obtains the shuffling type of each first shuffling unit, the output shuffling unit and the input shuffling unit corresponding to each output shuffling unit may be further determined in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule. The codec determines the output shuffling units and the processing procedure of the input shuffling units corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule, wherein the processing procedure is as follows:

Step one, if the first number of the first target shuffling units of the high coding type is smaller than or equal to the second number of the second target shuffling units of the low coding type, the first target shuffling units are determined as output shuffling units, and for each output shuffling unit, at least one second target shuffling unit is determined as an input shuffling unit corresponding to the output shuffling unit in the second target shuffling units.

In an implementation, the codec may determine the first target shuffle unit as an output shuffle unit if the first number of first target shuffle units of the high coding type is less than or equal to the second number of second target shuffle units of the low coding type. Then, for each output shuffling unit, the codec may determine at least one second target shuffling unit among the second target shuffling units as an input shuffling unit corresponding to the output shuffling unit. The second target shuffling unit may be only one input shuffling unit corresponding to one output shuffling unit, or may be input shuffling units corresponding to a plurality of output shuffling units, which is not limited in the embodiment of the present application. For example, taking 4 shuffle units as a shuffle group, each shuffle unit containing one encoded stream as an example, when the shuffle type of encoded stream 1 to encoded stream 4 is HMML, the codec may determine encoded stream 1 as an output shuffle unit and encoded stream 4 as an input shuffle unit of encoded stream 1; when the shuffle type of the encoded streams 1 to 4 is HMLL, the codec may determine the encoded stream 1 as an output shuffle unit and the encoded streams 3 and 4 as input shuffle units of the encoded stream 1; when the shuffle type of the encoded streams 1 to 4 is HLLL, the codec may determine the encoded stream 1 as an output shuffle unit and the encoded stream 2, the encoded stream 3, and the encoded stream 4 as input shuffle units of the encoded stream 1; when the shuffle types of the encoded streams 1 to 4 are HHLL, the codec may determine the encoded streams 1 and 2 as output shuffle units and the encoded stream 3 as input shuffle units of the encoded stream 1, the encoded stream 4 as input shuffle units of the encoded stream 2, the codec may also determine the encoded streams 3 and 4 as input shuffle units of the encoded stream 1, the sum encoded stream 3 and 4 as input shuffle units of the encoded stream 2, the codec may also determine the encoded stream 3 as input shuffle units of the encoded stream 1, and the sum encoded streams 3 and 4 as input shuffle units of the encoded stream 2.

And step two, if the first number of the first target shuffling units of the high coding type is larger than the second number of the second target shuffling units of the low coding type, determining the second number of the first target shuffling units as output shuffling units, and determining at least one second target shuffling unit as an input shuffling unit corresponding to the output shuffling unit in the second target shuffling units aiming at each output shuffling unit.

In an implementation, if the first number of first target shuffle units of the high coding type is greater than the second number of second target shuffle units of the low coding type, the codec may determine the second number of first target shuffle units as output shuffle units among the first target shuffle units. Then, for each output shuffling unit, the codec may determine at least one second target shuffling unit among the second target shuffling units as an input shuffling unit corresponding to the output shuffling unit. The second target shuffling unit may be only one input shuffling unit corresponding to one output shuffling unit, or may be input shuffling units corresponding to a plurality of output shuffling units, which is not limited in the embodiment of the present application. For example, taking 4 shuffle units as a shuffle group, each shuffle unit containing one encoded stream as an example, when the shuffle type of encoded streams 1 to 4 is HHML, the codec may determine encoded stream 2 as an output shuffle unit and encoded stream 4 as an input shuffle unit of encoded stream 2; when the shuffle type of the encoded streams 1 to 4 is HHHL, the codec may determine the encoded stream 3 as an output shuffle unit and the encoded stream 4 as an input shuffle unit of the encoded stream 1.

It should be noted that if the first number of first target shuffle units of the high coding type is greater than the second number of second target shuffle units of the low coding type, and there is a third target shuffle unit of the medium coding type among the respective shuffle units, the codec may further determine a number of first target shuffle units as output shuffle units (hereinafter referred to as second output shuffle units) among first target shuffle units other than the output shuffle unit (hereinafter referred to as first output shuffle units) determined as corresponding to the second target shuffle units, and for each of the second output shuffle units, the codec may determine at least one third target shuffle unit among the third target shuffle units as an input shuffle unit corresponding to the second output shuffle unit. For example, taking 4 shuffle units as a shuffle group, each shuffle unit containing one encoded stream as an example, when the shuffle type of encoded stream 1 to encoded stream 4 is HHML, the codec may determine encoded stream 2 as a first output shuffle unit and encoded stream 4 as an input shuffle unit of encoded stream 2; meanwhile, the codec may also determine the encoded stream 1 as a second output shuffling unit and the encoded stream 3 as an input shuffling unit of the encoded stream 1.

In addition, for the third target shuffle units of the mid-encoding type, for each third target shuffle unit, the codec may determine the third target shuffle unit as an output shuffle unit and determine itself as an input shuffle unit to which the third target shuffle unit corresponds. For example, taking 4 shuffle units as a shuffle group, each shuffle unit including one encoded stream as an example, when the shuffle type of encoded streams 1 to 4 is MMMM, the codec may determine encoded stream 1 as an output shuffle unit and encoded stream 1 as an input shuffle unit of encoded stream 1, encoded streams 2 to 4 are similar, and will not be described again here.

Step 505, a first preset number of to-be-shuffled stream blocks in the output shuffling unit are sent to the input shuffling unit to be used as the first to-be-encoded stream blocks in the input shuffling unit, so that the first to-be-encoded stream blocks are encoded according to a preset encoding algorithm through the input shuffling unit.

In an implementation, the first preset number may be pre-stored in the codec. The first preset number may be set empirically by the skilled person. After determining the output shuffling unit and the input shuffling unit, the codec may send a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as a first to-be-encoded stream block in the input shuffling unit, so as to encode the first to-be-encoded stream block according to a preset encoding algorithm through the input shuffling unit. Wherein the first preset number may be greater than or equal to the number of input shuffle units; the flow blocks to be shuffled received by each input shuffling unit can be the same or different; for each input shuffling unit, the codec may further input a third preset number of to-be-shuffled units in the input shuffling unit to a to-be-encoded queue of the input shuffling unit as a first to-be-encoded stream block of the input shuffling unit.

Optionally, for each output shuffling unit, a second preset number of to-be-shuffled stream blocks in the output shuffling unit are sent to an input shuffling unit corresponding to the output shuffling unit, and the second preset number of to-be-shuffled stream blocks are used as first to-be-encoded stream blocks in the input shuffling unit.

In an implementation, the second preset number may be pre-stored in the codec. The second preset number may be set empirically by the skilled person. For the case that the output shuffling unit corresponds to the input shuffling unit, for each output shuffling unit, the codec may send a second preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit corresponding to the output shuffling unit as the first to-be-encoded stream blocks in the input shuffling unit. The second preset number may be greater than or equal to the number of input shuffling units corresponding to the output shuffling unit, and the number of to-be-shuffled stream blocks received by each input shuffling unit may be the same or different. In this way, it is ensured that the coding lengths of the individual shuffle units are similar. For example, as shown in fig. 8A, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes one encoded stream as an example, the shuffling types of encoded stream 1 to encoded stream 4 are MHLM, encoded stream 2 is an output shuffling unit, encoded stream 3 is an input shuffling unit of encoded stream 2, encoded stream 1 may output 1 to-be-shuffled stream block to encoded stream 1, encoded stream 2 may output 1 to-be-shuffled stream block to encoded stream 3, and encoded stream 4 may output 1 to-be-shuffled stream block to encoded stream 4. Thus, as shown in fig. 9A, after the shuffling is completed, the encoded length of encoded stream 1 is n+8, the encoded length of encoded stream 2 is n+9, the encoded length of encoded stream 3 is n+3, the encoded length of encoded stream 4 is n+6, and the shuffling types of encoded streams 1 to 4 are MMMM. As another example, as shown in fig. 8B, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes one encoded stream as an example, the shuffling types of encoded stream 1 to encoded stream 4 are HHLL, then encoded stream 1 and encoded stream 2 are output shuffling units, encoded stream 3 and encoded stream 4 are input shuffling units of encoded stream 1, encoded stream 4 is an input shuffling unit of encoded stream 2, then encoded stream 1 can output 1 to-be-shuffled stream block to encoded stream 3 and encoded stream 4, respectively, and encoded stream 2 can output 2 to-be-shuffled stream blocks to encoded stream 4. Thus, as shown in fig. 9B, after the shuffling is completed, the encoded length of encoded stream 1 is n+15, the encoded length of encoded stream 2 is n+10, the encoded length of encoded stream 3 is n+11, the encoded length of encoded stream 4 is n+9, and the shuffling types of encoded streams 1 to 4 are MMMM.

As an alternative embodiment, the codec may determine at least one second shuffle group corresponding to the data block to be encoded according to the second shuffle level. Wherein the second shuffle group includes a plurality of second shuffle units, the second shuffle units including at least one encoded stream. Then, the codec may re-shuffle the to-be-encoded stream block in each second shuffling unit as the to-be-shuffled stream block until the encoding length of each second shuffling unit satisfies a preset proximity condition.

In an implementation, after the codec shuffles the encoded stream corresponding to the data block to be encoded according to the first shuffling level, the codec may further determine at least one second shuffling group corresponding to the data block to be encoded according to the second shuffling level. Wherein the second shuffle group includes a plurality of second shuffle units, the second shuffle units including at least one encoded stream. The processing procedure of the codec to determine at least one second shuffle group corresponding to the data block to be encoded according to the second shuffle level is similar to that of step 502, and will not be repeated here. Then, the codec may re-shuffle the to-be-encoded stream block in each second shuffling unit as the to-be-shuffled stream block until the encoding length of each second shuffling unit satisfies a preset proximity condition.

It should be noted that, after the codec shuffles the encoded stream corresponding to the data block to be encoded according to the second shuffling level, the codec may further shuffle the encoded stream corresponding to the data block to be encoded according to the third shuffling level. For example, the codec may shuffle the encoded stream corresponding to the data block to be encoded according to the shuffle level 1-shuffle level 2-shuffle level 3, shuffle the encoded stream corresponding to the data block to be encoded according to the shuffle level 1-shuffle level 3-shuffle level 2, and shuffle the encoded stream corresponding to the data block to be encoded according to the shuffle level 2-shuffle level 3-shuffle level 1.

As an alternative embodiment, when the codec detects that the encoding of the first encoded stream in the plurality of encoded streams is completed, a third preset number of stream blocks to be encoded in a second encoded stream that is not encoded in the plurality of encoded streams is sent to the first encoded stream as stream blocks to be encoded in the first encoded stream.

In practice, the coding rate at the time of small data block coding is increased in order to further balance the coding length of each coded stream. The codec may have a third preset number stored therein in advance. The third preset number may be set empirically by the skilled person. When the codec detects that encoding of a first encoded stream of the plurality of encoded streams is completed (i.e., the first encoded stream outputs an encoded stream end (STREAMFINISH) flag), a third preset number of to-be-encoded stream blocks in a second encoded stream of the plurality of encoded streams that is not completed may be transmitted to the first encoded stream as to-be-encoded stream blocks in the first encoded stream. The second encoded stream may be all of the encoded streams that have not been encoded, or may be some of the encoded streams that have not been encoded.

As an alternative embodiment, the codec encodes the first to-be-encoded stream block according to the preset encoding algorithm through the input shuffling unit as follows:

and firstly, coding preset characters contained in the first stream block to be coded according to a first coding rule corresponding to the preset characters to obtain first coded data.

In implementation, the codec may store a first encoding rule corresponding to the preset character in advance. The first encoding rule corresponding to the preset character may be set empirically by a technician. The codec may encode the preset character included in the first stream block to be encoded according to a first encoding rule corresponding to the preset character, to obtain first encoded data. Wherein, because the neural network data has sparse characteristics, a technician can set a plurality of all-zero characters in succession as preset characters. Table one shows the preset characters, as shown in Table one:

List one

And secondly, encoding the first encoded data according to a preset second encoding rule.

In an implementation, the second encoding rule may be pre-stored in the codec. The second encoding rule may be a prefix code encoding algorithm, or may be other encoding algorithms, which is not limited in the embodiment of the present application. After the codec obtains the first encoded data, one step of encoding the first encoded data according to a preset second encoding rule may be performed.

As an alternative implementation manner, the codec encodes the data blocks to be encoded, and after obtaining the encoded data blocks, the codec may generate the encoding header blocks of the encoded data blocks corresponding to each data block to be encoded. The coding header block comprises a storage address of a coding data block corresponding to the coding header block. The codec may then store each encoded header block and store each encoded data block after each encoded header block.

In implementation, for the encoded data block corresponding to the data block to be encoded, as shown in fig. 10, the codec uses a header-data block (header-data block) map for storage. The mapping form is divided into two major parts, namely a coding header (header) part and a coding data block (data block) part. The starting address of the coding head part is a coding head address (HEAD ADDRESS); the space occupied by the coding header portion is the size of the total number of coding header blocks (head total number), which may be greater than or equal to the number of valid coding header blocks. The size of each coding head block is 16 bits (byte), the information such as the initial address and the actual size of the coding data block corresponding to the coding head block is stored in each coding head block, and each coding head block is continuously stored in the storage device. The coded data block portion is stored next to the end address of the coding head portion. Each coded data block part comprises a plurality of coded data blocks, and each coded data block corresponds to an h-coded header block one by one. The storage format of the encoded data block is divided into two formats, namely a packing (packing) format and a non-packing (unpacking) format.

Optionally, for the packaging format, the encoded data block is composed of encoded data, and the size of the encoded data is equal to the size of the encoded data in the data block to be encoded.

In implementation, as shown in fig. 10, in the packing format, the encoded data blocks are closely stored, each encoded data block is composed of encoded data, and the size of the encoded data is equal to the size of the encoded data to be encoded in the data block to be encoded. I.e. the space occupied by the encoded data block is the actual size of the encoded data block to be encoded.

Optionally, for non-packed formats, the encoded data block consists of encoded data and padding data, the sum of the size of the encoded data and the size of the padding data being equal to the size of the data block to be encoded.

In practice, as shown in fig. 10, in the non-packed format, each encoded data block consists of encoded data and padding data, the sum of the size of the encoded data and the size of the padding data being equal to the size of the data block to be encoded. I.e. each coded data block occupies the same space and is equal to the size of the previous complete data block to be coded. Wherein the former part is encoded data and the latter part is reserved padding data.

As an alternative embodiment, the encoded data block is composed of encoded data sub-blocks corresponding to a plurality of source data sub-blocks belonging to the same source data block. In the packing format, the encoded data sub-block is composed of encoded data, and the size of the encoded data is equal to the size of the source data sub-block after the encoding of the data to be encoded. In the non-packed format, the encoded data sub-block is composed of encoded data and filler data, and the sum of the size of the encoded data and the size of the filler data is equal to the size of the source data sub-block.

In practice, when determining multiple coding regions in source data according to different coding instructions, a codec may split a complete source data block into multiple source data sub-blocks. The coding areas corresponding to the coding instructions are arranged on the adjacent side, or the coding areas are all used for coding the complete source data blocks, and the source data blocks are not overlapped with each other; or each encodes a portion of the same source data block (i.e., source data sub-blocks) and the boundaries of the source data sub-blocks within the source data block overlap exactly. As shown in fig. 11, the four coding regions defined in the source data by the four coding instructions divide a part of the source data block (represented by gray oblique line squares) into a plurality of source data sub-blocks, and the boundaries of the source data sub-blocks in the same source data block exactly overlap. The coder and decoder codes each source data sub-block in the same source data block according to different coding instructions to obtain coded data sub-blocks. When the codec stores the encoded data sub-blocks corresponding to the plurality of source data sub-blocks in the same source data block, the encoded data sub-blocks may be stored in the same encoded data block. Therefore, the splicing of the specific coding region can be achieved by the cooperation of a plurality of coding instructions, the effect is identical to that of coding by one coding instruction, and the execution sequence of the coding instructions can be executed arbitrarily or even simultaneously, so that coding tasks can be divided under the conditions of multi-core source data division and single-core internal division for running water execution.

As shown in fig. 12A, one source data block is divided into two parts in both low and high dimensions, so that the source data block is divided into 4 source data sub-blocks, TL (top-left), TR (top-right), BL (bottom-left) and BR (bottom-right), respectively. The size of each source data sub-block is shown in fig. 12A, and the codec may be specified by four parameters, i.e., block left, block right, block top, block bottom, inside the source data block. Each source data sub-block may be encoded data or non-encoded data. In the encoded data block in the non-packed format, the 4 source data sub-blocks corresponding to the 4 encoded data sub-blocks may be stored in a fixed order of TL-TR-BL-BR, and each encoded data sub-block is composed of encoded data and filler data, and a sum of a size of the encoded data and a size of the filler data is equal to a size of the source data sub-block. That is, the space occupied by each encoded data sub-block is the size of the source data sub-block corresponding to the encoded data sub-block before encoding. Therefore, the space of the coding data block is the same as the size of the source data block, so that if each source data sub-block of the source data block is coded by different coding instructions, each coding instruction can calculate the address of the responsible coding data sub-block according to the dividing information (which can be obtained by calculating the instruction domain), and the splicing of the results of a plurality of coding instructions is realized.

As shown in fig. 12A, the codec may convert the encoded data block in the non-packed format into the encoded data block in the packed format by means of a motion (Move) or the like. In the encoded data blocks in the packed format, each encoded data sub-block is composed of encoded data having a size equal to the size of the source data sub-block encoded with the data to be encoded. The starting address of each coded data sub-block needs to be obtained by accumulating the actual sizes of the previous coded data sub-blocks, and the space occupied by the whole coded data block is the sum of the sizes of all the coded data sub-blocks.

As shown in fig. 12B, one source data block is divided into two parts in a high dimension such that the source data block is divided into 2 source data sub-blocks TL and BL, or BL and BR, respectively. As shown in fig. 12C, one source data block is divided into two parts in a low dimension such that the source data block is divided into 2 source data sub-blocks TL and TR, or TR and BR, respectively.

One source data block is not divided in both low and high dimensions, there is only one source data sub-block, which may be TL, TR, BL or BR. As shown in fig. 12D, one source data block has only TL one source data sub-block.

In order to support splicing of different coding regions, the format of the coding head corresponding to the coding data block comprising a plurality of coding data sub-blocks is shown in table two.

Watch II

As an alternative embodiment, if the total coding length of the stream blocks to be coded in each first shuffle group is greater than the original length of the data blocks to be coded, the coding process of the data blocks to be coded is terminated.

In practice, in order to ensure that the encoded data blocks can be stored in a non-packed format, it is necessary to ensure that the size of the encoded data blocks cannot exceed the size of the data blocks to be encoded. When the codec performs shuffling processing on the data block to be encoded, if the total encoding length of the stream block to be encoded in each first shuffling group is detected to be greater than the original length of the data block to be encoded, the encoding processing of the data block to be encoded is terminated. And then the codec can resend the request to read the data block to be encoded, and the read data block to be encoded is directly stored without encoding processing. Optionally, the codec may further terminate the encoding process of the data block to be encoded if it is detected that the encoding length of the encoding process output is greater than the size of the data block to be encoded when encoding the data block to be encoded.

The embodiment of the application provides a shuffling method. The codec determines at least one first shuffling group corresponding to the data blocks to be encoded according to the first shuffling level. Wherein the first shuffle group includes a plurality of first shuffle units, the first shuffle units including at least one encoded stream. Then, the codec determines the coding length of each first shuffling unit, and determines an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit. And then, the coder and decoder sends a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit to serve as the first to-be-encoded stream blocks in the input shuffling unit, so that the first to-be-encoded stream blocks are encoded according to a preset encoding algorithm through the input shuffling unit. Thus, by shuffling each coded stream of the data block to be coded, the coding length of each coded stream can be ensured to be similar, so that excessive complement caused by alignment requirement among each coded stream is avoided, and the coding rate is reduced. Meanwhile, the deadlock condition among all the coded streams can be avoided.

The embodiment of the application also provides a shuffling method, which is applied to the decoding process. As shown in fig. 13, the specific steps are as follows:

Step 1301, a plurality of decoding streams corresponding to the data block to be decoded are obtained.

The processing procedure of step 1301 is similar to that of step 501, and will not be described again here.

In step 1302, the plurality of decoded streams are divided into a plurality of first shuffle units according to a first shuffle level, and the plurality of first shuffle units are divided into at least one first shuffle group. Wherein the first shuffle unit comprises at least one decoded stream and the first shuffle group comprises a plurality of first shuffle units.

The processing of step 1302 is similar to the processing of step 502 and will not be described in detail herein.

In step 1303, for each first shuffling unit in each first shuffling group, the total coding length of the stream blocks to be decoded in the decoded stream contained in the first shuffling unit is determined as the coding length of the first shuffling unit.

Alternatively, for the case where the shuffling process and the decoding process are performed simultaneously, the codec may further determine, as the encoding length of the first shuffling unit, the total encoding length of the stream blocks to be decoded and the decoded stream blocks in the decoded stream contained in the first shuffling unit.

The processing procedure of step 1303 is similar to the processing procedure of step 503, and will not be described in detail here.

In step 1304, an output shuffle unit and an input shuffle unit are determined in each first shuffle unit based on the encoding length of each first shuffle unit.

The processing of step 1304 is similar to the processing of step 504 and will not be described in detail herein.

Alternatively, as shown in fig. 14, the codec determines the processing procedure of the output shuffling unit and the input shuffling unit in each first shuffling unit according to the encoding length of each first shuffling unit as follows:

Step 1401, determining a shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit. The manner of determining the shuffling type of each first shuffling unit by the codec according to the encoding length of each first shuffling unit may be various, and the embodiments of the present application provide two possible manners, specifically as follows:

In example five, as shown in fig. 15, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes a decoding stream as an example, the encoding length of decoding stream 1 is n+5, the encoding length of decoding stream 2 is n+9, the encoding length of decoding stream 3 is N, the encoding length of decoding stream 4 is n+3, N is a positive integer, the first difference value corresponding to decoding stream 1 is { -4,5,2}, the first difference value corresponding to decoding stream 2 is {4,9,6}, the first difference value corresponding to decoding stream 3 is { -5, -9, -3}, and the first difference value corresponding to decoding stream 4 is { -2, -6,3}.

And step three, if a target difference value with an absolute value larger than or equal to a first preset difference value threshold exists in the first difference value corresponding to the first shuffling unit, and the target difference values are positive numbers, determining that the shuffling type of the first shuffling unit is a high coding type.

For example, in the fifth example, as shown in fig. 15, if the first preset difference threshold is 7, the shuffle type of the decoded stream 2 is H.

And step four, if a target difference value with an absolute value larger than or equal to a first preset difference value threshold exists in the first difference value corresponding to the first shuffling unit, and a target difference value with a negative number exists in the target difference value, determining that the shuffling type of the first shuffling unit is a low coding type.

For example, in the fifth example described above, as shown in fig. 15, if the first preset difference threshold is 7, the shuffle type of the decoded stream 3 is L.

And fifthly, if the target difference value with the absolute value larger than or equal to the first preset difference value threshold value does not exist in the first difference value corresponding to the first shuffling unit, determining that the shuffling type of the first shuffling unit is the middle encoding type.

For example, in the fifth example described above, as shown in fig. 15, if the first preset difference threshold is 7, the shuffle type of the decoded stream 1 and the decoded stream 4 is M.

And step four, if the absolute value of the second difference value corresponding to the first shuffling unit is larger than or equal to a second preset difference value threshold value and the second difference value is a positive number, determining that the shuffling type of the first shuffling unit is a high coding type.

And fifthly, if the absolute value of the second difference value corresponding to the first shuffling unit is larger than or equal to a second preset difference value threshold value and the second difference value is a negative number, determining that the shuffling type of the first shuffling unit is a low coding type.

And step six, if the absolute value of the second difference value corresponding to the first shuffling unit is smaller than a second preset difference value threshold value, determining that the shuffling type of the first shuffling unit is a medium encoding type.

The processing procedure of step 1401 is similar to the processing procedure of step 601, and will not be described here again.

Step 1402, determining an output shuffling unit and an input shuffling unit corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule.

Optionally, the codec determines, in each first shuffling unit, an output shuffling unit and a processing procedure of an input shuffling unit corresponding to each output shuffling unit according to a shuffling type of each first shuffling unit and a preset shuffling rule, as follows:

Step one, if the first number of the first target shuffling units of the high decoding type is greater than the second number of the second target shuffling units of the low decoding type, determining the second target shuffling units as output shuffling units, and determining at least one first target shuffling unit as an input shuffling unit corresponding to the output shuffling unit in the first target shuffling units for each output shuffling unit.

And step two, if the first number of the first target shuffling units of the high decoding type is smaller than or equal to the second number of the second target shuffling units of the low decoding type, determining the first number of the second target shuffling units as output shuffling units, and determining at least one first target shuffling unit as an input shuffling unit corresponding to the output shuffling unit in the first target shuffling units for each output shuffling unit.

The processing of step 1402 is similar to the processing of step 602 and will not be described in detail herein.

Step 1305, a first preset number of to-be-shuffled stream blocks in the output shuffling unit are sent to the input shuffling unit to serve as first to-be-decoded stream blocks in the input shuffling unit, so that the first to-be-decoded stream blocks are decoded by the input shuffling unit according to a preset decoding algorithm.

Optionally, for each output shuffling unit, a second preset number of to-be-shuffled stream blocks in the output shuffling unit are sent to an input shuffling unit corresponding to the output shuffling unit, and the to-be-decoded stream blocks are used as first to-be-decoded stream blocks in the input shuffling unit.

For example, as shown in fig. 16, 4 shuffling units are used as a shuffling group for shuffling, each shuffling unit includes a decoding stream as an example, and the shuffling types of decoding stream 1 to decoding stream 4 are MHLM, then decoding stream 3 is an output shuffling unit, decoding stream 2 is an input shuffling unit of decoding stream 3, then decoding stream 1 may output 1 to-be-shuffled stream block to decoding stream 1, decoding stream 3 may output 1 to-be-shuffled stream block to decoding stream 2, and decoding stream 4 may output 1 to-be-shuffled stream block to decoding stream 4. Thus, as shown in fig. 17, after the shuffling is completed, the encoded length of the decoded stream 1 is n+8, the encoded length of the decoded stream 2 is n+9, the encoded length of the decoded stream 3 is n+3, the encoded length of the decoded stream 4 is n+6, and the shuffling types of the decoded streams 1 to 4 are MMMM.

The processing of step 1305 is similar to the processing of step 505 and will not be described in detail herein.

As an alternative embodiment, the codec determines at least one second shuffle group corresponding to the data block to be decoded according to the second shuffle level. Wherein the second shuffle group includes a plurality of second shuffle units, the second shuffle units including at least one decoded stream. And then, the codec re-performs shuffling processing by taking the stream blocks to be decoded in each second shuffling unit as the stream blocks to be shuffled until the coding length of each second shuffling unit meets the preset proximity condition.

In an implementation, after the codec shuffles the decoded stream corresponding to the data block to be decoded according to the first shuffling level, the codec may further determine at least one second shuffling group corresponding to the data block to be decoded according to the second shuffling level. Wherein the second shuffle group includes a plurality of second shuffle units, the second shuffle units including at least one decoded stream. The processing procedure of the codec to determine at least one second shuffle group corresponding to the data block to be decoded according to the second shuffle level is similar to step 1301, and will not be repeated here. Then, the codec may re-shuffle the stream block to be decoded in each second shuffling unit as the stream block to be shuffled until the encoding length of each second shuffling unit satisfies a preset proximity condition.

It should be noted that, after the codec shuffles the decoded stream corresponding to the data block to be decoded according to the second shuffling level, the codec may further shuffle the decoded stream corresponding to the data block to be decoded according to the third shuffling level. For example, the codec may shuffle the decoded stream corresponding to the data block to be decoded according to the shuffle level 1-shuffle level 2-shuffle level 3, shuffle the decoded stream corresponding to the data block to be decoded according to the shuffle level 1-shuffle level 3-shuffle level 2, and shuffle the decoded stream corresponding to the data block to be decoded according to the shuffle level 2-shuffle level 3-shuffle level 1.

As an alternative embodiment, when the codec detects that the decoding of the first decoding stream in the plurality of decoding streams is completed, a second preset number of stream blocks to be decoded in a second decoding stream which is not completed in the plurality of decoding streams are sent to the first decoding stream as stream blocks to be decoded in the first decoding stream.

In practice, the decoding rate at the time of decoding small data blocks is increased in order to further balance the encoding length of each decoded stream. The codec may have a third preset number stored therein in advance. The third preset number may be set empirically by the skilled person. When the codec detects that decoding of a first decoding stream of the plurality of decoding streams is completed (i.e., the first decoding stream outputs a decoding stream end (STREAMFINISH) flag), a third preset number of stream blocks to be decoded in a second decoding stream of the plurality of decoding streams, which is not completed, may be transmitted to the first decoding stream as stream blocks to be decoded in the first decoding stream. The second decoded stream may be all of the plurality of decoded streams or some of the plurality of decoded streams.

As an alternative embodiment, the codec obtains the decoded stream as follows:

Step one, obtaining coding head data corresponding to data to be decoded.

In implementation, for encoded data blocks, as shown in FIG. 10, the codec stores in the form of a header-data block (header-data block) map. When the codec needs to acquire the data to be decoded, the codec needs to acquire the encoding head data corresponding to the data to be decoded. As shown in fig. 18, the format of the encoding side data is a two-dimensional format, the low dimension is X, gao Weiwei Y, and the size of the encoding side data is the low dimension encoding side size (head size X) and the high dimension encoding side size (head size Y). The coding head data may be further divided uniformly in two dimensions into a plurality of coding head blocks (blocks).

And step two, determining a decoding area which comprises at least one coding head block and corresponds to the decoding instruction in the wharf data according to the received decoding instruction, and acquiring at least one data block to be decoded and corresponding to the at least one coding head block. The coding header block comprises a storage address of a coding data block corresponding to the coding header block.

In an implementation, after the codec obtains the encoding header data, a decoding area including at least one encoding header block corresponding to the decoding instruction may be determined in the dock data according to the received decoding instruction. As shown in fig. 18, the start address of the decoding area is the coding head start (HEAD START) position, and can be obtained by a low-dimensional coding head offset (head offset X) and a high-dimensional coding head offset (head offset Y). The decoding area contains a low-dimensional number of encoding header blocks (head number X) in the low-dimensional direction and a high-dimensional number of encoding header blocks (head number Y) in the high-dimensional direction. The coding header block comprises a storage address of a coding data block corresponding to the coding header block. After the codec obtains the decoding area, for each encoding header block in the decoding area, the codec may obtain a data block to be decoded according to a storage address of the encoded data block stored in the encoding header block.

And thirdly, dividing the data block to be decoded into a plurality of stream blocks aiming at each data block to be decoded in at least one data block to be decoded, and sending the plurality of stream blocks to a plurality of decoding streams corresponding to the data block to be decoded, wherein the plurality of stream blocks are used as stream blocks to be shuffled in the plurality of decoding streams.

In an implementation, to improve the throughput of decoding, a codec may decode the same block of data to be decoded in parallel through multiple decoding streams. Similarly, when the codec performs shuffling, for each data block to be decoded, the codec may divide the data block to be decoded into a plurality of stream blocks, and send the plurality of stream blocks to a plurality of decoding streams corresponding to the data block to be decoded, as a stream block to be shuffled in the plurality of decoding streams. Alternatively, the codec may divide the data block to be decoded into a plurality of row sequences, each row sequence containing at least one sub-block of data to be decoded. Then, for each sub-block of data to be decoded in each row sequence, the codec may divide the sub-block of data to be decoded into a plurality of stream blocks. In this way, it is ensured that the number of stream blocks in each decoded stream is similar.

Optionally, in the packing format, the data block to be decoded is composed of encoded data, and the size of the encoded data is equal to the size of the encoded data in the data block to be encoded.

In practice, as shown in fig. 10, in the packed format, the decoded data blocks are closely stored, each decoded data block is composed of encoded data, and the size of the encoded data is equal to the size of the encoded data in the data block to be encoded. I.e. the space occupied by the decoded data block is the actual size of the data block to be encoded after encoding.

Optionally, in the non-packed format, the data block to be decoded is composed of encoded data and filler data, and the sum of the size of the encoded data and the size of the filler data is equal to the size of the data block to be encoded.

In practice, as shown in fig. 10, in the non-packed format, each decoded data block consists of encoded data and padding data, the sum of the size of the encoded data and the size of the padding data being equal to the size of the data block to be encoded. I.e. each decoded data block occupies the same space and is equal to the size of the previous complete data block to be encoded. Wherein the former part is encoded data and the latter part is reserved padding data.

Optionally, the data block to be decoded is composed of encoded data sub-blocks corresponding to a plurality of source data sub-blocks belonging to the same source data block. In the packing format, the encoded data sub-block is composed of encoded data, and the size of the encoded data is equal to the size of the source data sub-block after the encoding of the data to be encoded. In the non-packed format, the encoded data sub-block is composed of encoded data and filler data, and the sum of the size of the encoded data and the size of the filler data is equal to the size of the source data sub-block.

Alternatively, the data division at decoding may be different from that at encoding, and when the data division of the decoding instruction is not aligned with the division of the encoded data block, there will be a boundary data block at encoding, for which only the required part may be read out if it has only TL part, if it is in the form of uncoded source data, and if it is in the form of encoded data, only the required part will be written to the destination address after the entire encoded data block has to be decompressed. If the boundary data block itself contains four parts and is different from the division at encoding, then a subdivision of the 4 encoded data sub-blocks occurs. As shown in fig. 19, the solid line box indicates the data division at the time of encoding (encoding), and the dotted line box indicates the data division at the time of decoding, and the division points are different, so that the data block to be decoded at the time of decoding may be a part of one encoded data block, such as TL block indicated by left diagonal line block, or may be a part of at most 4 sub data blocks spliced, such as BR block indicated by vertical line block.

The four cases shown in fig. 20 are used for the division, and the positions of the cross points of the decoding division and the cross points of the encoding division are respectively corresponding to different positions, so that 4 different data blocks (TL, TR, BL, BR) are needed to be taken in decoding in each case, and 16 different data blocks are used in decoding. Only the most general case is considered here, and other cases where the division is less than 4 blocks or where the division at the time of encoding and decoding is aligned can be regarded as special cases of these 4 cases.

The embodiment of the application provides a shuffling method. The codec determines at least one first shuffle group corresponding to the data block to be decoded according to the first shuffle level. Wherein the first shuffle group includes a plurality of first shuffle units, the first shuffle units including at least one decoded stream. Then, the codec determines the coding length of each first shuffling unit, and determines an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit. And then, the coder and decoder sends a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit to serve as the first to-be-decoded stream blocks in the input shuffling unit, so that the first to-be-decoded stream blocks are decoded according to a preset decoding algorithm through the input shuffling unit. In this way, by shuffling each decoding stream of the data block to be decoded, the coding length of each decoding stream can be guaranteed to be similar, so that excessive complements caused by alignment requirements among each decoding stream are avoided, and the decoding streams are reduced. Meanwhile, the deadlock condition among the decoding streams can be avoided.

The embodiment of the application also provides a computing device, which comprises a processor;

The processor is used for receiving the source data and encoding the source data by using the method to obtain encoded data.

the processor is used for receiving the coded data and decoding the coded data by using the method to obtain source data.

It should be noted that, for simplicity of description, the foregoing method embodiments are all depicted as a series of acts, but it should be understood by those skilled in the art that the present disclosure is not limited by the order of acts described, as some steps may occur in other orders or concurrently in accordance with the disclosure. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all alternative embodiments, and that the acts and modules referred to are not necessarily required by the present disclosure.

It should be further noted that, although the steps in the flowcharts of fig. 2 to 4 are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2-4 may comprise a plurality of sub-steps or phases, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed sequentially, but may be performed alternately or alternately with at least a portion of the sub-steps or phases of other steps or other steps.

It should be understood that the above-described device embodiments are merely illustrative and that the device of the present disclosure may be implemented in other ways. For example, the division of the units/modules in the above embodiments is merely a logic function division, and there may be another division manner in actual implementation. For example, multiple units, modules, or components may be combined, or may be integrated into another system, or some features may be omitted or not performed.

In addition, unless specifically stated, each functional unit/module in the embodiments of the present disclosure may be integrated into one unit/module, or each unit/module may exist alone physically, or two or more units/modules may be integrated together. The integrated units/modules described above may be implemented either in hardware or in software program modules.

The integrated units/modules, if implemented in hardware, may be digital circuits, analog circuits, etc. Physical implementations of hardware structures include, but are not limited to, transistors, memristors, and the like. The artificial intelligence processor may be any suitable hardware processor, such as CPU, GPU, FPGA, DSP and an ASIC, etc., unless otherwise specified. The Memory unit may be any suitable magnetic or magneto-optical storage medium, such as resistive Random Access Memory RRAM (Resistive Random Access Memory), dynamic Random Access Memory DRAM (Dynamic Random Access Memory), static Random Access Memory SRAM (Random-Access Memory), enhanced dynamic Random Access Memory EDRAM (ENHANCED DYNAMIC Random Access Memory), high-Bandwidth Memory HBM (High-Bandwidth Memory), hybrid Memory cube HMC (Hybrid Memory Cube), etc., unless otherwise indicated.

The integrated units/modules may be stored in a computer readable memory if implemented in the form of software program modules and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present disclosure may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, containing several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in the various embodiments of the present disclosure. And the aforementioned memory includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments. The technical features of the foregoing embodiments may be arbitrarily combined, and for brevity, all of the possible combinations of the technical features of the foregoing embodiments are not described, however, all of the combinations of the technical features should be considered as being within the scope of the disclosure.

The foregoing may be better understood in light of the following clauses:

Clause A1, corresponding to claim 1; clause A2, correspondence 2; clause A3, correspondence 3; clause A4, correspondence 4; clause A5, correspondence 5; clause A6, correspondence 6; clause A7, correspondence 7; clause A8, correspondence 8; clause A9, correspondence 9; clause a10, correspondence 10; clause a11, correspondence 11; clause a12, correspondence 12; clause a13, corresponding to claim 13; clause a14, correspondence 14; clause a15, correspondence 15; clause a16, correspondence 16; clause a17, correspondence 17; clause a18, correspondence 18; clause a19, correspondence 19; clause a20, correspondence 20; clause a21, corresponding to claim 21; clause a22, correspondence 22; clause a23, corresponding to weight 23; clause a24, correspondence 24; clause a25, correspondence 25; clause a26, corresponding to weight 26; clause a27, correspondence 27; clause a28, correspondence 28; clause a29, correspondence 29; clause a30, correspondence 30; clause a31, correspondence 31; clause a32, correspondence 32; clause a33, correspondence 33; clause a34, correspondence 34; clause a35, corresponds to claim 35.

For example, clause A1, a shuffling method, the method being applied to a general purpose processor, the method comprising:

Clause A2, the method according to clause A1, wherein determining the output shuffling unit and the input shuffling unit in each of the first shuffling units according to the coding length of each of the first shuffling units, comprises:

Determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit;

Determining an output shuffling unit and an input shuffling unit corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule;

The sending the first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as the first to-be-encoded stream blocks in the input shuffling unit includes:

And for each output shuffling unit, sending a second preset number of to-be-shuffled stream blocks in the output shuffling unit to an input shuffling unit corresponding to the output shuffling unit as a first to-be-encoded stream block in the input shuffling unit.

Clause A3, the method according to clause A2, wherein determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit includes:

Determining, for each of the first shuffling units, a first difference of the encoding length of that first shuffling unit from the encoding lengths of the other first shuffling units;

And determining the shuffling type of the first shuffling unit according to the first difference value corresponding to the first shuffling unit.

Clause A4, the method according to clause A3, wherein determining the shuffling type of the first shuffling unit according to the first difference value corresponding to the first shuffling unit includes:

If a target difference value with an absolute value larger than or equal to a first preset difference value threshold exists in a first difference value corresponding to the first shuffling unit, and the target difference values are all positive numbers, determining that the shuffling type of the first shuffling unit is a high coding type;

And if a target difference value with an absolute value larger than or equal to a first preset difference value threshold exists in the first difference value corresponding to the first shuffling unit, and a negative target difference value exists in the target difference value, determining that the shuffling type of the first shuffling unit is a low coding type.

Clause A5, the method of clause A4, further comprising:

And if the target difference value with the absolute value larger than or equal to the first preset difference value threshold value does not exist in the first difference value corresponding to the first shuffling unit, determining that the shuffling type of the first shuffling unit is the middle coding type.

Clause A6, the method according to clause A2, wherein determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit includes:

determining an average coding length according to the coding length of each first shuffling unit;

Determining, for each of the first shuffle units, a second difference of the code length of the first shuffle unit from the average code length;

And determining the shuffling type of the first shuffling unit according to the second difference value corresponding to the first shuffling unit.

Clause A7, the method according to clause A6, wherein determining the shuffling type of the first shuffling unit according to the second difference value corresponding to the first shuffling unit includes:

If the absolute value of a second difference value corresponding to the first shuffling unit is larger than or equal to a second preset difference value threshold value and the second difference value is a positive number, determining that the shuffling type of the first shuffling unit is a high coding type;

And if the absolute value of the second difference value corresponding to the first shuffling unit is larger than or equal to a second preset difference value threshold value and the second difference value is a negative number, determining that the shuffling type of the first shuffling unit is a low coding type.

Clause A8, the method of clause A7, further comprising:

And if the absolute value of the second difference value corresponding to the first shuffling unit is smaller than a second preset difference value threshold value, determining that the shuffling type of the first shuffling unit is a medium coding type.

Clause A9, the method according to clause A2, wherein determining an output shuffling unit and an input shuffling unit corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule, includes:

if the first number of the first target shuffling units of the high coding type is smaller than or equal to the second number of the second target shuffling units of the low coding type, determining the first target shuffling units as output shuffling units, and determining at least one second target shuffling unit as an input shuffling unit corresponding to the output shuffling unit in the second target shuffling units for each output shuffling unit;

If the first number of the first target shuffling units of the high encoding type is greater than the second number of the second target shuffling units of the low encoding type, the second number of the first target shuffling units is determined as output shuffling units, and for each output shuffling unit, at least one second target shuffling unit is determined as an input shuffling unit corresponding to the output shuffling unit in the second target shuffling units.

Clause a10, the method of clause A1, further comprising:

Acquiring source data to be encoded;

According to the received coding instruction, determining a coding region corresponding to the coding instruction in the source data, wherein the coding region comprises at least one data block to be coded, and the data block to be coded is a source data block or a source data sub-block;

For each data block to be coded in the at least one data block to be coded, dividing the data block to be coded into a plurality of stream blocks, and sending the plurality of stream blocks to a plurality of coded streams corresponding to the data block to be coded, wherein the coded streams are used as the stream blocks to be shuffled in the plurality of coded streams.

Clause a11, the method of clause A1, further comprising:

when the first coded stream in the plurality of coded streams is detected to be coded, a fourth preset number of stream blocks to be coded in a second coded stream which is not coded in the plurality of coded streams are sent to the first coded stream to serve as stream blocks to be coded in the first coded stream.

Clause a12, the method of clause A1, further comprising:

And if the total coding length of the stream blocks to be coded in each first shuffling group is larger than the original length of the data blocks to be coded, terminating the coding processing of the data blocks to be coded.

Clause a13, the method of clause A1, the method comprising:

determining at least one second shuffling group corresponding to the data blocks to be encoded according to the second shuffling grade, wherein the second shuffling group comprises a plurality of second shuffling units, and the second shuffling units comprise at least one encoding stream;

and re-performing shuffling treatment on the stream blocks to be coded in each second shuffling unit as the stream blocks to be shuffled until the coding length of each second shuffling unit meets the preset proximity condition.

Clause a14, the method according to clause A1, wherein the encoding, by the input shuffling unit, the first to-be-encoded stream block according to a preset encoding algorithm, includes:

coding preset characters contained in the first stream block to be coded according to a first coding rule corresponding to the preset characters to obtain first coded data;

And encoding the first encoded data according to a preset second encoding rule.

Clause a15, the method of clause a10, further comprising:

Generating a coding head block of a coding data block corresponding to each data block to be coded, wherein the coding head block comprises a storage address of the coding data block corresponding to the coding head block;

Storing each coding end block, and storing each coding data block after each coding end block.

Clause a16, the method of clause a15, wherein the encoded data block consists of encoded data, and the size of the encoded data is equal to the size of the encoded data in the data block to be encoded; or alternatively

The encoded data block is composed of encoded data and padding data, and the sum of the size of the encoded data and the size of the padding data is equal to the size of the data block to be encoded.

Clause a17, the method of clause a15, wherein the encoded data block consists of encoded data sub-blocks corresponding to a plurality of source data sub-blocks belonging to the same source data block;

The coded data sub-block consists of coded data, and the size of the coded data is equal to the size of the source data sub-block after the data to be coded are coded; or alternatively

The encoded data sub-block is composed of encoded data and padding data, and a sum of a size of the encoded data and a size of the padding data is equal to a size of the source data sub-block.

Clause a18, a shuffling method, the method comprising:

Clause a19, the method of clause a18, wherein determining the output shuffle unit and the input shuffle unit in each of the first shuffle units according to the coding length of each of the first shuffle units, comprises:

The sending the first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as the first to-be-decoded stream blocks in the input shuffling unit includes:

And for each output shuffling unit, sending a second preset number of to-be-shuffled stream blocks in the output shuffling unit to an input shuffling unit corresponding to the output shuffling unit as a first to-be-decoded stream block in the input shuffling unit.

Clause a20, the method according to clause a19, wherein determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit, comprises:

Clause a21, the method according to clause a20, wherein determining the shuffling type of the first shuffling unit according to the first difference value corresponding to the first shuffling unit includes:

Clause a22, the method of clause a21, the method further comprising:

Clause a23, the method according to clause a19, wherein determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit, comprises:

Clause a24, the method according to clause a23, wherein determining the shuffling type of the first shuffling unit according to the second difference value corresponding to the first shuffling unit includes:

Clause a25, the method of clause a24, further comprising:

Clause a26, the method according to clause a19, wherein the determining, according to the shuffling type and the preset shuffling rule of each first shuffling unit, the output shuffling unit and the input shuffling unit corresponding to each output shuffling unit in each first shuffling unit includes:

If the first number of the first target shuffling units of the high decoding type is greater than the second number of the second target shuffling units of the low decoding type, determining the second target shuffling units as output shuffling units, and determining at least one first target shuffling unit as an input shuffling unit corresponding to the output shuffling unit in the first target shuffling units for each output shuffling unit;

If the first number of the first target shuffling units of the high decoding type is smaller than or equal to the second number of the second target shuffling units of the low decoding type, the first number of the second target shuffling units are determined as output shuffling units, and for each output shuffling unit, at least one first target shuffling unit is determined as an input shuffling unit corresponding to the output shuffling unit among the first target shuffling units.

Clause a27, the method of clause a18, further comprising:

acquiring coding head data corresponding to data to be decoded;

according to the received decoding instruction, determining a decoding area corresponding to the decoding instruction and containing at least one coding head block in the coding head data, and acquiring at least one data block to be decoded corresponding to the at least one coding head block, wherein the coding head block contains a storage address of the coding data block corresponding to the coding head block;

for each data block to be decoded in the at least one data block to be decoded, dividing the data block to be decoded into a plurality of stream blocks, and sending the plurality of stream blocks to a plurality of decoding streams corresponding to the data block to be decoded, wherein the plurality of stream blocks are used as the stream blocks to be shuffled in the plurality of decoding streams.

Clause a28, the method according to clause a27, wherein the data block to be decoded consists of encoded data, and the size of the encoded data is equal to the size of the encoded data in the data block to be encoded; or alternatively

The data block to be decoded consists of encoded data and padding data, and the sum of the size of the encoded data and the size of the padding data is equal to the size of the data block to be encoded.

Clause a29, the method of clause a27, wherein the data block to be decoded consists of coded data sub-blocks corresponding to a plurality of source data sub-blocks belonging to the same source data block;

Clause a30, the method of clause a18, the method further comprising:

And when detecting that the decoding of a first decoding stream in a plurality of decoding streams is finished, sending a second preset number of stream blocks to be decoded in a second decoding stream which is not finished in the plurality of decoding streams to the first decoding stream as stream blocks to be decoded in the first decoding stream.

Clause a31, the method of clause a18, the method further comprising:

Determining at least one second shuffling group corresponding to the data blocks to be decoded according to the second shuffling grade, wherein the second shuffling group comprises a plurality of second shuffling units, and the second shuffling units comprise at least one decoding stream;

And re-performing shuffling treatment on the stream blocks to be decoded in each second shuffling unit as the stream blocks to be shuffled until the coding length of each second shuffling unit meets the preset proximity condition.

Clause a32, a computing device comprising a processor;

the processor is configured to receive source data and encode the source data using the method of any of clauses A1-17 to obtain encoded data.

Clause a33, the computing device of clause a32, the processor comprising a control module, an on-chip memory module, and an arithmetic module;

Clause a34, a computing device comprising a processor;

the processor is configured to receive encoded data and decode the encoded data using the method of any of clauses a18-31 to obtain source data.

Clause a35, the computing device of clause a34, the processor comprising a control module, an on-chip memory module, and an arithmetic module;

The foregoing has outlined rather closely the embodiments of the present disclosure, and detailed description of the principles and embodiments of the present disclosure have been presented herein with the application of specific examples, the description of the examples above being merely intended to facilitate an understanding of the method of the present disclosure and its core ideas. Also, those skilled in the art, based on the teachings of the present disclosure, may make modifications or variations in the specific embodiments and application scope of the present disclosure, all falling within the scope of the protection of the present disclosure. In view of the foregoing, this description should not be construed as limiting the disclosure.

Claims

1. A method of shuffling, the method being applied to a general purpose processor, the method comprising:

Transmitting a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as first to-be-encoded stream blocks in the input shuffling unit, so as to encode the first to-be-encoded stream blocks according to a preset encoding algorithm through the input shuffling unit;

And determining an output shuffling unit and an input shuffling unit in each first shuffling unit according to the coding length of each first shuffling unit, comprising: determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit; determining an output shuffling unit and an input shuffling unit corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule;

The sending the first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as the first to-be-encoded stream blocks in the input shuffling unit includes: for each output shuffling unit, sending a second preset number of to-be-shuffled stream blocks in the output shuffling unit to an input shuffling unit corresponding to the output shuffling unit as a first to-be-encoded stream block in the input shuffling unit;

The determining the shuffling type of each first shuffling unit according to the encoding length of each first shuffling unit comprises:

Determining, for each of the first shuffling units, a first difference of the encoding length of that first shuffling unit from the encoding lengths of the other first shuffling units; determining the shuffling type of the first shuffling unit according to a first difference value corresponding to the first shuffling unit;

the method further comprises the steps of:

2. The method of claim 1, wherein determining the shuffling type of the first shuffling unit based on the first difference value corresponding to the first shuffling unit, comprises:

3. The method according to claim 2, wherein the method further comprises:

4. The method of claim 2, wherein said determining the shuffling type of each first shuffling unit based on the encoding length of each first shuffling unit comprises:

5. The method of claim 1, wherein determining the shuffling type of the first shuffling unit based on the second difference value corresponding to the first shuffling unit, comprises:

6. The method of claim 5, wherein the method further comprises:

7. The method according to claim 1, wherein determining the output shuffling units and the input shuffling units corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule, comprises:

8. The method according to claim 1, wherein the method further comprises:

Acquiring source data to be encoded;

9. The method according to claim 1, wherein the method further comprises:

10. The method according to claim 1, characterized in that the method comprises:

11. The method according to claim 1, wherein said encoding the first stream block to be encoded by the input shuffling unit according to a preset encoding algorithm, comprises:

And encoding the first encoded data according to a preset second encoding rule.

12. The method of claim 8, wherein the method further comprises:

13. The method according to claim 12, wherein the encoded data block is composed of encoded data, and the size of the encoded data is equal to the size of the encoded data in the data block to be encoded; or alternatively

14. The method of claim 12, wherein the encoded data block is comprised of encoded data sub-blocks corresponding to a plurality of source data sub-blocks belonging to the same source data block;

15. A method of shuffling, the method comprising:

Transmitting a first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as first to-be-decoded stream blocks in the input shuffling unit, so as to decode the first to-be-decoded stream blocks according to a preset decoding algorithm through the input shuffling unit;

the sending the first preset number of to-be-shuffled stream blocks in the output shuffling unit to the input shuffling unit as the first to-be-decoded stream blocks in the input shuffling unit includes: for each output shuffling unit, sending a second preset number of to-be-shuffled stream blocks in the output shuffling unit to an input shuffling unit corresponding to the output shuffling unit as a first to-be-decoded stream block in the input shuffling unit;

the method further comprises the steps of:

16. The method of claim 15, wherein determining the shuffling type of the first shuffling unit based on the first difference value corresponding to the first shuffling unit, comprises:

17. The method of claim 16, wherein the method further comprises:

18. The method of claim 15, wherein said determining the shuffling type of each first shuffling unit based on the encoding length of each first shuffling unit comprises:

19. The method of claim 15, wherein determining the shuffling type of the first shuffling unit based on the second difference value corresponding to the first shuffling unit, comprises:

20. The method of claim 19, wherein the method further comprises:

21. The method according to claim 15, wherein determining the output shuffling units and the input shuffling units corresponding to each output shuffling unit in each first shuffling unit according to the shuffling type of each first shuffling unit and a preset shuffling rule, comprises:

22. The method of claim 15, wherein the method further comprises:

acquiring coding head data corresponding to data to be decoded;

23. The method of claim 22, wherein the block of data to be decoded consists of encoded data having a size equal to the size of the block of data to be encoded; or alternatively

24. The method of claim 22, wherein the data block to be decoded consists of encoded data sub-blocks corresponding to a plurality of source data sub-blocks belonging to the same source data block;

25. The method of claim 15, wherein the method further comprises:

26. A computing device, the computing device comprising a processor;

The processor is configured to receive source data and encode the source data using the method of any one of claims 1-14 to obtain encoded data.

27. The computing device of claim 26, wherein the processor comprises a control module, an on-chip memory module, and an arithmetic module;

28. A computing device, the computing device comprising a processor;

the processor is configured to receive encoded data and decode the encoded data using the method of any of claims 15-25 to obtain source data.

29. The computing device of claim 28, wherein the processor comprises a control module, an on-chip memory module, and an arithmetic module;