RU2815733C1 - Image encoding method, image decoding method, image encoding device, image decoding device, and image encoding and decoding device - Google Patents

Abstract

FIELD: image encoding and decoding.

SUBSTANCE: image encoding method includes: performing arithmetic encoding on a first flag (greater1_flag) that indicates whether the absolute value of a target coefficient in the target coefficient block is greater than 1 (S701); and performing arithmetic encoding on a second flag (greater2_flag) that indicates whether the absolute value is greater than 2 (S702). In the first flag arithmetic coding and the second flag arithmetic coding, it is determined whether the immediately preceding coefficient block that was encoded immediately before the target coefficient block includes a coefficient having an absolute value greater than a threshold value (S711). Based on the result of the determination, appropriate contexts are selected for use in arithmetic coding on the first and second flags (S712).

EFFECT: reducing the amount of processing while simultaneously suppressing the deterioration in encoding efficiency.

17 cl, 64 dwg

Description

Field of technology

[0001] The present invention relates to image encoding methods and image decoding methods.

Prior Art

[0002] In recent years, with the rapid advancement of digital video technology, there have been more opportunities to compress-encode video signals (temporal sequences of moving images) and record the resulting video signals onto a recording medium such as a digital versatile disk (DVD) or hard drive, or distribute them over the network. An example of image coding standards is H.264/AVC (MPEG-4 AVC). The High Efficiency Image Coding (HEVC) standard has recently been studied as a next-generation standard (see, for example, Non-Patent Document 1).

List of links

Non-patent literature

[0003] [Non-Patent Document 1] Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 9th Meeting: Geneva, CH, 27 Apr. - 7 May 2012 JCTVC-11003_d4.doc, High efficiency video coding (HEVC) text specification draft 7 http://phenix.it-sudparis.eu/jct/doc_end_user/documents/9_Geneva/wg11/JCTVC-I1003-v5.zip

The essence of the invention

Technical problem

[0004] For such an image encoding method and an image decoding method, it was necessary to reduce the amount of processing while suppressing degradation in encoding efficiency.

[0005] The present invention provides an image encoding method and an image decoding method, each of which is capable of reducing the amount of processing while suppressing degradation in encoding efficiency.

The solution of the problem

[0006] To solve the above object, according to an aspect of the present invention, there is provided a method for encoding an image using arithmetic encoding, wherein the image encoding method includes: performing arithmetic encoding on a first flag that indicates whether the absolute value of a target coefficient in a target block coefficients greater than 1; and performing arithmetic encoding on a second flag that indicates whether the absolute value is greater than 2, wherein, by performing arithmetic encoding on the first flag and performing arithmetic encoding on the second flag, determining whether the immediately preceding coefficient block includes has been encoded immediately before the target block of coefficients, a coefficient having an absolute value greater than the threshold value, and a context to be used in arithmetic encoding over the first flag and a context to be used in arithmetic encoding over the second flag are selected based on the determination result.

[0007] These general and specific aspects may be implemented using a system, method, integrated circuit, computer program, or computer-readable storage medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or computer-readable media records.

Preferred Effects of the Invention

[0008] The present invention provides an image encoding method and an image decoding method, each of which is capable of reducing the amount of processing while suppressing degradation in encoding efficiency.

Brief description of drawings

[0009]

Fig. 1 is a block diagram of an image encoding apparatus according to Embodiment 1.

Fig. 2 is a flowchart of image encoding according to Image Embodiment 1.

Fig. 3 is a block diagram of a variable word length encoding block according to Embodiment 1.

Fig. 4 is a flowchart of variable word length encoding according to Embodiment 1.

Fig. 5 is a block diagram of a layer encoding unit according to Embodiment 1.

Fig. 6 is a flowchart of layer encoding according to Embodiment 1.

Fig. 7 is a flowchart of greater1_flag encoding according to Embodiment 1.

Fig. 8 is a flowchart of greater2_flag encoding according to Embodiment 1.

Fig. 9 is a flowchart of remainder encoding according to Embodiment 1.

Fig. 10 is a flowchart of context set selection according to Embodiment 1.

Fig. 11 is a flowchart for selecting a greater1_flag context according to Embodiment 1.

Fig. 12 is a table showing an example of context numbers according to Embodiment 1.

Fig. 13 is a flowchart of residual binarization according to Embodiment 1.

Fig. 14 is a flowchart for determining a prefix and a determining suffix according to Embodiment 1.

Fig. 15A is a table showing an example of correspondence relationships between a residue, prefix and suffix bin according to Embodiment 1.

Fig. 15B is a table showing an example of matching relationships between a residue, prefix and suffix bin according to Embodiment 1.

Fig. 15C is a table showing an example of correspondence relationships between a residue, prefix and suffix bin according to Embodiment 1.

Fig. 16 is a flowchart of outputting a prefix bin according to Embodiment 1.

Fig. 17 is a flowchart of outputting a suffix bin according to Embodiment 1.

Fig. 18 is a flowchart for updating binarization parameters according to Embodiment 1.

Fig. 19 is a table showing evaluation results of an image encoding method according to Embodiment 1.

Fig. 20 is a block diagram of an image decoding apparatus according to Embodiment 2.

Fig. 21 is a flowchart of image decoding according to Embodiment 2.

Fig. 22 is a block diagram of a variable word length decoding unit according to Embodiment 2.

Fig. 23 is a flowchart of variable word length decoding according to Embodiment 2.

Fig. 24 is a block diagram of a layer decoding unit according to Embodiment 2.

Fig. 25 is a flowchart of layer decoding according to Embodiment 2.

Fig. 26 is a flowchart of greater1_flag decoding according to Embodiment 2.

Fig. 27 is a flowchart of greater2_flag decoding according to Embodiment 2.

Fig. 28 is a flowchart of remainder decoding according to Embodiment 2.

Fig. 29 is a flowchart of multi-valued remainder conversion according to Embodiment 2.

Fig. 30 is a flowchart of prefix decoding according to Embodiment 2.

Fig. 31 is a flowchart of suffix decoding and prefix and suffix combining according to Embodiment 2.

Fig. 32A is a flowchart of an image encoding method according to Embodiment 1.

Fig. 32B is a flowchart of first flag encoding and second flag encoding according to Embodiment 1.

Fig. 33A is a flowchart of an image decoding method according to Embodiment 2.

Fig. 33B is a flowchart of decoding a first flag and decoding a second flag according to Embodiment 2.

Fig. 34 is a block diagram of a layer encoding unit according to Embodiment 3.

Fig. 35 is a flowchart of layer encoding according to Embodiment 3.

Fig. 36 is a flowchart of context set selection according to Embodiment 3.

Fig. 37 is a table showing evaluation results of an image encoding method according to Embodiment 3.

Fig. 38 is a block diagram of a layer decoding apparatus according to Embodiment 4.

Fig. 39 is a flowchart of layer decoding according to Embodiment 4.

Fig. 40 is a general configuration of a content delivery system for implementing content distribution services.

Fig. 41 - general configuration of a digital broadcasting system.

Fig. 42 is a block diagram illustrating an example of a TV configuration.

Fig. 43 is a block diagram illustrating a configuration example of an information reproducing/recording unit that reads and writes information from and to a recording medium that is an optical disc.

Fig. 44 shows an example of a configuration of a recording medium that is an optical disc.

Fig. 45A shows an example of a cell phone.

Fig. 45B is a block diagram showing a configuration example of a cellular phone.

Fig. 46 illustrates the structure of multiplexed data.

Fig. 47 schematically shows how each stream is multiplexed in the multiplexed data.

Fig. 48 shows in more detail how a video stream is stored in a PES packet stream.

Fig. 49 shows the structure of TS packets and source packets in multiplexed data.

Fig. 50 shows the data structure of the RMT.

Fig. 51 shows the internal information structure of multiplexed data.

Fig. 52 shows the internal structure of stream attribute information.

Fig. 53 shows steps for identifying video data.

Fig. 54 shows an example of an integrated circuit configuration for implementing a moving image encoding method and a moving image decoding method according to each of the embodiments.

Fig. 55 shows the configuration for switching between drive frequencies.

Fig. 56 shows steps for identifying video data and switching between excitation frequencies.

Fig. 57 shows an example of a lookup table in which video data standards are associated with drive frequencies.

Fig. 58A is a diagram showing an example configuration for sharing a signal processing unit module.

Fig. 58B is a diagram showing another configuration example for sharing a signal processing unit module.

Description of Embodiments

[0010] Basic knowledge forming the basis of the present invention

With respect to the traditional image encoding method, the inventors have discovered the following problem.

[0011] An image encoding method according to the present HEVC standard (see, for example, Non-Patent Document 1) includes a step of predicting the encoded image, a step of calculating the difference between the predicted image and the target image, a step of converting the resulting differential image into frequency coefficients, and a step performing arithmetic coding on frequency coefficients. In arithmetic coding, coefficients in a coefficient block to be encoded (encoding coefficient block) are encoded sequentially in order from the higher frequency component to the lower frequency component. Here, the context is selected according to the encoded coefficient, and arithmetic encoding is performed on the coefficient to be encoded (encoding coefficient) according to the symbol occurrence probability determined based on the selected context.

[0012] In a normal image, the coefficient with a lower frequency component is likely to have a larger value. Therefore, it is possible that the probability of occurrence of a symbol having an offset will be caused by determining the context with reference to the encoded coefficient (a coefficient with a higher frequency than the encoding coefficient frequency). For example, if the encoded coefficient (a coefficient with a higher frequency than the encoding coefficient frequency) is large, then there is a high probability that the encoding coefficient is also large. Therefore, the image encoding apparatus can reduce the resulting encoding amount using context for a large value. In the present HEVC standard, the number of encoded coefficients, each of which has an absolute value of 2 or more, is counted, and the context for the encoding coefficient is determined according to the number.

[0013] However, the inventors have discovered that the traditional image encoding method requires counting the number of encoded coefficients, each of which has an absolute value of 2 or more, and therefore must have processing for counting and a resistor that stores the count value.

[0014] In order to solve the above problem, according to an aspect of the present invention, there is provided a method for encoding an image using arithmetic encoding, wherein the image encoding method includes: performing arithmetic encoding on a first flag that indicates whether the absolute value of a target coefficient is target block of coefficients greater than 1; and performing arithmetic encoding on a second flag indicating whether the absolute value is greater than 2, wherein, by performing arithmetic encoding on the first flag and performing arithmetic encoding on the second flag, determining whether the immediately preceding block of coefficients that was encoded immediately before the target block of coefficients, a coefficient having an absolute value greater than the threshold value, and a context to be used in arithmetic encoding over the first flag and a context to be used in arithmetic encoding over the second flag are selected based on the determination result.

[0015] Thus, the image encoding method selects a context according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image encoding method can reduce the amount of processing compared to the case where the context is selected depending on the number of coefficients. In addition, the image encoding method can suppress the reduction in encoding efficiency and reduce the processing amount.

[0016] For example, it is possible that when performing arithmetic encoding on the first flag and performing arithmetic encoding on the second flag, said determination is made based on a variable generated in the arithmetic encoding on the immediately preceding block of coefficients.

[0017] Thus, the image encoding method can reduce the amount of processing by performing text selection based on a variable that has been used in other processing.

[0018] For example, it is also possible that the image encoding method further includes binarizing (binarizing) the remainder according to a lookup table designated by a binarization parameter from among a plurality of lookup tables when the absolute value of the target coefficient is greater than two, wherein the remainder is a value obtained by subtracting 3 from the absolute value, wherein the binarization parameter is determined according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than the threshold value, and wherein the variable is a binarization parameter.

[0019] For example, it is further possible that the threshold value is 1.

[0020] According to another aspect of the present invention, there is provided a method for decoding an image using arithmetic decoding, wherein the method for decoding the image includes: performing arithmetic decoding on a first flag that indicates whether the absolute value of a target coefficient in a target coefficient block is greater than 1; and performing arithmetic decoding on a second flag that indicates whether the absolute value is greater than 2, wherein, by performing arithmetic decoding on the first flag and performing arithmetic decoding on the second flag, determining whether the immediately preceding coefficient block includes was decoded immediately before the target block of coefficients, a coefficient having an absolute value greater than the threshold value, and a context to be used in arithmetic decoding over the first flag and a context to be used in arithmetic decoding over the second flag are selected based on the determination result.

[0021] Thus, the image decoding method selects a context according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image decoding method can reduce the amount of processing compared to the case where the context is selected depending on the number of coefficients. In addition, the image decoding method can suppress the reduction in encoding efficiency and reduce the processing amount.

[0022] For example, it is possible that when performing arithmetic decoding on the first flag and performing arithmetic decoding on the second flag, said determination is made based on a variable generated in the arithmetic decoding on the immediately preceding block of coefficients.

[0023] In this way, the image decoding method can reduce the amount of processing by performing text selection based on a variable that has been used in other processing.

[0024] For example, it is also possible that the image decoding method further includes: converting the remainder into a multi-valued value according to a lookup table designated by a binarization parameter from among a plurality of lookup tables when the absolute value of the target coefficient is greater than 2, wherein the remainder is the value obtained by subtracting 3 from the absolute value; and determining a binarization parameter according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than the threshold value, wherein the variable is a binarization parameter.

[0025] For example, it is further possible that the threshold value is 1.

[0026] According to another aspect of the present invention, there is provided an image encoding device that performs arithmetic encoding, the image encoding device comprising: a control circuit and storage means accessible from the control circuit, wherein the control circuit executes an image encoding method according to claim 1 of the claims. .

[0027] Here, the image decoding apparatus selects a context according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image decoding apparatus can reduce the amount of processing compared with the case where the context is selected according to the number of coefficients. In addition, the image encoding device can suppress the reduction in encoding efficiency and reduce the processing amount.

[0028] According to another aspect of the present invention, there is provided an image decoding device that performs arithmetic decoding, wherein the image decoding device includes: a control circuit and storage means accessible from the control circuit, wherein the control circuit executes an image decoding method according to claim 5 inventions.

[0029] Here, the image decoding device selects a context according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. As a result, the image decoding apparatus can reduce the amount of processing compared with the case where the context is selected according to the number of coefficients. In addition, the image decoding apparatus can suppress the reduction in encoding efficiency and reduce the processing amount.

[0030] According to another aspect of the present invention, there is provided an image encoding and decoding apparatus including the above-described image encoding apparatus and an image decoding apparatus.

[0031] These general and specific aspects may be implemented using a system, method, integrated circuit, computer program, or computer readable storage medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or computer readable media records.

[0032] Next, some exemplary embodiments of the image encoding device and the image decoding device are described with reference to the accompanying drawings.

[0033] Each of the exemplary embodiments described below shows a specific example of the present invention. The numerical values, shapes, materials, structural elements, arrangement and connection of structural elements, steps, processing order of steps, etc. presented in the following exemplary embodiments are only examples and therefore do not limit the present invention. Therefore, among the structural elements in the following exemplary embodiments, structural elements not mentioned in any of the independent claims are described as arbitrary structural elements.

[0034] Embodiment 1

Embodiment 1 describes an embodiment of an image encoding apparatus.

[0035] General structure

Fig. 1 is a block diagram of a structure of an image encoding apparatus according to Embodiment 1.

[0036] The image encoding device 100 shown in FIG. 1 encodes the input image 121 to generate a codestream 125 (coded bitstream). The image encoding device 100 includes an encoding block division unit 101, a subtraction unit 102, a transform unit 103, a variable word length encoding unit 104, an inverse transform unit 105, an adder unit 106, a frame memory 107, and a prediction unit 108.

[0037] Processing (General)

Next, with reference to FIG. 2 describes the general encoding flow.

[0038] Step S101

The encoding block division unit 101 divides the input image 121 into blocks to be encoded (encoding blocks) 122, and sequentially supplies the encoding blocks 122 to both the subtraction unit 102 and the prediction unit 108. Here, each of the encoding blocks 122 has a variable size, and the encoding block division unit 101 thus accesses the features of the input image 121 to divide the input image 121 into encoding blocks 122. For example, one encoding block 122 has a minimum horizontal of 4 pixels x vertical of 4 pixels and a maximum horizontal of 32 pixels x vertical of 32 pixels.

[0039] Step S102

The prediction unit 108 generates a prediction unit 129 using the encoding unit 122 and the decoded image 128 stored in the frame memory 107 .

[0040] Step S103

The subtraction unit 102 generates a differential block 123, which is the difference between the encoding block 122 and the prediction block 129.

[0041] Step S104

The conversion module 103 converts the differential block 123 into frequency coefficients 124 .

[0042] Step S105

Variable length encoding module 104 performs variable length encoding on frequency coefficients 124 to generate coded stream 125.

[0043] Step S106

An inverse conversion module 105 converts the frequency coefficients 124 into pixel data to reconstruct the differential block 126.

[0044] Step S107

The summing unit 106 sums the reconstructed differential block 126 and the prediction block 129 together to generate a decoded block 127 and stores, as a decoded image 128, the generated decoded block 127 in the frame memory 107.

[0045] Step S108

The processing from step S102 to step S107 is repeated until all encoding blocks in the target image to be encoded have been encoded.

[0046] The variable word length encoding module 104 is described in more detail below.

[0047] Structure of variable word length encoding module 104

Fig. 3 is a block diagram showing the structure of the variable word length encoding unit 104.

[0048] In the present embodiment, the variable word length encoding module 104 encodes the target frequency coefficient 124 (hereinafter referred to simply as "coefficient") based on five parameters: significant_flag, greater1_flag, greater2_flag, remainder, and sign_flag.

[0049] significant_flag is a flag indicating whether the corresponding coefficient is 0. If significant_flag is 0, it means the coefficient is 0. If significant_flag is 1, it means the coefficient is not 0.

[0050] greater1_flag only exists when significant_flag is 1 (in other words, the coefficient is not 0). greater1_flag indicates whether the absolute value of the coefficient is 2 or greater (in other words, greater than 1). If greater1_flag is 0, it means the absolute value is 1. If greater1_flag is 1, it means the absolute value is 2 or more.

[0051] greater2_flag only exists when greater1_flag has a value of 1 (in other words, the coefficient has an absolute value of 2 or greater). greater2_flag indicates whether the absolute value of the coefficient is 3 or greater (in other words, greater than 2). If greater2_flag is 0, it means the absolute value is 2. If greater2_flag is 1, it means the absolute value is 3 or more.

[0052] The remainder only exists when greater2_flag is 1 (in other words, the absolute value is 3 or greater). The remainder indicates the value obtained by subtracting 3 from the absolute value.

[0053] sign_flag only exists when significant_flag is 1 (in other words, the coefficient is not 0). sign_flag indicates whether the coefficient has a negative value (in other words, whether the coefficient value is negative or positive). If sign_flag is 0, it means the coefficient is positive. If sign_flag is 1, it means the coefficient is negative.

[0054] As shown in FIG. 3, the variable word length encoding unit 104 includes a coefficient blocking unit 131, a significant_flag encoding unit 132, a level encoding unit 133, and a sign_flag encoding unit 134.

[0055] The significant_flag encoding module 132 encodes the significant_flag. Level encoding module 133 encodes greater1_flag, greater2_flag, and the remainder. Sign_flag encoding unit 134 encodes sign_flag.

[0056] Processing (Variable Length Coding)

Next, with reference to FIG. 4, a variable word length encoding flow is described.

[0057] Step S121

The coefficient block division unit 131 divides the target encoding block (frequency coefficients 124) into coefficient blocks each having 4 pixels x 4 pixels (hereinafter also referred to as “4x4 coefficient blocks”). More specifically, when the target encoding block has a size of 32x32, the coefficient block division unit 131 divides it horizontally into 8 parts and vertically into 8 parts. When the encoding target block has a size of 4x4, the coefficient block division unit 131 does not divide such a block. Subsequent processing from step S122 to step S124 is performed for each of the coefficient blocks. The coefficient blocks are processed sequentially in order from the coefficient block having a higher component to the coefficient block having a lower component.

[0058] Step S122

The significant_flag encoding unit 132 encodes the significant_flag of each of the coefficients included in the target coefficient block.

[0059] Step S123

The level encoding unit 133 encodes greater1_flag, greater2_flag, and the remainder of each of the coefficients included in the target block of coefficients.

[0060] Step S124

The sign_flag encoding unit 134 encodes the sign_flag of each of the coefficients included in the target coefficient block.

[0061] Step S125

The processing from step S122 to step S124 is repeated until all coefficient blocks in the encoding target block have been processed.

[0062] Below is a more detailed description of the level encoding unit 133.

[0063] Layer coding module 133 structure

Fig. 5 is a block diagram showing the structure of the layer encoding unit 133. As shown in FIG. 5, the level encoding module 133 includes a greater1_flag setting module 141, a greater2_flag setting module 142, a remainder setting module 143, a binarization parameter update module 144, a context set selection module 145, a greater1_flag memory 146, a greater1_flag context selection module 147, a greater1_flag context memory 148 , greater2_flag context selection module 149, greater2_flag context memory 150, arithmetic encoding module 151, and remainder binarization module 152. The remainder binarization module 152 includes a prefix-suffix determination module 153, a suffix bin output module 154, and a prefix bin output module 155.

[0064] Level encoding module 133 first selects appropriate context sets to be used in arithmetic encoding over greater1_flag and greater2_flag. Then, the level encoding unit 133 encodes greater1_flag, greater2_flag, and the remainder sequentially in order. Before arithmetic encoding, level encoding module 133 converts the remainder of the multi-valued signal into a binarized signal (bin). In this case, the level encoding module 133 adaptively changes the length of the binarized signal (bin) based on the binarization parameter (cParam). Layer encoding module 133 also uses this binarization parameter to adaptively select context sets.

[0065] Processing (Level Coding)

Next, with reference to FIG. 6-9, the encoding flow performed by the layer encoding unit 133 is described in more detail. Fig. 6 is a flowchart of encoding performed by the layer encoding unit 133.

[0066] Step S141

Context set selection module 145 sets the appropriate numbers of context sets to be used in the arithmetic encoding above greater1_flag and greater2_flag. This stage will be described in more detail below.

[0067] Step S142

The binarization parameter update module 144 initializes the binarization parameter (cParam) to 0. At this point, cParam is set to 0 at the beginning of the target 4x4 block of coefficients. It should be noted that steps S141 and S142 are performed once for the target coefficient block. However, the following steps from step S143 are performed for each of the coefficients.

[0068] Step S143

The level encoding unit 133 encodes the greater1_flag of each of the coefficients included in the target coefficient block. Fig. 7 is a flowchart explaining this step in more detail.

[0069] Step S151

The greater1_flag setting unit 141 determines whether the absolute value of the target coefficient (the coefficient to be processed) is 1 or more. If the absolute value is 1 or more, then greater1_flag is encoded in steps S152 to S154. If the absolute value is 0, greater1_flag is not encoded.

[0070] Step S152

If the absolute value of the target coefficient is 2 or more, then the greater1_flag setting unit 141 sets greater1_flag to 1. If the absolute value is 1, then the greater1_flag setting unit 141 sets greater1_flag to 0.

[0071] Step S153

The greater1_flag context selection unit 147 sets the number of contexts based on the number of context sets selected in step S141. This stage will be described in more detail below.

[0072] Step S154

The arithmetic encoding unit 151 loads the context to be used (use context) from the greater1_flag context memory 148 based on the number of contexts selected in step S153, and then performs arithmetic encoding on the greater1_flag using the context. In addition, arithmetic encoding module 151 stores the context that was updated in arithmetic encoding back to the same location in greater1_flag context memory 148.

[0073] Step S155

The processing from step S151 to step S155 is repeated until all coefficients of the target 4x4 coefficient block have been processed.

[0074] As a result, greater1_flag of the corresponding coefficients are encoded in the target coefficient block.

[0075] Step S144

The level encoding unit 133 encodes the greater2_flag of each of the coefficients included in the target coefficient block. Fig. 8 is a flowchart explaining this step in more detail.

[0076] Step S161

The greater2_flag setting unit 142 determines whether the absolute value of the target coefficient is 2 or greater. If the absolute value is 2 or more, then greater2_flag is encoded in steps S162 to S165. If the absolute value is 1 or less, greater2_flag is not encoded.

[0077] Step S162

If the absolute value of the target coefficient is 3 or more, then greater2_flag setting unit 142 sets greater2_flag to 1. If the absolute value is 2, then greater2_flag setting unit 142 sets greater2_flag to 0.

[0078] Step S163

The greater2_flag context selection unit 149 sets the number of context sets selected in step S141 as the number of contexts. Unlike greater1_flag, the number of context sets for greater2_flag is directly set to the number of contexts. In other words, a single context set for greater2_flag includes only one context.

[0079] Step S164

The arithmetic encoding unit 151 loads the context to be used (use context) from the greater2_flag context memory 150 based on the number of contexts selected in step S150, and then performs arithmetic encoding on the greater2_flag using the context. In addition, the arithmetic encoding module 151 stores the context that was updated in the arithmetic encoding back in the same location in the greater2_flag context memory 150.

[0080] Step S165

The processing from step S161 to step S164 is repeated until all coefficients of the target 4x4 coefficient block have been processed.

[0081] As a result, greater2_flag of the corresponding coefficients in the target coefficient block are encoded.

[0082] Step S145

The level encoding unit 133 encodes the remainder in each of the coefficients in the target coefficient block. Fig. 9 is a flowchart explaining this step in more detail.

[0083] Step S171

The remainder setting unit 143 determines whether the absolute value of the target ratio is 3 or greater. If the absolute value is 3 or more, the remainder is encoded in steps S172 to S175. If the absolute value is 2 or less, the remainder is not encoded.

[0084] Step S172

The remainder setting unit 143 calculates a numeric value by subtracting 3 from the absolute value of the target coefficient and sets the numeric value as the remainder (multi-value signal).

[0085] Step S173

The remainder binarization unit 152 converts the multi-valued remainder signal into a binarized signal. This stage will be described in more detail below.

[0086] Step S174

Arithmetic encoding unit 151 performs arithmetic encoding on the remainder. Unlike greater1_flag and greater2_flag, remainder is applied with bypass arithmetic encoding, which does not use any context.

[0087] Step S175

The binarization parameter update module 144 initializes the binarization parameter (cParam). This stage will be described in more detail below.

[0088] Step S176

The processing from step S171 to step S175 is repeated until all coefficients in the target coefficient block have been processed.

[0089] Processing (Select Context Sets)

Next, with reference to FIG. 10, the selection of context sets (S141 in FIG. 6) will be described in more detail.

[0090] Steps S181 to S183

The context set selector 145 determines whether the target coefficient block has the lowest frequency in the target encoding block. If the target coefficient block has the lowest frequency, then the number of context sets is set to 0. Otherwise, the number of context sets is set to 2. More specifically, if the target coefficient block is located in the upper left corner of the target encoding block, then the context set selecting unit 145 sets the number of context sets to 0. Otherwise, the number of context sets is set to 2.

[0091] Steps S184 and S185

If the updated binarization parameter (cParam) of the coefficient block (immediately preceding the coefficient block) that was processed immediately before the target coefficient block is greater than 0, then the context set selection module 145 increases the number of context sets by 1. Thus, if cParam has been updated at least once for the immediately preceding block of coefficients, the context set is changed.

[0092] As described above, in steps S181 to S185, the number of context sets is in the range of 0 to 3. In other words, there are four kinds of context sets, from which the context set selection unit 145 selects one.

[0093] Processing (Context selection greater1_Flag)

Next, with reference to FIG. 11, context selection greater1_flag (S153 in FIG. 7) is described in more detail.

[0094] Step S191

From the number of coefficients that have been processed in the target coefficient block, the greater1_flag context selector 147 counts the number (G1NUM) of greater1_flag with a value of 1. In other words, the greater1_flag context selector 147 counts the number of coefficients having an absolute value of 2 or greater. Here, the greater1_flag of the processed coefficients are stored in the greater1_flag memory 146, so that the greater1_flag context selector 147 receives these greater1_flags from the greater1_flag memory 146.

[0095] Steps S192 to S193

If G1NUM calculated in step S191 is greater than or equal to 1, then the greater1_flag context selection unit 147 sets the context offset to 3. If G1NUM is 0, then processing proceeds to step S194. In other words, when there is at least one processed coefficient having an absolute value of 2 or greater in the target block of coefficients, the context offset is set to 3.

[0096] Step S194

From the processed coefficients in the target coefficient block, greater1_flag context selection module 147 counts the number (G1NUM2) of coefficients having greater1_flag. In other words, greater1_flag context selection module 147 counts the number of coefficients having an absolute value of 1 or greater. Here, the greater1_flag context selector 147 obtains the greater1_flag of processed coefficients from the geater1_flag memory 146.

[0097] Steps S195 to S197

If G1NUM2 calculated in step S194 is greater than 2, then the context selection unit 147 greater1_flag sets the context offset to 2. Otherwise, the context selection unit 147 greater1_flag sets the context offset to the value G1NUM2.

[0098] Thus, in steps S191 to S197, the context offset is set to a value in the range of 0 to 3.

[0099] Step S198

The greater1_flag context selector 147 sets the number of contexts to "(number of context sets ×4)+context offset" and then the resulting number of contexts into the greater1_flag context memory 148. Since the number of context sets is from 0 to 3, and the context offset is from 0 to 3, as shown in FIG. 12, the number of contexts ultimately has a value ranging from 0 to 15. In other words, there are four kinds of context sets, and there are four contexts in each of the context sets for greater1_flag. The context set selection module 145 selects one kind of context sets from four kinds, and the greater1_flag context selection module 147 selects one of the four contexts in the selected context set.

[0100] Processing (Binarization of remainder)

Next, with reference to FIG. 13-16, the binarization of the residue (S173 in FIG. 9) is described in more detail. Thus, a binarized signal (bin) includes a prefix and a suffix. The way the prefix and suffix are determined changes according to the binarization parameter (cParam).

[0101] Step S201

The prefix-suffix determination unit 153 determines the prefix and the suffix. Fig. 14 is a flow diagram explaining this step in more detail.

[0102] Steps S211 to S222

Prefix-suffix determination module 153 uses cParam in determining the prefix and suffix. Determination processing is mainly classified into the following two methods. If the remainder is small, the prefix and suffix are determined in steps S213 and S214. On the other hand, if the remainder is large, the prefix and suffix are determined in steps S215 to S222. cParam is used as a threshold value based on which it is determined whether the remainder is small or large. As cParam is larger, the threshold value is larger. cParam is associated with the bin length of the prefix and suffix.

[0103] FIG. 15A-15C show a prefix and suffix bin in association with a cParam and a remainder. As shown in FIG. 15A-15C, if the remainder is small, a smaller cParam results in a shorter bin length, and if the remainder is larger, a larger cParam results in a shorter bin length. In other words, if the remainder were small, a smaller cParam would be more likely to improve encoding efficiency. On the other hand, if the remainder were large, a larger cParam would be more likely to improve encoding efficiency.

[0104] Step S202

The prefix bin output module 155 provides the prefix bin to the arithmetic encoding module 151 . Fig. 16 is a flowchart explaining this step in more detail.

[0105] Steps S231 to S235

The prefix bin output unit 155 first outputs "1" values in the same number as the prefix value, and finally outputs the "0" value.

[0106] Step S203

Suffix bin output module 154 provides a suffix bin to arithmetic encoding module 151 . Fig. 17 is a flowchart explaining this step in more detail.

[0107] Steps S241 to S245

The suffix bin output module 154 converts the suffix value into binary-encoded form and outputs the resulting values in binary-encoded form sequentially in order from the highest order bit. Suffix bin output module 154 outputs tmpLen number of bin(s). However, if the number of bits in the suffix value is less than tmpLen, the highest order bit is output as 0.

[0108] Processing (Binarization parameter update)

Next, with reference to FIG. 18, the binarization parameter update (S175 in FIG. 9) is described in more detail.

[0109] Step S251

If the binarization parameter (cParam) is less than 4, the binarization parameter update unit 144 updates cParam in steps S252 to S254. If cParam is 4 or greater, then the binarization parameter update unit 144 does not update cParam and ends processing.

[0110] Steps S252 to S254

If the absolute value of the target coefficient is greater than the threshold value, then the binarization parameter update module 144 increases cParam by 1. The threshold value is determined using the mathematical formula "3x(1<< cParam)". "<<" means shift left. When cParam has a larger value, the threshold value has a larger value. As described above, cParam is initialized to 0 at the start of processing for the target coefficient block. Each time a coefficient greater than the threshold value occurs due to the processing of the binarization parameter update module 144, cParam is incremented by 1 until cParam reaches a maximum value of 4.

[0111] Effects

As described above, in the image encoding apparatus 100 according to the present embodiment, if there is a coefficient having an absolute value greater than the threshold value in the target processed coefficient block, the context for greater1_flag and the context for greater2_flag are changed. As a result, encoding efficiency can be improved with a small amount of processing.

[0112] More specifically, the binarization parameter (cParam) is incremented by 1 each time the absolute value of the coefficient is found to be greater than the threshold value. If, at the beginning of the target coefficient block, the value of cParam that was updated in the processing of the immediately preceding coefficient block is greater than 0, in other words, if the immediately preceding coefficient block includes at least one coefficient greater than the threshold value, then the number of context sets changes. More specifically, if there is a coefficient having a large value in the immediately preceding coefficient block (a coefficient block with a higher frequency than the frequency of the target coefficient block), then the image encoding apparatus 100 determines that there is a high probability that the target coefficient block also includes itself a coefficient of great importance. Therefore, a context is selected that is suitable for the coefficient that is of great significance.

[0113] In the present HVEC standard (see Non-Patent Document 1), the number of processed coefficients having an absolute value of 2 or more is counted, and the context for the target coefficient is determined according to the number. However, the processed odds are not limited to the odds included in the immediately preceding odds block. These may be a plurality of coefficients included in a plurality of coefficient blocks that are located closer to the high frequency side than the target coefficient block.

[0114] On the other hand, in the method according to the present embodiment, it is not determined whether or not a coefficient exceeding a threshold value exists in the immediately preceding coefficient block. Thus, the image encoding device 100 according to the present embodiment need neither count the count nor have a resistor for storing count values.

[0115] In addition, the image encoding method according to the present embodiment was applied to test the software of the present HEVC standard for experiments. Fig. 19 shows the experimental results compared with the state before use. The experimental conditions are based on the general experimental conditions of the HEVC standard group. A higher numerical value indicates lower coding efficiency. A negative value indicates an improvement in encoding efficiency. As shown in FIG. 19, each of the values ranges from -0.01% to 0.03%. This means that coding efficiency is virtually unaffected even if coefficient counting is eliminated.

[0116] In addition, by combining information to be used in selecting contexts and information to be used in encoding a remainder, the size of the circuit is reduced. More specifically, cParam, which is incremented whenever an absolute coefficient value is found that is greater than the threshold value, is used to change the remainder binarization parameter and to select a greater1_flag context and a greater2_flag context so that one structure can perform different functions. Thus, compared with the existing HEVC standard, the image encoding method according to the present embodiment can eliminate coefficient counting, and deterioration in encoding efficiency is suppressed without adding a new structure.

[0117] It should be noted that it was described above that the set of contexts is changed based on the binarization parameter (cParam) that was updated for the immediately preceding coefficient block. However, the image encoding apparatus 100 may make changes based on a binarization parameter that has been updated for a coefficient block different from the immediately preceding coefficient block. It is possible, for example, that image encoder 100 may change the set of contexts when cParam exceeds 0 in processing for any coefficient block being processed. Coefficient blocks are processed sequentially in order from the highest frequency block. Thus, if the cParam of any one of the coefficient blocks is greater than 0, in other words, if a coefficient appears that has an absolute value that is greater than or equal to the threshold value, there is a high probability that all subsequent coefficient blocks have larger coefficient values. In such a case, it is possible to increase encoding efficiency by selecting a context set to be used for the above capability.

[0118] In addition, the image encoding apparatus 100 initializes the binarization parameter (cParam) to 0 at the start of processing for the target coefficient block. However, it is also possible to support the use of a cParam updated for the immediately preceding coefficient block without initializing the cParam to 0. This allows the state of the high-frequency coefficient block to continue to be used. As a result, encoding efficiency sometimes increases depending on the input image.

[0119] It should also be noted that it has been described above that the image encoding device 100 changes the context set when cParam is greater than 0. However, the present embodiment is not limited to the above. It is also possible to change the context set if, for example, cParam is greater than 1 or cParam is greater than 2. Depending on the input image, it is sometimes possible to improve encoding efficiency by changing the context set when cParam is greater than 1.

[0120] It should also be noted that it has been described above that four kinds of context sets are used, but the present embodiment is not limited to the above. There may be four or more kinds of context sets, or four or fewer kinds of context sets. For example, although it is described above that the image encoding apparatus 100 increases the number of context sets by 1 when cParam is greater than 0, when the number of context sets is 0 or 2. However, it is also possible that the number of context sets is increased by 1 only when the number There are 2 context sets.

[0121] It should also be noted that it has been described above that the image encoding apparatus 100 changes the context set when the target block of coefficients has the lowest frequency, but the change may be unnecessary. It is also possible that the image encoding apparatus 100 increases the number of context sets by 1 when cParam is greater than 0, and further increases the resulting number of context sets by 1 when cParam is greater than 1. Depending on the input image, the encoding efficiency can be increased , when the number of kinds of context sets increases. On the other hand, depending on the input image, when the number of context sets is reduced, the kinds of contexts can be reduced, thereby reducing the size of the circuit for processing selection and reducing the memory for storing contexts.

[0122] It should also be noted that in FIG. 11, it has been shown that the image encoding apparatus 100 sets the greater1_flag context based on G1NUM and G1NUM2, but the present embodiment is not limited to the above. For example, the image encoding apparatus 100 may set the context based on the position of the target coefficient in the target coefficient block or based on G1NUM or G1NUM2.

[0123] It should also be noted that it was described above that when selecting a context for greater2_flag, the image encoding apparatus 100 sets the number of context sets as the actual context number, but the present embodiment is not limited to the above. The image encoding device 100 may select the context for greater2_flag by the same method as used for greater1_flag.

[0124] It should also be noted that it was described above that greater1_flag exists when the absolute value of the target coefficient is 1 or greater, and that greater2_flag exists when the absolute value of the target coefficient is 2 or greater. However, it is also possible to limit the corresponding maximum number of flags in the coefficient block in the same way as in the present HEVC standard (Non-Patent Document 1). It is possible that there are at most eight greater1_flags and at most one greater2_flags.

[0125] It should also be noted that it has been described above that in the binarization of the remainder of the image encoding apparatus 100, the prefix and the suffix form a bin. However, the binarized remainder signal can be any signal as long as the bin length can be changed based on the cParam. For example, the binarized remainder signal may have only a prefix or may be expressed using the Golomb exponent method from the H.264 moving image coding standard.

[0126] It should also be noted that it was described above that the last bin of the prefix is set to 0, but the last 0 may be removed when the prefix is at its maximum. If the prefix is not maximal, a "0" must be inserted to qualify the bin at the prefix/suffix boundary. However, if the image decoder knows the maximum coefficient value, then the maximum prefix value is obvious. Thus, if the prefix is maximal, the image decoding device can determine the boundary between the prefix and the suffix without the "0". Eliminating the insertion of "0" in the maximum prefix case can significantly reduce the bin length when there are a large number of maximum prefix coefficients (in other words, when the encoding bit rate is high). As a result, coding efficiency is improved.

[0127] It has been described above that the threshold value used in updating the binarization parameter (cParam) is set according to the mathematical formula "3*(1<<cParam)", but the present embodiment is not limited to the above. For example, the image encoding device may use a different numeric value, such as "4" instead of "3" in a mathematical formula, or use a different mathematical formula.

[0128] It should also be noted that it was described above that the image encoding device uses context in the arithmetic encoding on the greater1_flag and greater2_flag for all coefficients, but it is also possible that bypass arithmetic encoding that does not use context could be performed on the greater1_flag or greater2_flag portions coefficients in the same way as for the remainder. For example, if the greater1_flag of the immediately preceding coefficient is 1, the image encoder may perform bypass arithmetic encoding on the greater1_flag of the target coefficient. It is also possible that the image encoder performs bypass arithmetic encoding on greater1_flag at the beginning of the block of target coefficients. Using bypass arithmetic encoding can eliminate context loading and updating. Additionally, using bypass arithmetic encoding can eliminate the dependency relationship with other syntax using the same context. As a result, processing can be performed at a higher speed.

[0129] It should also be noted that it has been described above that a coding block has a size of a maximum of 32 x 32 pixels and a minimum of 4 x 4 pixels, but the size of the coding blocks is not limited to the above. It is also possible that the coding blocks have a fixed size.

[0130] It should also be noted that it was described above that the encoding block is divided into 4x4 coefficient blocks, but it is not necessary to divide the encoding block into coefficient blocks each having 4 pixels x 4 pixels. For example, one can also divide the encoding block into coefficient blocks, each of which has 8 pixels × 8 pixels. The encoding block may be divided into rectangular coefficient blocks, each of which is, for example, 8 pixels x 4 pixels.

[0131] Moreover, the processing according to the present embodiment may be implemented in software. This software may be distributed through downloads or the like. Or the software, for distribution purposes, may be recorded on a recording medium such as a CD-ROM. The above applies to other embodiments in this specification.

[0132] Embodiment 2

Embodiment 2 describes an image decoding apparatus that decodes a coded bit stream generated by the image encoding apparatus according to Embodiment 1.

[0133] General structure

Fig. 20 is a block diagram showing the structure of an image decoding apparatus 200 according to the present embodiment.

[0134] The image decoding apparatus 200 shown in FIG. 20 decodes the code stream 221 to generate a decoded image 225. Here, the code stream 221 corresponds to, for example, the code stream 125 generated by the above-described image encoding device 100. The image decoding apparatus 200 includes a variable word length decoding unit 201, an inverse transform unit 202, a summing unit 203, a decoded block combining unit 204, and a frame memory 205.

[0135] Processing (General)

Next, with reference to FIG. 21, a general decoding flow is described.

[0136] Step S301

The variable length decoding unit 201 performs variable length decoding on the encoded stream 221 to generate frequency coefficients 222, and provides the frequency coefficients 222 to the inverse transform unit 202.

[0137] Step S302

The inverse conversion module 202 converts the coefficients 222 of the frequency 222 into pixel data to generate a differential block 223 .

[0138] Step S303

Adder 203 adds decoded image 226 stored in frame memory 205 with differential block 223, thereby generating decoded block 224.

[0139] Step S304

The processing from step S301 to step 303 is repeated until all blocks in the target image have been decoded.

[0140] Step S305

The decoded block combiner 204 combines a plurality of decoded blocks 224 together to generate a decoded image 225 and stores, as the decoded image 226, the decoded image 225 in a frame memory 205.

[0141] Below is a more detailed description of the variable word length decoding module 201.

[0142] Structure of variable word length decoding module 201

Fig. 22 is a block diagram showing the structure of the variable word length decoding unit 201. In the present embodiment, similar to Embodiment 1, the frequency coefficient 222 (hereinafter also referred to simply as "coefficient") is expressed based on five parameters: significant_flag, greater1_flag, greater2_flag, remainder, and sign_flag. The corresponding parameters indicate the same as described in Embodiment 1, so the explanation will not be given again.

[0143] As shown in FIG. 22, the variable word length decoding unit 201 includes a significant_flag decoding unit 231, a level decoding unit 232, a sign_flag 233 decoding unit 233, and a coefficient decoding unit 234.

[0144] Processing (Variable Length Decoding)

Next, with reference to FIG. 23, a variable word length decoding flow is described.

[0145] Step S321

The significant_flag decoding module 231 decodes the significant_flag of each of the coefficients in the target coefficient block and provides the resulting significant_flag to the level decoding module 232, the sign_flag decoding module 234, and the coefficient decoding module 234.

[0146] Step S322

Level decoding module 232 decodes greater1_flag, greater2_flag, and the remainder of each of the coefficients in the target coefficient block and provides the resulting greater1_flag, greater2_flag, and remainder to coefficient decoding module 234. Here, level decoding module 232 decodes greater1_flag, greater2_flag, and the remainder only when significant_flag is 1.

[0147] Step S323

The sign_flag decoding unit 233 decodes the sign_flag of each of the coefficients in the target coefficient block and provides the resulting sign_flag to the coefficient decoding unit 234 . Here, sign_flag decoding unit 233 decodes sign_flag only when significant_flag is 1.

[0148] Step S324

The coefficient decoding unit 234 decodes the target coefficient based on significant_flag, greater1_flag, greater2_flag, remainder, and sign_flag. The corresponding parameters indicate the same as those described in Embodiment 1, so that the coefficient decoding unit 234 decodes the target coefficient in accordance with the indication of the parameters.

[0149] Below is a more detailed description of the level decoding module 232.

[0150] Layer decoding module 232 structure

Fig. 24 is a block diagram showing the structure of the layer decoding unit 232. As shown in FIG. 24, the level decoding module 232 includes a context set selection module 241, a greater1_flag context selection module 242, a greater1_flag memory 243, a greater1_flag context memory 244, a greater2_flag context selection module 245, a greater2_flag context memory 246, an arithmetic decoding module 247, a parameter update module 248 binarization and modulus 249 of a multivalued remainder. The multi-valued remainder unit 249 includes a prefix decoding unit 250, a suffix decoding unit 251, and a prefix-suffix combining unit 252.

[0151] Processing (Level Decoding)

Next, with reference to FIG. 25-28, level decoding is described in more detail. Fig. 25 is a flowchart of decoding performed by the layer decoding unit 232.

[0152] Step S341

The context set selector 241 sets the respective numbers of context sets to be used in the arithmetic decoding over greater1_flag and the arithmetic decoding over greater2_flag. The setting method is the same as when selecting context sets (FIG. 10) according to Embodiment 1.

[0153] Step S342

The binarization parameter update module 248 initializes the binarization parameter (cParam) to 0. At this point, cParam is set to 0 at the beginning of the target coefficient block. It should be noted that steps S341 and S342 are performed once for the target coefficient block, while the following steps from step S343 are performed for each of the coefficients in the target coefficient block.

[0154] Step S343

The level decoding unit 232 decodes the greater1_flag of each of the coefficients in the target coefficient block. Fig. 26 is a flow diagram explaining this step in more detail.

[0155] Step S351

The level decoding unit 232 determines whether the significant_flag of the target coefficient is 1. If the significant_flag is 1, the level decoding unit 232 decodes the greater1_flag in steps S352 and S353. On the other hand, if significant_flag is 0, level decoding unit 232 does not decode greater1_flag.

[0156] Step S352

The greater1_flag context selection unit 242 sets the number of contexts based on the number of context sets selected in step S341. The setting method is the same as when selecting the greater1_flag context (FIG. 11) according to Embodiment 1.

[0157] Step S353

The arithmetic decoding unit 247 loads the context to be used (using context) from the greater1_flag context memory 244 according to the number of contexts selected in step S352, and performs arithmetic decoding on the greater1_flag by using the context. In addition, the arithmetic decoding module 247 stores the context that was updated in the arithmetic decoding back in the same location in the greater1_flag context memory 244.

[0158] Step S354

The processing from step S351 to step S353 is repeated until all coefficients in the target coefficient block have been processed.

[0159] As a result, greater1_flag of the corresponding coefficients in the target coefficient block is decoded.

[0160] Step S344

The level decoding unit 232 decodes the greater2_flag of each of the coefficients in the target coefficient block. Fig. 27 is a flow diagram explaining this step in more detail.

[0161] Step S361

The level decoding unit 232 determines whether the greater1_flag of the target coefficient is equal to 1. If the greater1_flag is 1, the decoding unit 232 decodes the greater2_flag in steps S362 and S363. On the other hand, if greater1_flag is 0, the level decoding unit 232 does not decode greater2_flag. It should be noted that if the significant_flag of the target coefficient is 0, the level decoding unit 232 does not decode either the greater1_flag or the greater2_flag.

[0162] Step S362

The greater2_flag context selection unit 245 sets the number of context sets selected in step S341 as the number of contexts. Unlike greater1_flag, the number of context sets for greater2_flag is directly set to the number of contexts. In other words, a single context set for greater2_flag includes only one context.

[0163] Step S363

The arithmetic decoding unit 247 loads the context to be used (use context) from the greater2_flag context memory 246 according to the number of contexts selected in step S362, and performs greater2_flag arithmetic decoding by using the context. In addition, the arithmetic decoding module 247 stores the context that was updated in the arithmetic decoding back in the same location in the greater2_flag context memory 246.

[0164] Step S364

The processing from step S361 to step S363 is repeated until all coefficients in the target coefficient block have been processed.

[0165] As a result, greater2_flag of the corresponding coefficients in the target coefficient block are decoded.

[0166] Step S345

The layer decoding unit 232 decodes the remainder of each of the coefficients in the target coefficient block. Fig. 28 is a flowchart explaining this step in more detail.

[0167] Step S371

The level decoding unit 232 determines whether the greater2_flag of the target coefficient is 1. If the greater2_flag is 1, the level decoding unit 232 decodes the remainder in steps S372 and S373. On the other hand, if greater2_flag is 0, the level decoding unit 232 does not decode the remainder. It should be noted that if the significant_flag of the target coefficient is 0 or the greater1_flag of the target coefficient is 0, the level decoding unit 232 does not decode either the greater2_flag or the remainder.

[0168] Step S372

Arithmetic decoding unit 247 performs remainder arithmetic decoding. Unlike greater1_flag and greater2_flag, remainder is applied with bypass arithmetic encoding, which does not use any context.

[0169] Step S373

The multi-valued remainder module 249 converts the multi-valued remainder signal into a binarized signal. This stage will be described in more detail below.

[0170] Step S374

The binarization parameter update module 248 updates the binarization parameter (cParam). The updating method is the same as updating the binarization parameter (FIG. 18) according to Embodiment 1.

[0171] Step S375

The processing from step S371 to step S374 is repeated until all coefficients in the target coefficient block have been processed.

[0172] Processing (Convert remainder to multi-valued signal)

Next, with reference to FIG. 29-31, converting the remainder to a multi-valued signal (S373 in FIG. 28) is described in more detail.

[0173] Step S401

The prefix decoding unit 250 decodes the prefix. Fig. 30 is a flowchart explaining this step in more detail.

[0174] Steps S411 to S415

The prefix decoding unit 250 determines the prefix. Finally, the prefix decoding module 250 receives codes of 1 bit each sequentially from the arithmetic decoding module 247 until a "0" is found. The number of continuous "1s" is set to the prefix value.

[0175] Step S402

The suffix decoding unit 251 decodes the suffix.

[0176] Step S403

The prefix-suffix combiner 252 combines the prefix and the suffix to generate a multi-valued remainder signal.

[0177] FIG. 31 is a flowchart explaining steps S402 and S403 in more detail.

[0178] Steps S421 to S433

The suffix decoding unit 251 determines the suffix. Finally, if the prefix is less than 8, then the suffix decoding unit 251 sets the value of cParam as the bin length and generates the suffix by ordering the arithmetic decoding results of the bin length in order from higher order. On the other hand, if the prefix is 8 or greater, then the suffix decoding unit 251 calculates the bin length from the prefix and cParam and generates a suffix by ordering the arithmetic decoding results of the bin length in order from higher order.

[0179] Steps S441 to S442

Prefix-suffix combiner 252 calculates the remainder based on the prefix, suffix, and cParam.

[0180] Effects

As described above, the image decoding apparatus according to Embodiment 2 can provide the same effects as described in Embodiment 1.

[0181] FIG. 32A is a flowchart of image encoding performed by the above-described image encoding apparatus 100.

[0182] As described above, the image encoding apparatus 100 performs arithmetic encoding on the first flag (greater1_flag) indicating whether the absolute value of the target coefficient included in the target coefficient block is greater than 1 (S701). Next, the image encoding apparatus 100 performs arithmetic encoding on a second flag (greater2_flag) indicating whether the absolute value is greater than 2 (S702).

[0183] FIG. 32B is a flowchart of steps S701 and S702.

[0184] In steps S701 and S702, the image encoding apparatus 100 determines whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than the threshold value (S711), and based on the determination result, selects corresponding contexts to be used in arithmetic encoding over the first flag and arithmetic coding over the second flag (S712). In other words, the image encoding apparatus 100 performs the above determination with reference only to the coefficients included in the immediately preceding coefficient block and not the coefficients included in coefficient blocks other than the immediately preceding coefficient block. In other words, the image encoding apparatus 100 performs arithmetic encoding on the first flag and the second flag using contexts corresponding to the determination result among a plurality of contexts.

[0185] The immediately preceding coefficient block refers to a block of coefficients that were processed immediately before the target coefficient block.

[0186] More specifically, if the absolute value of the target coefficient is greater than the threshold value, then the image encoding device 100 increments the binarization parameter (cParam) (S253 and S254 in FIG. 18). Then, if the cParam of the immediately preceding coefficient block is greater than 0, the image encoding apparatus 100 changes the context (in other words, increments the number of context sets) (S184 and S185 in FIG. 10).

[0187] In other words, based on the variable generated in arithmetic encoding over the immediately preceding coefficient block, the image encoding apparatus 100 determines whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than a threshold value.

[0188] More specifically, if the absolute value of the target coefficient is greater than 2, the image encoding device 100 binarizes the remainder, that is, the value obtained by subtracting 3 from the absolute value, according to the lookup table designated by the binarization parameter (cParam), among the set conversion tables. This binarization parameter is determined by whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. The above variable is this binarization parameter.

[0189] It was described above that the threshold value for the absolute values of the coefficients referred to when changing the context is determined based on the binarization parameter (S252 in FIG. 18), but the threshold value may be a predefined value (“1”) as described in relation to conventional technology. In other words, the image encoding apparatus 100 can determine whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than 1 (2 or more), and based on the determination result, change the context to be used to another. As described above, in the traditional method, the number of coefficients having an absolute value greater than a threshold value is counted. This processing is different from the processing of determining whether there is a coefficient having an absolute value greater than a threshold value as described in the present embodiment.

[0190] In this case, as in the previous case, the image encoding device 100 can determine whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than 1 based on a variable generated in the arithmetic encoding above the immediately preceding block coefficients For example, this variable is greater1_flag. In other words, the image encoding apparatus 100 may change the context according to whether the immediately preceding block of coefficients includes a coefficient that has greater1_flag having a value equal to 1.

[0191] FIG. 33A is a flowchart of image decoding performed by the above-described image decoding apparatus 200.

[0192] As shown in FIG. 33A, the image decoding apparatus 200 performs arithmetic decoding on a first flag (greater1_flag) that indicates whether the absolute value of the target coefficient included in the target coefficient block is greater than 1 (S751). The image decoding apparatus 200 performs arithmetic decoding on the second flag (greater2_flag) indicating whether the absolute value is greater than 2 (S752).

[0193] FIG. 33B is a flowchart of steps S751 and S752.

[0194] In steps S751 and S752, the image decoding apparatus 200 200 determines whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than the threshold value (S761), and based on the determination result, changes the corresponding context to be used in the arithmetic decoding on the first flag and the second flag (S762). In other words, the image decoding apparatus 200 performs arithmetic decoding on the first flag and the second flag by using corresponding contexts corresponding to the determination result among the plurality of contexts.

[0195] More specifically, if the absolute value of the target coefficient is greater than the threshold value, the image decoding apparatus 200 increments the binarization parameter (cParam) (S253 and S254 in FIG. 18). Then, if the cParam of the immediately preceding coefficient block is greater than 0, the image decoding apparatus 200 changes the context (in other words, increments the number of context sets) (S184 and S185 in FIG. 10).

[0196] In other words, based on the variable generated in arithmetic decoding over the immediately preceding coefficient block, the image decoding apparatus 200 determines whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than a threshold value.

[0197] More specifically, if the absolute value of the target coefficient is greater than 2, the image decoding apparatus 200 converts the remainder, which is obtained by subtracting 3 from the absolute value, into a multi-valued value in accordance with the lookup table designated by the binarization parameter among the plurality of lookup tables. This binarization parameter is determined by whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value. The above variable is this binarization parameter.

[0198] As above, the threshold value may be a predefined value (“1”). In this case, as in the previous case, the image decoding apparatus 200 can determine whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than 1 based on a variable generated in the arithmetic encoding above the immediately preceding coefficient block. For example, this variable is greater1_flag. In other words, the image decoding apparatus 200 may change the context according to whether the immediately preceding block of coefficients includes a coefficient that has greater1_flag having a value of 1.

[0199] Embodiment 3

Embodiment 3 describes a variation of Embodiment 1. The image encoding apparatus according to Embodiment 3 differs from the image encoding apparatus according to Embodiment 1 in that the layer encoding unit 133A operates differently from the layer encoding unit 133. Below is a description of the level encoding unit 133A.

[0200] Layer coding module 133A structure

Fig. 34 is a block diagram showing the structure of the level encoding unit 133A. The level encoding unit 133A includes not only the structure of the level encoding unit 133 according to Embodiment 1, but also the greater_flag switching unit 160. In addition, the functions of the remainder setting unit 143A and the context set selecting unit 145A are different from the functions of the remainder setting unit 143 and the context set selecting unit 145.

[0201] Processing (Level Coding)

Next, with reference to FIG. 35 level coding will be described in more detail. The layer coding according to Embodiment 1 is supplemented with new steps S501 to S503. In addition, steps S141 and S171 are different from steps S141 and S171. Steps identical to those in Embodiment 1 are not explained again below.

[0202] Step S141

Context set selector 145A sets the number of context sets to be used in the arithmetic encoding over greater1_flag and greater2_flag. This stage will be described in more detail below.

[0203] Step S501

The greater_flag switch module 160 determines whether the binarization parameter (cParam) that was updated in processing on the immediately preceding coefficient block is greater than 0. In the same manner as described in Embodiment 1, if the binarization parameter is 0, then greater1_flag and greater2_flag are encoded. On the other hand, if the binarization parameter is greater than 0, then greater1_flag and greater2_flag are not encoded.

[0204] Steps S502, S503 and S171

If the binarization parameter (cParam) that was updated for the immediately preceding coefficient block is greater than 0, then the residual setting unit 143A sets the base value of the remainder to 1. If the binarization parameter (cParam) is 0, then the residual setting unit 143A sets the base value the remainder is 3. Then, if the absolute value of the target coefficient is greater than or equal to the base value of the remainder, the level encoding unit 133A encodes the remainder in steps S172 to S175. On the other hand, if the absolute value is less than the base value of the remainder, then the level encoding unit 133A does not encode the remainder. If the remainder is encoded, then the level encoding unit 133A subtracts the base value of the remainder from the absolute value and sets the resulting value to be both the remainder and encoded. This is because greater1_flag and greater2_flag are sometimes not encoded depending on cParam, and the value that should be set to remainder is therefore changed. In embodiment 1, since greater1_flag and greater2_flag are present, whatever the cParam is, the base value of the remainder is fixed to "3".

[0205] Processing (Select Context Sets)

Next, with reference to FIG. 35, the selection of context sets (S141 in FIG. 34) is described in more detail.

[0206] Steps S511 to S513

Context set selection unit 145A determines whether the target coefficient block has the lowest frequency in the target encoding block. If the target coefficient block has the lowest frequency, then the context set selection unit 145A sets the number of context sets to 0. Otherwise, the context set selection unit 145A sets the number of context sets to 1. In other words, if the target coefficient block is located in the upper left corner target encoding block, then the context set selecting unit 145A sets the number of context sets to 0. Otherwise, the context set selecting unit 145A sets the number of context sets to 1.

[0207] Unlike Embodiment 1, context set selector 145A does not change the context set when cParam is greater than 0. This is because greater1_flag and greater2_flag are not encoded and context is not selected if cParam is greater than 0, as shown in the previously described layer coding flow.

[0208] Effects

As described above, the image encoding apparatus according to the present embodiment determines based on the binarization parameter that was updated for the immediately preceding coefficient block that greater1_flag and greater2_flag should not be encoded, and therefore the coefficient is encoded based on the remainder instead of greater1_flag and greater2_flag. That is, if there is at least one coefficient greater than a threshold value in the processing for the immediately preceding block of coefficients, the image encoding apparatus performs bypass arithmetic encoding that does not use any context, instead of arithmetic encoding that does use the context. Bypass arithmetic encoding does not require context loading and updating. In addition, bypass arithmetic encoding allows processing to begin without waiting for the context to be updated in the preliminary stage. Therefore, bypass arithmetic encoding can increase processing speed more than arithmetic encoding that uses context.

[0209] Additionally, if there is a high probability that the absolute value of the target coefficient is small, greater1_flag and greater2_flag are sometimes equal to 0, so encoding the remainder is not necessary. However, this situation rarely occurs when there is a high probability that the absolute value of the target ratio is large. Thus, the total bin value can often be reduced if greater1_flag and greater2_flag are not encoded.

[0210] If cParam after processing on the immediately preceding block of coefficients is greater than 0 (in other words, if there is at least one coefficient greater than the threshold value), then the image encoding apparatus according to the present embodiment expects that there is a high probability getting coefficients that have absolute value in the target coefficient block, and therefore does not encode greater1_flag and greater2_flag. As a result, processing can be performed at a higher speed by reducing arithmetic encoding steps using contexts while suppressing degradation in encoding efficiency.

[0211] In addition, greater1_flag and greater2_flag are not encoded when cParam is greater than 0. As a result, the set of contexts that is used when cParam is greater than 0, in other words, the set of contexts that is used when there is a high probability of providing coefficients of great importance is not necessary. More specifically, if the coefficient target block is located in the upper left corner of the encoding target block, the image encoding device uses one context. If the target coefficient block is not in the upper left corner, the image encoder uses one common set of context. As a result, it is possible to reduce the number of contexts used and thereby reduce the memory size for storing contexts and the size of the circuit for selecting contexts.

[0212] By combining information to be used in determining whether greater1_flag and greater2_flag should be encoded and information to be used in encoding the remainder, the size of the circuit is reduced. More specifically, cParam, which is incremented each time the absolute value of the target coefficient exceeds a threshold, is used to change the remainder binarization parameter and to determine whether greater1_flag and greater2_flag should be encoded. As a result, one structure can perform different functions. Compared with the existing HEVC standard (Non-Patent Document 1), the image encoding device according to the present embodiment can determine whether greater1_flag and greater2_flag should be encoded without including an additional structure.

[0213] In addition, the image encoding method according to the present embodiment was applied to test software in accordance with the present HEVC standard (Non-Patent Document 1) for experiments. Fig. 37 shows the experimental results compared with those without using the present embodiment. The experimental conditions are based on the general experimental conditions of the HEVC standards group. Numerical values in Fig. 37 are the results relative to the first 49 frames of the test image. A higher numerical value indicates lower coding efficiency. A negative value indicates an improvement in encoding efficiency. As shown in FIG. 37, each of the values ranges from -0.00% to 0.06%. This means that encoding efficiency is largely unaffected, even if processing speed is increased by eliminating arithmetic encoding that uses contexts, or even if the total number of contexts is decreased.

[0214] It should be noted that it was described above that the image encoding device determines, based on the binarization parameter (cParam) that was updated for the immediately preceding coefficient block, whether greater1_flag and greater2_flag should be encoded. However, the determination may be made based on a binarization parameter that has been updated for a coefficient block other than the immediately preceding coefficient block. For example, an image encoder may determine that greater1_flag and greater2_flag should not be encoded for subsequent blocks of coefficients if cParam exceeds 0 in processing for any processed block of coefficients. Blocks of coefficients are processed sequentially from a block of higher frequency. Thus, if the cParam of any of the coefficient blocks is greater than 0, in other words, if a coefficient is found that has an absolute value that is greater than or equal to the threshold value, then there is a high probability that all subsequent coefficient blocks have larger coefficient values. In the above case, even if greater1_flag and greater2_flag are not encoded, the encoding efficiency is sometimes not reduced.

[0215] It should also be noted that it was described above that the image encoding device does not encode greater1_flag and greater2_flag if cParam is greater than 0, but the present embodiment is not limited to the above. For example, it is also possible to not encode greater1_flag and greater2_flag if cParam is greater than 1 or greater than 2. Depending on the input image, encoding efficiency can sometimes be improved if the determination of whether greater1_flag and greater2_flag should be encoded is made based on whether cParam is greater than 1.

[0216] The rest is as described in Embodiment 1, so it will not be explained again here.

[0217] Embodiment 4

Embodiment 4 describes an image decoding apparatus that decodes the encoded bit stream generated by the image encoding apparatus of Embodiment 3. The image decoding apparatus of Embodiment 4 differs from the image decoding apparatus of Embodiment 2 in that the layer decoding unit 232 replaced by level decoding module 232A. Below is a description of the level decoding unit 232A.

[0218] Structure of level decoding module 232A

Fig. 38 is a block diagram showing the structure of the level decoding unit 232A. The level decoding unit 232A includes not only the structure of the level decoding unit 232 according to Embodiment 2, but also the greater_flag switching unit 260. In addition, the functions of the context set selection unit 241A and the arithmetic decoding unit 247A are different from the corresponding functions of the context set selection unit 241 and the arithmetic decoding unit 247.

[0219] Processing (Level Decoding)

Next, with reference to FIG. 39, level decoding will be described in more detail. The layer decoding according to Embodiment 2 is supplemented with new steps S601 to S603. In addition, steps S341A and S371A are different from steps S341 and S371, respectively. It should be noted that steps identical to those in Embodiment 2 are not further explained again.

[0220] Step S341A

The context set selector 241A sets (selects) the respective numbers of context sets to be used in the arithmetic decoding over greater1_flag and the arithmetic decoding over greater2_flag. The selection method is the same as for context set selection (FIG. 36) according to Embodiment 3. In other words, the context set selection unit 241A does not select a context based on cParam.

[0221] Step S601

The greater_flag 260 switch module 260 determines whether the binarization parameter (cParam) that was updated in processing on the immediately preceding coefficient block is greater than 0. In the same manner as described in Embodiment 2, the greater_flag switch module 260 decodes the greater1_flag and greater2_flag if the binarization parameter is 0, but does not decode greater1_flag and greater2_flag if the binarization parameter is greater than 0.

[0222] Steps S602, S603 and S371A

If the binarization parameter (cParam) that was updated for the immediately preceding coefficient block is greater than 0, then the arithmetic decoding unit 247A sets the base value of the remainder to 1. If the binarization parameter (cParam) is 0, and the remainder setting unit 247A sets the base value the remainder is 3. If the base value of the remainder is 3 and the greater2_flag of the target coefficient is 1, or if the base value of the remainder is 1 and the significant_flag of the target coefficient is 1, then the level decoding unit 232A decodes the remainder in steps S372 and S373. Otherwise, the layer decoding unit 232A does not decode the remainder. If the remainder is decoded, the numerical value obtained by adding the base residual value to the decoded remainder is the absolute value of the target coefficient.

[0223] Effects

As described above, the image decoding device according to the present embodiment can provide the same effects as described in Embodiment 3.

[0224] In each of the above-described embodiments, each of the functional blocks may be implemented typically as an MPU (microprocessor), memory, or the like. In addition, the processing performed by each of the functional blocks may be implemented generally by software (program), and the software is recorded on a recording medium such as a read-only memory (ROM). Such software may then be distributed, for example, by downloading, or may be recorded on a recording medium such as a compact disc read-only (CD-ROM) for subsequent distribution. In addition, each of the functional blocks can, of course, be implemented in hardware (specialized circuitry).

[0225] The processing described in each of the embodiments may be performed as centralized processing using a single device (system) or as decentralized processing using multiple devices. In addition, the above program can be executed by one computer or multiple computers. In other words, both centralized processing and decentralized processing can be executed by program.

[0226] Although the image encoding device and the image decoding device have been described with reference to many embodiments as stated above, the present invention is not limited to these embodiments.

[0227] It should also be noted that the processing modules in each of the image encoding device and the image decoding device according to the above-described embodiments are generally implemented on a high degree of integration (LSI) integrated circuit. They may be integrated separately, or some or all of them may be combined on a single chip.

[0228] It should also be noted that the integrated circuit technique is not limited to LSI, and it may be implemented as an application-specific circuit or a general-purpose processor. It is also possible to use a field programmable gate array (FPGA), which can be programmed after the LSI is manufactured, or a reconfigurable processor, in which the interconnection and configuration of the circuit cells within the LSI can be reconfigured.

[0229] Each of the building blocks in each of the above-described embodiments may be configured in the form of a dedicated hardware product or may be implemented by executing software suitable for the building block. Each of the structural elements may be implemented by a program execution unit such as a CPU and a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0230] In other words, each of the image encoding device and the image decoding device includes: a control circuit and storage means electrically connected to the control circuit and accessible from the control circuit. The control circuit contains at least one of specialized hardware and a program execution module. The storage means stores a software program executed by the program execution unit when the control circuit includes the program execution unit.

[0231] Moreover, the present invention may be the above-described program or may be a non-transitory computer-readable recording medium on which the program is recorded. Of course, the program may be distributed over a transmission medium such as the Internet.

[0232] In addition, all numerical expressions in the above description are examples to explain the present invention in more detail. The present invention is not limited to such numerical examples.

[0233] In addition, the division of functional blocks in the block diagram is given as examples. It is also possible that multiple function blocks are implemented as a single function block, that one function block is divided into multiple parts, or that a partial function is moved to another function block. In addition, the functions of multiple functional blocks having similar functions may be performed in parallel or time-shared by a common hardware or software.

[0234] It should also be noted that the execution order of the steps included in each of the above-described image encoding method and the above-described image decoding method is an example for explaining the present invention in more detail. Thus, various orders other than the above described order may be used. Some of the stages can be executed simultaneously (in parallel) with another stage.

[0235] Thus, although only some exemplary embodiments of the image encoding device and the image decoding device according to the present invention have been described in detail above, the present invention is not limited to these embodiments. Those skilled in the art will readily appreciate that various modifications of exemplary embodiments and combinations of structural elements of various embodiments are possible without significantly deviating from the teachings and advantages of the present invention. Accordingly, all such modifications and combinations are intended to be included within the scope of the present invention.

[0236] Embodiment 5

The processing described in each of the embodiments can be simply implemented in an independent computer system by recording, on a recording medium, a program for implementing the configurations of a moving image encoding method (image encoding method) and a moving image decoding method (image decoding method) described in each of the embodiments of the invention. The recording medium may be any recording medium as long as the program can be recorded, such as a magnetic disk, an optical disk, a magnetic-optical disk, an IC card, and a semiconductor memory.

[0237] Next, applications of the moving image encoding method (image encoding method) and the moving image decoding method (image decoding method) described in each of the embodiments and systems using them will be described. The system has the feature of having an image encoding and decoding device, which includes an image encoding device using an image encoding method and an image decoding device using an image decoding method. Other configurations in the system can be changed as needed depending on the cases.

[0238] FIG. 40 illustrates a general configuration of a content delivery system ex100 for implementing content distribution services. The communication service area is divided into cells of desired size, and base stations ex106, ex107, ex108, ex109 and ex110, which are fixed wireless stations, are located in each of the cells.

[0239] The content delivery system ex100 is connected to devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular phone ex114, and a gaming machine ex115, via the Internet ex101, an Internet service provider ex102, a telephone network ex104, and also base stations ex106 - ex110, respectively.

[0240] However, the configuration of the content provision system ex100 is not limited to the configuration shown in FIG. 40, and a combination in which any of the elements are connected is acceptable. In addition, each device can be directly connected to the ex104 telephone network, rather than through ex106 - ex110 base stations, which are fixed wireless stations. Additionally, these devices can be connected to each other via short range wireless communication, etc.

[0241] A camera ex113, such as a digital video camera, is capable of capturing video. The EX116 camera, for example, a digital camera, is capable of capturing both still images and video. In addition, the cellular telephone ex114 may be one that satisfies any of the standards such as Global System for Mobile Communications (GSM) (registered trademark), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA). ), Long Term Evolution (LTE), and High Speed Packet Access (HSPA). In addition, the cellular telephone ex114 may be a personal mobile phone system (PHS).

[0242] In the content delivery system ex100, the streaming server ex103 is connected to the camera ex113 and other elements via the telephone network ex104 and the base station ex109, allowing distribution of live images and the like. With such distribution, content (eg, video of a live music show) captured by a user using camera ex113 is encoded as described above in each of the embodiments (i.e., the camera functions as an image encoder in accordance with an aspect of the present invention), and the encoded content is transmitted to the streaming server ex103. On the other hand, the streaming server ex103 distributes the stream of transmitted content data to clients upon their requests. Clients include a computer ex111, a PDA ex112, a camera ex113, a cellular phone ex114, and a gaming machine ex115, which are capable of decoding the above encoded data. Each of the devices that have received the distributed data decodes the data and reproduces the encoded data (ie, functions as an image decoding device in accordance with one aspect of the present invention).

[0243] The captured data may be encoded by the camera ex113 or the streaming server ex103 that transmits the data, or the encoding processes may be shared between the camera ex113 and the streaming server ex103. In addition, the distributed data may be decoded by clients or the streaming server ex103, or the decoding processes may be shared between the clients and the streaming server ex103. In addition, still image and video data captured not only by the ex113 but also by the ex116 can be transferred to the ex103 streaming server via the ex111 computer. Encoding processes can be performed or shared between the ex116 camera, the ex111 computer, or the ex103 streaming server.

[0244] In addition, encoding and decoding processes may be performed by the LSI ex500 typically included in each of the ex111 computer and devices. The ex500 LSI can be configured from a single chip or multiple chips. The video encoding and decoding software may be integrated into some type of recording medium (eg, CD-ROM, floppy disk, and hard disk) that can be read by a computer ex111 and the like, and the encoding and decoding processes may be performed using software. In addition, when the cell phone ex114 is equipped with a camera, video data obtained by the camera can be transmitted. The video data is data encoded by the ex500 LSI included in the ex114 cell phone.

[0245] In addition, the streaming server ex103 may be composed of servers and computers and may perform data decentralization and process decentralized data, record or distribute data.

[0246] As described above, clients can receive and reproduce encoded data in the content delivery system ex100. In other words, clients can receive and decode information transmitted by the user, and play back the decoded data in real time in the content delivery system ex100, so that a user who does not have any specific right and equipment can realize personal broadcasting.

[0247] In addition to the example of a content provision system ex100, at least one of a moving image encoding device (image encoder) and a moving image decoding device (image decoder) described in each of the embodiments may be implemented in a digital broadcast system ex200 , shown in Fig. 41. More specifically, the broadcast station ex201 communicates or transmits via radio waves to the broadcast satellite ex202 multiplexed data obtained by multiplexing audio data and the like. to video data. The video data is data encoded by the moving image encoding method described in each of the embodiments (ie, data encoded by an image encoding apparatus according to one aspect of the present invention). After receiving the multiplexed data, the broadcast satellite ex202 transmits radio waves for broadcasting. Then, the EX204 home use antenna with satellite broadcast reception function receives radio waves. Next, a device such as a television set (receiver) ex300 and a set-top box (STB) ex217 decodes the received multiplexed data and reproduces the decoded data (ie, functions as a picture decoding device in accordance with one aspect of the present invention).

[0248] In addition, the reader/writer ex218(i) reads and decodes multiplexed data recorded on the recording medium ex215, such as DVD and BD, or (i) encodes video signals on the recording medium ex215, and in some cases writes the data, obtained by multiplexing an audio signal onto encoded data. The reader/writer ex218 may include a moving picture decoding device or a moving picture encoding device as shown in each embodiment of the invention. In this case, the reproduced video signals are displayed on the monitor ex219 and can be reproduced by another device or system using the recording medium ex215 on which the multiplexed data is recorded. In addition, it is possible to implement a moving picture decoding apparatus in a set-top box device ex217 connected to a cable ex203 for cable television or to an antenna ex204 for satellite and/or terrestrial broadcasting to display video signals on a monitor ex219 of a television ex300. The moving image decoding device can be implemented not in a set-top box, but in an EX300 TV.

[0249] FIG. 42 illustrates a television set ex300 that uses the moving picture encoding method and the moving picture decoding method described in each of the embodiments. The TV ex300 includes: a tuner ex301, which receives and provides multiplexed data obtained by multiplexing audio data on video data, through antenna ex204 or cable ex203, etc., which receive broadcast; a modulation/demodulation module ex302 that demodulates the received multiplexed data or modulates the data into multiplexed data to be provided externally; and a multiplexing/demultiplexing unit ex303 that demultiplexes the modulated multiplexed data into video data and audio data, or multiplexes video data and audio data encoded by the signal processing unit ex306 into data.

[0250] The television ex300 further includes: a signal processing module ex306 including an audio signal processing module ex304 and a video signal processing module ex305 for decoding audio data and video data and encoding audio data and video data, respectively (which function as an image encoding device and a decoding device images in accordance with aspects of the present invention); and an output module ex309 including a speaker ex307 that provides a decoded audio signal, and a display module ex308 that displays a decoded video signal such as a display. In addition, the television ex300 includes an interface module ex317 including an operation input module ex312 that receives user operation input. In addition, the television ex300 includes a control module ex310, which generally controls each of the component elements of the television ex300, and a power circuit module ex311, which supplies power to each of the elements. In addition to the operational input module ex312, the interface module ex317 may include: a bridge ex313 that is connected to an external device such as a read/write module ex218; an EX314 socket module for securing a recording medium such as an SD card; an ex315 driver to be connected to an external recording medium such as a hard disk; and an ex316 modem to be connected to the telephone network. Here, the recording medium ex216 can electrically record information using a non-volatile/volatile storage memory element. The components of the EX300 TV are connected to each other via a synchronous bus.

[0251] First, a configuration in which the TV ex300 decodes multiplexed data received externally through antenna ex204 and the like and reproduces the decoded data will be described. In the TV ex300, when the user operates through the remote controller ex220 and the like, the multiplexing/demultiplexing unit ex303 demultiplexes the multiplexed data demodulated by the modulation/demodulation unit ex302 under the control of the control unit ex310 including the CPU. In addition, the audio signal processing unit ex304 decodes the demultiplexed audio data, and the video signal processing unit ex305 decodes the demultiplexed video data using the decoding method described in each of the embodiments in the television ex300. The output module ex309 outputs the decoded video signal and audio signal externally, respectively. When the output unit ex309 outputs a video signal and an audio signal, the signals can be temporarily stored in buffers ex318 and ex319 and others, so that the signals are reproduced in sync with each other. In addition, the TV ex300 can read multiplexed data not through broadcasting or the like, but from recording media ex215 and ex216 such as magnetic disk, optical disk and SD card. Next, a configuration in which the TV ex300 encodes an audio signal and a video signal and transmits the data externally or records the data on a recording medium will be described. In the TV ex300, when operated by the user through the remote controller ex220 and the like, the audio signal processing unit ex304 encodes the audio signal, and the video signal processing unit ex305 encodes the video signal, under the control of the control unit ex310 using the encoding method described in each of the embodiments. The multiplexing/demultiplexing module ex303 multiplexes the encoded video signal and audio signal and provides the resulting signal to the outside. When the multiplexing/demultiplexing unit ex303 multiplexes a video signal and an audio signal, the signals can be temporarily stored in buffers ex320 and ex321 and others, so that the signals are reproduced synchronously with each other. Here, buffers ex318, ex319, ex320 and ex321 may be multiple as shown, or at least one buffer may be shared in the TV ex300. In addition, data can be stored in a buffer so that system overload or underload can be avoided, for example, between the modulation/demodulation unit ex302 and the multiplexing/demultiplexing unit ex303.

[0252] In addition, the television ex300 may include a configuration for receiving AV input from a microphone or camera that is different from a configuration for receiving audio and video data from a broadcast or recording medium, and may encode the received data. Although the ex300 TV can encode, multiplex and externalize data as described, it may only be capable of receiving, decoding and externalizing data, but not encoding, multiplexing and externalizing data.

[0253] In addition, when the read/write module ex218 reads or writes multiplexed data from or to a recording medium, one of the TV set ex300 and the read/write module ex218 can decode or encode the multiplexed data, and the TV set ex300 and the read/write module ex218 can share decoding or encoding.

[0254] As an example, FIG. 43 illustrates the configuration of the information reproducing/recording unit ex400 when data is read or written from or to an optical disc. The information reproducing/recording module ex400 includes components ex401, ex402, ex403, ex404, ex405, ex406 and ex407, which will be described below. The optical head ex401 irradiates a laser spot on the recording surface of the recording medium ex215, which is an optical disc for recording information, and detects reflected light from the recording surface of the recording medium ex215 to read the information. The modulation recording module ex402 electrically drives a semiconductor laser included in the optical head ex401 and modulates the laser light in accordance with the recorded data. The reproduction demodulation unit ex403 amplifies the reproduction signal obtained by electrically detecting reflected light from the recording surface using a photodetector included in the optical head ex401, and demodulates the reproduction signal by separating the signal component recorded on the recording medium ex215 to reproduce the necessary information. Buffer ex404 temporarily stores information to be recorded on recording medium ex215 and information reproduced from recording medium ex215. The disc motor ex405 rotates the recording medium ex215. The servo control module ex406 moves the optical head ex401 to a predetermined information track while controlling the rotation drive of the disk motor ex405 to follow the laser spot. The system control module ex407 performs general control of the information playback/recording module ex400. Reading and writing processes can be implemented by the system control module ex407, which uses various information stored in the buffer ex404 and generates and adds new information as needed, as well as by the modulation recording module ex402, playback demodulation module ex403 and servo control module ex406, which record and reproduce information through the optical head ex401 during operation in a coordinated manner. The system control unit ex407 includes, for example, a microprocessor, and executes processing causing a computer to execute a program for reading and writing.

[0255] Although the optical head ex401 irradiates the laser spot as described, it can perform high-density recording using near-field light.

[0256] FIG. 44 illustrates recording medium ex215, which is an optical disc. Guide grooves are helically formed on the recording surface of the recording medium ex215, and the information track ex230 records in advance address information indicating an absolute position on the disk in accordance with the change in the shape of the guide grooves. The address information includes information for determining positions of recording blocks ex231, which are a unit for recording data. Playing back the information track ex230 and reading address information in a device that records and plays back data can result in the positions of recording blocks being determined. In addition, the recording medium ex215 includes a data recording area ex233, an inner peripheral area ex232, and an outer peripheral area ex234. The data recording area ex233 is an area for use in recording user data. The inner peripheral area ex232 and the outer peripheral area ex234, which are inside and outside the data recording area ex233, respectively, are intended for special uses other than user data recording. The information reproducing/recording unit 400 reads and writes encoded audio, encoded video data, or multiplexed data obtained by multiplexing encoded audio and video data from and onto the data recording area ex233 of the recording medium ex215.

[0257] Although an optical disc having a layer such as DVD and BD is described as an example in the specification, the optical disc is not limited to such, and may be an optical disc having a multi-layer structure and capable of being recorded on a portion rather than the entire surface. In addition, the optical disc may be structured for multi-dimensional recording/reproduction, for example, recording information using light colors of different wavelengths in the same part of the optical disc, and recording information using different layers at different angles.

[0258] In addition, a vehicle ex+210 having an antenna ex205 can receive data from a satellite ex202 and others, and display video on a display device, such as an in-vehicle navigation system ex+211 installed in a vehicle ex210, on a digital broadcast system ex200 . Here, the configuration of the car navigation system ex211 will be a configuration, for example, including a GPS receiving module from the configuration shown in FIG. 42. The same will be true for the configuration of the ex111 computer, ex114 cell phone, etc.

[0259] FIG. 45A illustrates a cellular telephone ex114 that uses the moving image encoding method and the moving image decoding method described in the embodiments. The ex114 cell phone includes: an ex350 antenna for transmitting and receiving radio waves through the ex110 base station; EX365 camera module capable of capturing moving and still images; and a display module ex358, such as a liquid crystal display for displaying data such as decoded video captured by the camera module ex365 or received by the antenna ex350. The cellular phone ex114 further includes: a main body including an operation key module ex366; an audio output module ex357, such as a speaker for audio output; an audio input module ex356, such as a microphone for audio input; EX367 memory module for storing captured video or photographs, recorded audio, encoded or decoded data of received video, photographs, email or other; and an ex364 socket module, which is an interface module for a recording medium that stores data in the same manner as an ex367 memory module.

[0260] Next, a configuration example of cellular phone ex114 will be described with reference to FIG. 45B. In the cellular phone ex114, the main control module ex360, designed for overall control of each main body module, including the display module ex358, as well as the operation key module ex366, is mutually connected through the synchronous bus module ex370 to the power circuit module ex361, the operational input control module ex362, video processing module ex355, camera interface module ex363, liquid crystal display (LCD) control module ex359, modulation/demodulation module ex359, multiplexing/demultiplexing module ex353, audio processing module ex354, socket module ex364 and memory module ex367.

[0261] When the end call key or the power key is turned on by a user operation, the power management module ex361 supplies the corresponding modules with power from the battery to activate the cellular phone ex114.

[0262] In the cellular telephone ex114, the audio signal processing module ex354 of the ex354 unit converts audio signals received by the audio input module ex356 in voice dialogue mode into digital audio signals under the control of the main control module ex360, including the CPU, ROM and RAM. Next, the modulation/demodulation module ex352 performs spread spectrum processing on the digital audio signals, and the transmit and receive module ex351 performs digital-to-analog conversion and frequency conversion on the data so as to transmit the resulting data through the antenna ex350. In addition, in the cellular phone ex114, the transmission and reception module ex351 amplifies the data received by the antenna ex350 in voice dialogue mode, and performs frequency conversion and analog-to-digital conversion on the data. The modulation/demodulation module ex352 then performs inverse spread spectrum processing on the data, and the audio signal processing module ex354 converts it into analog audio signals to output them through the audio output module ex357.

[0263] In addition, when email is transmitted in the data transfer mode, email text data input by operating the operation key module ex366, etc. of the main body is sent to the main control module ex360 by the operation input control module ex362. The main control module ex360 causes the modulation/demodulation module ex352 to perform spread spectrum processing on the text data, and the transmit and receive module ex351 performs digital-to-analog conversion and frequency conversion on the resulting data to transmit the data to the base station ex110 via the antenna ex350. When email is received, processing, which is approximately the inverse of the processing for transmitting email, is performed on the received data, and the resulting data is provided to the display unit ex358.

[0264] When video, still images, or video and audio are transmitted in data mode, the video signal processing module ex355 compresses and encodes video signals supplied from the camera module ex365 using the moving image encoding method shown in each of the embodiments (i.e. i.e., functions as an image encoding device in accordance with an aspect of the present invention), and transmits encoded video data to the multiplexing/demultiplexing module ex353. In contrast, when the camera module ex365 captures video, still images and others, the audio signal processing module ex354 encodes the audio signals received by the audio input module ex356 and transmits the encoded audio data to the multiplexing/demultiplexing module ex353.

[0265] The multiplexing/demultiplexing unit ex353 multiplexes encoded video data transmitted from the video signal processing unit ex355 and encoded audio data transmitted from the audio signal processing unit ex354 using a predetermined method. Then, the modulation/demodulation unit ex352 (modulation/demodulation circuit unit) performs spreading processing on the multiplexed data, and the transmitting and receiving unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resultant data through the antenna ex350.

[0266] When receiving data from a video file that is associated with a web page, etc. in data mode, or when receiving an email message with a video and/or audio attachment, in order to decode the multiplexed data received through the antenna ex350, the module The multiplexing/demultiplexing ex353 demultiplexes the multiplexed data into a video bitstream and an audio bitstream, and supplies the video processing unit ex355 with the encoded video data and the audio processing unit ex354 with the encoded audio data via the synchronous bus ex370. The video signal processing unit ex355 decodes the video signal using a moving image decoding method corresponding to the moving image encoding method shown in each of the embodiments (i.e., functions as an image decoding apparatus according to an aspect of the present invention), and then the display unit ex358 displays for example, videos and still images included in a video file associated with a web page via the LCD control module ex359. In addition, the audio signal processing unit ex354 decodes the audio signal, and the audio output unit ex3578 outputs audio.

[0267] In addition, like the TV ex300, a terminal such as a cellular phone ex114 is likely to have 3 types of implementation configurations, including not only (i) a transmitting and receiving terminal, including both an encoding device and a decoding device , but also (ii) a transmitting terminal including only an encoding device, and (iii) a receiving terminal including only a decoding device. Although the digital broadcast system ex200 in the description receives and transmits multiplexed data obtained by multiplexing audio data onto video data, the multiplexed data may be data obtained by multiplexing not audio data but video-related character data onto video data, and may be non-multiplexed data. but by the video data itself.

[0268] Thus, the moving image encoding method and the moving image decoding method in each of the embodiments can be used in any of the described devices and systems. Thus, the advantages described in each of the embodiments can be obtained.

[0269] In addition, various modifications and changes may be made to any of the embodiments in the present invention.

[0270] Embodiment 6

Video data may be generated by switching, as necessary, between (i) a moving image encoding method or a moving image encoding apparatus shown in each of the embodiments, and (ii) a moving image encoding method or a moving image encoding apparatus in accordance with another standard, such as MPEG-2, MPEG-4 AVC and VC-1.

[0271] Here, when a plurality of video data that conforms to different standards is generated and then decoded, decoding methods must be selected to conform to the different standards. However, since it is impossible to determine the standard to which each of the plurality of video data to be decoded corresponds, there is a problem that a suitable decoding method cannot be selected.

[0272] In order to solve this problem, multiplexed data obtained by multiplexing audio data etc. onto video data has a structure including identification information indicating which standard the video data conforms to. The specific structure of multiplexed data including video data generated in the moving picture encoding method and the moving picture encoding apparatus shown in each of the embodiments will be described below. The multiplexed data is a digital stream in MPEG-2 transport stream format.

[0273] FIG. 46 illustrates the structure of multiplexed data. As shown in FIG. 46, multiplexed data may be obtained by multiplexing at least one of a video stream, an audio stream, a presentation graphics (PG) stream, and an interactive graphics stream. The video stream represents the primary video and secondary video from the movie, the audio stream (IG) represents the primary audio portion and the secondary audio portion that is to be mixed with the primary audio portion, and the presentation graphics stream represents the subtitles of the movie. Here, the primary video is a normal video to be displayed on the screen, and the secondary video is a video to be displayed in a smaller window in the primary video. Additionally, an interactive graphics stream is an interactive screen that will be generated by placing graphical user interface (GUI) components on the screen. The video stream is encoded in a moving image encoding method or a moving image encoding apparatus shown in each of the embodiments, or in a moving image encoding method or a moving image encoding apparatus in accordance with a conventional standard such as MPEG-2, MPEG-4 AVC and VC- 1. The audio stream is encoded in accordance with standards such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD and linear PCM.

[0274] Each stream included in the multiplexed data is identified by a PID. For example, 0x1011 is allocated to a video stream to be used for movie video, 0x1100 to 0x111F are allocated to an audio stream, 0x1200 to 0x121F are allocated to presentation graphics streams, 0x1400 to 0x141F are allocated to interactive graphics streams, 0x1B00 to 0x1B1F are allocated to video streams to be used for lying use for secondary movie video, and 0x1A00 to 0x1A1F are allocated for audio streams to be used for secondary audio to be mixed with the primary audio.

[0275] FIG. 47 schematically illustrates how data is multiplexed. First, the video stream ex235 consisting of video frames and the audio stream ex238 consisting of audio frames are converted into a PES packet stream ex236 and a PES packet stream ex239 and further into TS packets ex237 and TS packets ex240, respectively. In addition, the presentation graphics stream data ex241 and the interactive graphics stream data ex244 are converted into a PES packet stream ex242 and a PES packet stream ex245 and further into TS packets ex243 and TS packets ex246, respectively. These TS packets are multiplexed into a stream to obtain multiplexed ex247 data.

[0276] FIG. 48 shows in more detail how the video stream is stored in the PES packet stream. The first line in Fig. 48 shows a stream of video frames in a video stream. The second line shows the flow of PES packets. As shown by the arrows labeled yy1, yy2, yy3 and yy4 in FIG. 48, the video stream is divided into images (pictures) as I-pictures, B-pictures and P-pictures, each of which is a video presentation unit, and the pictures are stored in the payload of each of the PES streams. Each of the PES packets has a PES header, and the PES header stores a presentation timestamp (PTS) indicating the time the picture is displayed, and a decoding timestamp (DTS) indicating the time the picture is decoded.

[0277] FIG. 49 illustrates the format of TS packets to be finally written to multiplexed data. Each of the TS packets is a 188-byte fixed-length packet, including a 4-byte TS header containing information such as a PID for stream identification, and a 184-byte TS payload for data storage. PES packets are divided and stored in TS payloads, respectively. When using a BD ROM, each of the TS packets is assigned a 4-byte TP_Extra_Header, resulting in 192-byte original packets. The original packets are recorded on the multiplexed data. TP_Extra_Header stores information such as Arrival_Time_Stamp (ATS). The ATS indicates the initial transmission time at which each of the TS packets must be transmitted to the PID filter. The original packets are arranged into multiplexed data, as shown at the bottom of FIG. 49. The numbers increasing from the beginning of the multiplexed data are called source packet numbers (SPN).

[0278] Each of the TS packets included in the multiplexed data includes not only audio, video, subtitle and other streams, but also a table listing the stream's programs with their identifiers (PAT), a program structure table (PMT) and a field reference clock (PCR). The PAT indicates which PID in the PMT is used in the multiplexed data, and the PID of the PAT itself is registered as zero. PMT stores the PID of video, audio, subtitle and other streams included in the multiplexed data, and attribute information of the streams corresponding to the PID. PMT also has various descriptors related to the multiplexed data. The descriptors have information, such as copy control information, indicating whether copying of the multiplexed data is permitted or not. The PCR stores STC time information corresponding to the ATS indicating when a PCR packet is transmitted to the decoder to achieve synchronization between the arrival time clock (ATC), which represents the time axis of the ATS, and the system time clock (STC), which represents the time axis PTS and DTS.

[0279] FIG. 50 illustrates the RMT data structure in more detail. The RMT header is located on top of the RMT. The RMT header describes the length of the data included in the RMT, and more. A plurality of descriptors related to the multiplexed data are located after the PMT header. Information such as copy control information is described in descriptors. Following the descriptors are a plurality of pieces of flow information related to the flows included in the multiplexed data. Each piece of stream information includes stream descriptors, each of which describes information such as a stream type to identify the stream's compression codec, a stream PID, and stream attribute information (such as frame rate or frame format). The number of stream descriptors is equal to the number of streams in the multiplexed data.

[0280] When multiplexed data is recorded on a recording medium and others, it is recorded together with multiplexed data information files.

[0281] Each of the multiplexed data information files is multiplexed data management information, as shown in FIG. 51. The multiplexed data information files are in one-to-one correspondence with the multiplexed data, and each of the files contains multiplexed data information, stream attribute information, and a record element map.

[0282] As shown in FIG. 51, the multiplexed data information includes system speed, playback start time, and playback end time. The system speed indicates the maximum bit rate at which the target system decoder, described next, passes the multiplexed data to the PID filter. The ATS intervals included in the multiplexed data are set to a value no higher than the system speed. The playback start time indicates the PTS in the video frame at the beginning of the multiplexed data. An interval of one frame is added to the PTS in the video frame at the end of the multiplexed data, and the PTS is set to the end time of playback.

[0283] As shown in FIG. 52, a portion of the attribute information is registered in the stream attribute information for each PID of each stream included in the multiplexed data. Each piece of attribute information has different information depending on whether the corresponding stream is a video stream, an audio stream, a presentation graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including what type of compression codec is used to compress the video stream, and the resolution, frame format, and frame rate of the portions of picture data that are included in the video stream. Each piece of audio stream attribute information carries information including what type of compression codec is used to compress the audio stream, how many channels are included in the audio stream, what language the audio stream supports, and how high the sampling rate is. The video stream attribute information and the audio stream attribute information are used to initialize the decoder before the player plays the information.

[0284] In the present embodiment, the multiplexed data to be used is a stream type included in the PMT. In addition, when multiplexed data is recorded on a recording medium, video stream attribute information included in the multiplexed data information is used. More specifically, the moving image encoding method or the moving image encoding device described in each of the embodiments includes a step or module for distributing unique information indicating video data generated by the image encoding method or the moving image encoding device in each of the embodiments, for the type of stream included in the PMT or video stream attribute information. With such a configuration, video data generated by the moving image encoding method or the moving image encoding apparatus described in each of the embodiments may be different from video data that conforms to another standard.

[0285] Additionally, FIG. 53 illustrates steps of a moving image decoding method according to the present embodiment. In step exS100, the stream type included in the PMT or the video stream attribute information included in the multiplexed data information is obtained from the multiplexed data. Next, in step exS101, it is determined whether the stream type or the video stream attribute information indicates that the multiplexed data is generated by the moving image encoding method or the moving image encoding apparatus in each of the embodiments. When it is determined that the stream type or video stream attribute information indicates that the multiplexed data is generated by the moving image encoding method or the moving image encoding apparatus in each of the embodiments, decoding is performed by the moving image decoding method in each of the embodiments in step exS102. In addition, if the stream type or video stream attribute information indicates compliance with conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1, in step exS103, decoding is performed by a moving picture decoding method in accordance with conventional standards.

[0286] Thus, allocating a new unique value to the stream type or video stream attribute information makes it possible to determine whether the image decoding method or the moving image decoding apparatus described in each of the embodiments can perform decoding. Even when multiplexed data corresponding to some other standard is input, an appropriate decoding method or device can be selected. Thus, it becomes possible to decode information without any errors. Moreover, the moving image encoding method or apparatus or the moving image decoding method or apparatus in the present embodiment can be used in the devices and systems described above.

[0287] Embodiment 7

Each of the moving image encoding method, the moving image encoding device, the moving image decoding method, and the moving image decoding device in each of the embodiments is generally implemented in the form of an integrated circuit or a highly integrated circuit (LSI). As an example of an LSI, FIG. 54 illustrates the configuration of LSI ex500, which is implemented in a single chip. The ex500 LSI includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508 and ex509, which will be described below, the elements being connected to each other via the ex510 bus. The power circuit module ex505 is activated by applying power to each of the elements when the power circuit module ex505 is turned on.

[0288] For example, when encoding is performed, LSI ex500 receives AV signal from microphone ex117, camera ex113 and others through AV I/O ex509 under control of control module ex501 including CPU ex502, memory controller ex503, stream controller ex504 and module ex512 excitation frequency control. The received AV signal is temporarily stored in an external memory ex511 such as SDRAM. Under the control of the control module ex501, the stored data is segmented into data pieces according to the processing volume and speed to be transmitted to the signal processing module ex507. The signal processing module ex507 then encodes the audio signal and/or video signal. Here, the encoding of the video signal is the encoding described in each of the embodiments. In addition, the signal processing unit ex507 sometimes multiplexes the encoded audio data and the encoded video data, and the input/output (I/O) of the stream ex506 outputs the multiplexed data to the outside. The output multiplexed data is transmitted to the base station ex107 or recorded on the recording medium ex215. If data sets are multiplexed, this data must be temporarily stored in buffer ex508 so that the data sets are synchronized with each other.

[0289] Although the memory ex511 is external to the ex500 LSI, it may be included in the ex500 LSI. The ex508 buffer is not limited to one buffer, but may consist of buffers. In addition, the ex500 LSI can be implemented on a single chip or multiple chips.

[0290] In addition, although the control unit ex501 includes a CPU ex502, a memory controller ex503, a flow controller ex504, a driving frequency control unit ex512, the configuration of the control unit ex501 is not limited to this. For example, the signal processing module ex507 may further include a CPU. Including another CPU in the ex507 signal processing module may improve processing speed. Additionally, as another example, CPU ex502 may serve as or be part of a signal processing module ex507 and, for example, may include an audio signal processing module. In such a case, the control unit ex501 includes a signal processing unit ex507, or the CPU ex502 includes a portion of the signal processing unit ex507.

[0291] The term used here is LSI, but it can also be defined as an integrated circuit (IC), LSI system, super-LSI or ultra-LSI depending on the degree of integration.

[0292] In addition, methods for achieving integration are not limited to LSI, and application-specific circuit or general purpose processor, etc. can also implement integration. A field programmable gate array (FPGA), which can be programmed after the LSI is manufactured, or a reconfigurable processor, which allows the interconnection or configuration of the LSI to be changed, can be used for the same purpose.

[0293] In the future, with advances in semiconductor technology, an entirely new technology may replace LSI. Functional blocks can be integrated using such technology. It is also possible that the present invention will be applied to biotechnology.

[0294] Embodiment 8

When video data generated by the moving picture encoding method or the moving picture encoding apparatus described in each of the embodiments is decoded, compared with when video data that conforms to conventional standards such as MPEG-2, MPEG-4 AVC and VC- 1, processing volume is likely to increase. Thus, the ex500 LSI needs to be set to a higher drive frequency than the ex502 CPU, which must be used when video data is decoded according to the conventional standard. However, when the driving frequency is set higher, there is a problem that power consumption increases.

[0295] In order to solve this problem, a moving picture decoding apparatus such as the TV ex300 and the LSI ex500 is configured to determine which standard the video data conforms to and switch between excitation frequencies in accordance with the determined standard. Fig. 55 illustrates the configuration of ex800 in the present embodiment. The driving frequency switching unit ex803 sets the driving frequency to a higher driving frequency when video data is generated by the moving image encoding method or the moving image encoding apparatus described in each of the embodiments. Next, the drive frequency switching unit ex803 instructs the decoding processing unit ex801, which executes the moving image decoding method described in each of the embodiments to decode video data. When the video data conforms to the conventional standard, the driving frequency switching unit ex803 sets the driving frequency to a lower driving frequency than that of the video data obtained by the moving image encoding method or the moving image encoding device described in each of the embodiments. Next, the driving frequency switching unit ex803 instructs the decoding processing unit ex802, which conforms to the conventional standard, to decode the video data.

[0296] More specifically, the drive frequency switching unit ex803 includes a CPU ex502 and a drive frequency control unit ex512 of FIG. 54. Here, each of the decoding processing unit ex801, which executes the moving image decoding method described in each of the embodiments, and the decoding processing unit ex802, which conforms to the conventional standard, corresponds to the signal processing unit ex507 of FIG. 54. CPU ex502 determines which standard the video data conforms to. Then, the drive frequency control unit ex512 determines the drive frequency based on the signal from CPU ex502. In addition, the signal processing module ex507 decodes video data based on the signal from the CPU ex502. For example, the identification information described in Embodiment 6 can be used to identify video data. The identification information is not limited to that described in Embodiment 6, but may be any information as long as the information indicates which standard the video data conforms to. For example, if which standard the video data conforms to can be determined based on an external signal to determine that the video data is used for a TV or a disc, etc., the determination can be made based on such an external signal. In addition, CPU ex502 selects a drive frequency based on, for example, a lookup table in which video data standards are associated with drive frequencies, as shown in FIG. 57. The driving frequency can be selected by storing a lookup table in the buffer ex508 and in the internal memory of the LSI and referring to the lookup table by the CPU ex502.

[0297] FIG. 56 illustrates steps for executing the method in the present embodiment. First, in step exS200, the signal processing unit ex507 obtains identification information from the multiplexed data. Next, in step exS201, CPU ex502 determines whether video data is generated by the encoding method and encoding apparatus described in each of the embodiments based on the identification information. If video data is generated by the encoding method and encoding apparatus described in each of the embodiments, in step exS202, CPU ex502 transmits a signal for setting the driving frequency to a higher driving frequency to the driving frequency control unit ex512. Then, the drive frequency control module ex512 sets the drive frequency to a higher drive frequency. On the other hand, when the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1, in step exS203, the CPU ex502 transmits a signal to set the driving frequency to a lower frequency to the frequency control unit ex512 excitement. Then, the driving frequency control unit ex512 sets the driving frequency to a lower driving frequency than in the case where video data is generated by the moving image encoding method and the moving image encoding apparatus described in each embodiment.

[0298] In addition, along with switching driving frequencies, the power saving effect can be improved by changing the voltage supplied to the ex500 LSI or a device including the ex500 LSI. For example, when the driving frequency is set lower, the voltage supplied to the ex500 LSI or a device including the ex500 LSI may be set to a lower voltage than when the driving frequency is set higher.

[0299] In addition, when the amount of processing for decoding is larger, the driving frequency can be set higher, and when the amount of processing for decoding is smaller, the driving frequency can be set lower as a method of setting the driving frequency. Therefore, the installation method is not limited to the above. For example, when the amount of processing for decoding video data according to MPEG-4 AVC is greater than the amount of processing for decoding video data generated by the moving image encoding method and the moving image encoding apparatus described in each of the embodiments, the driving frequency may be set in the order the reverse of the setup described above.

[0300] Moreover, the method for setting the driving frequency is not limited to the method for setting the driving frequency lower. For example, when the identification information indicates that video data is generated by the moving image encoding method and the moving image encoding device described in each of the embodiments, the voltage supplied to the LSI ex500 or the device including the LSI ex500 may be set higher. When the identification information indicates that the video data conforms to a generally accepted standard, such as MPEG-2, MPEG-4 AVC and VC-1, the voltage supplied to the ex500 LSI or a device including the ex500 LSI may be set lower. As another example, when the identification information indicates that the video data is generated by the moving image encoding method and the moving image encoding apparatus described in each of the embodiments, the operation of the CPU ex502 may not need to be suspended. When the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1, the operation of the CPU ex502 is likely to be suspended at a given time because the CPU ex502 has additional processing capacity. Even when the identification information indicates that the video data is generated by the moving image encoding method and the moving image encoding apparatus described in each of the embodiments, in the case where the CPU ex502 has additional processing capacity, the operation of the CPU ex502 may be suspended at a given time. . In such a case, the pause time is likely set to be less than in the case where the identification significant_flag information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG-4 AVC and VC-1.

[0301] Accordingly, the power saving effect can be improved by switching between driving frequencies in accordance with the standard to which the video data conforms. In addition, when the ex500 LSI or a device including the ex500 LSI is driven using a battery, the service life of the battery can be extended due to the power saving effect.

[0302] Embodiment 9

There are cases where a variety of video data that conform to different standards are supplied to devices and systems such as television and cell phone. In order to decode a variety of video data that conforms to different standards, the signal processing module ex507 of the ex500 LSI must comply with various standards. However, problems arise with increasing the degree of integration of the ex500 LSI and increasing the cost when individually using ex507 signal processing modules that meet the relevant standards.

[0303] In order to solve this problem, a configuration is proposed in which a decoding processing unit for implementing the moving image decoding method described in each of the embodiments, and a decoding processing unit that conforms to a conventional standard such as MPEG-2, MPEG-2 4 AVC and VC-1, partially shared. Ex900 in fig. 58A shows an example of such a configuration. For example, the moving picture decoding method described in each of the embodiments and the moving picture decoding method that corresponds to MPEG-4 AVC have partially common processing details such as entropy encoding, inverse quantization, deblocking filtering, and movement. The processing details for sharing may include the use of a decoding processing unit ex902 that corresponds to MPEG-4 AVC. In contrast, the dedicated decoding processing module ex901 may be used for other processing that is unique to an aspect of the present invention and not compliant with MPEG-4 AVC. Since an aspect of the present invention is characterized by entropy decoding, in particular, for example, a dedicated decoding processing unit ex901 is used for entropy decoding. Otherwise, the decoding processing module may be shared for one of inverse quantization, deblocking filtering, and motion compensation or all processing. A decoding processing unit for implementing the moving picture decoding method described in each of the embodiments may be jointly used for shared processing, and a dedicated decoding processing unit may be used for MPEG-4 AVC-unique processing.

[0304] Additionally, ex1000 in FIG. 58B shows another example that the processing is partially divided. This example uses a configuration including a dedicated decoding processing module ex1001 that supports processing unique to an aspect of the present invention, a dedicated decoding processing module ex1002 that supports processing unique to another conventional standard, and a decoding processing module ex1003 that supports processing shared between a moving picture decoding method according to an aspect of the present invention and a conventional moving picture decoding method. Here, the dedicated decoding processing units ex1001 and ex1002 are not necessarily specialized for processing in accordance with an aspect of the present invention and conventional standard processing, respectively, and may be any capable of implementing general processing. Moreover, the configuration of the present embodiment can be implemented by LSI ex500.

[0306] Thus, reducing the degree of LSI integration and reducing cost is possible by sharing a decoding processing unit for processing shared between a moving image decoding method according to an aspect of the present invention and a moving image decoding method according to a conventional standard.

[0306] Industrial applicability

The present invention can be applied to image encoding methods, image decoding methods, image encoding devices and image decoding devices. The present invention is also applicable to high-definition display or imaging devices such as televisions, digital video recorders, car navigation systems, portable telephones, digital cameras, each of which includes an image encoder.

List of reference items

[0307]

100 image encoder

101 coding block division modules

102 subtraction module

103 conversion module

104 variable word length encoding module

105, 202 inverse conversion module

106, 203 summation module

107, 205 frame memory

108 prediction module

121 input images

122 coding block

123, 126, 223 differential block

124, 222 frequency factor (ratio)

125, 221 coded streams

127, 224 decoded block

128, 225, 226 decoded image

129 prediction block

131 coefficient block division modules

132 coding module significant_flag

133, 133A level coding module

134 encoding module sign_flag

141 installation modules greater1_flag

142 installation module greater2_flag

143, 143A remainder installation module

144, 248 binarization parameter update module

145, 145A, 241, 241A context set selection module

146, 243 memory greater1_flag

147, 242 context selection module greater1_flag

148, 244 context memory greater1_flag

149, 245 context selection module greater2_flag

150, 246 context memory greater2_flag

151 arithmetic coding modules

152 remainder binarization module

153 prefix-suffix determination module

154 suffix bin output module

155 prefix bin output module

160, 260 switching module greater_flag

200 image decoding device

201 variable word length decoding modules

204 module for combining decoded blocks

231 significant_flag decoding modules

232, 232A level decoding module

233 sign_flag decoding module

234 coefficient decoding module

247, 247A arithmetic decoding module

249 modulus of multivalued remainder

250 prefix decoding module

251 suffix decoding modules

252 prefix-suffix combining module

Claims (37)

1. An image encoding method containing: determining whether the immediately preceding block of coefficients, which was encoded immediately before the target block of coefficients, includes a coefficient having an absolute value greater than a threshold value; selecting a first context to be used in arithmetic encoding of a first target coefficient flag in the target coefficient block, wherein selecting a context to be used in arithmetic encoding of the first flag is based at least in part on the result of determining whether the immediately preceding coefficient block includes a coefficient , having an absolute value greater than the threshold value; arithmetic encoding of the first flag using the first context; selecting a second context to be used in arithmetic encoding of a second target coefficient flag in the target coefficient block, wherein selecting a context to be used in arithmetic encoding of the second flag is based at least in part on the result of determining whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than said threshold value; And arithmetic encoding of the second flag using the second context; in this case the threshold value is 1. 2. The method of claim 1, wherein the first flag indicates whether the absolute value of the target coefficient in the target coefficient block is greater than 1. 3. The method of claim 1, wherein the second flag indicates whether the absolute value of the target coefficient in the target coefficient block is greater than 2. 4. The method of claim 1, wherein the determination of whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value is based at least in part on a variable generated after arithmetic encoding over the immediately preceding block of coefficients . 5. The method of claim 1, further comprising binarizing the remainder according to a lookup table designated by a binarization parameter from among the plurality of lookup tables when the absolute value of the target coefficient is greater than 2, the remainder being a value obtained by subtracting 3 from the absolute value. 6. The method of claim 5, wherein the binarization parameter is determined according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value that is greater than said threshold value, and wherein said variable is a binarization parameter. 7. An image encoding device comprising: at least one processor; at least one memory including computer program code, wherein the memory and computer program code are configured, operative with the processor, to cause the device to do at least the following: determining whether the immediately preceding block of coefficients, which was encoded immediately before the target block of coefficients, includes a coefficient having an absolute value greater than a threshold value; selecting a first context to be used in arithmetic encoding of a first target coefficient flag in a target coefficient block, wherein selecting a context to be used in arithmetic encoding of the first flag is based at least in part on the result of determining whether the immediately preceding coefficient block includes a coefficient , having an absolute value greater than the threshold value; arithmetic encoding the first flag using the first context; selecting a second context to be used in arithmetic encoding of a second target coefficient flag in the target coefficient block, wherein selecting a context to be used in arithmetic encoding of the second flag is based at least in part on the result of determining whether the immediately preceding coefficient block includes a coefficient , having an absolute value greater than the threshold value; And arithmetic encoding the second flag using the second context; in this case the threshold value is 1. 8. The apparatus of claim 7, wherein the first flag indicates whether the absolute value of the target coefficient in the target coefficient block is greater than 1. 9. The apparatus of claim 7, wherein the second flag indicates whether the absolute value of the target coefficient in the target coefficient block is greater than 2. 10. The apparatus of claim 7, wherein the determination of whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value is based at least in part on a variable generated after arithmetic encoding over the immediately preceding block of coefficients . 11. The apparatus of claim 7, wherein the memory includes computer program code configured to, operatively with the processor, cause the apparatus to binarize the remainder according to a lookup table designated by a binarization parameter from among the plurality of lookup tables when the absolute value of the target coefficient greater than 2, with the remainder being the value obtained by subtracting 3 from the absolute value. 12. The apparatus of claim 11, wherein the binarization parameter is determined according to whether the immediately preceding block of coefficients includes a coefficient having an absolute value that is greater than said threshold value, and wherein said variable is a binarization parameter. 13. A computer-readable durable medium on which instructions are encoded that, when executed by the processor, perform: determining whether the immediately preceding block of coefficients, which was encoded immediately before the target block of coefficients, includes a coefficient having an absolute value greater than a threshold value; selecting a first context to be used in arithmetic encoding of a first target coefficient flag in the target coefficient block, wherein selecting a context to be used in arithmetic encoding of the first flag is based at least in part on the result of determining whether the immediately preceding coefficient block includes a coefficient , having an absolute value greater than the threshold value; arithmetic encoding of the first flag using the first context; selecting a second context to be used in arithmetic encoding of a second target coefficient flag in the target coefficient block, wherein selecting a context to be used in arithmetic encoding of the second flag is based at least in part on the result of determining whether the immediately preceding coefficient block includes a coefficient having an absolute value greater than said threshold value; And arithmetic encoding of the second flag using the second context; in this case the threshold value is 1. 14. The medium of claim 13, wherein the first flag indicates whether the absolute value of the target coefficient in the target coefficient block is greater than 1. 15. The medium of claim 13, wherein the second flag indicates whether the absolute value of the target coefficient in the target coefficient block is greater than 2. 16. The medium of claim 13, wherein the determination of whether the immediately preceding block of coefficients includes a coefficient having an absolute value greater than a threshold value is based at least in part on a variable generated after arithmetic encoding over the immediately preceding block of coefficients . 17. The medium according to claim 13, on which instructions are additionally encoded that, when executed by the processor, perform binarization of the remainder according to a lookup table designated by the binarization parameter from among a plurality of lookup tables when the absolute value of the target coefficient is greater than 2, wherein the remainder is the value obtained by subtracting 3 from the absolute value.